我有一个表与以下字段:
id (Unique)
url (Unique)
title
company
site_id
现在，我需要删除具有相同标题、company和site_id的行。一种方法是使用下面的SQL和脚本(PHP):
SELECT title, site_id, location, id, count( * )
FROM jobs
GROUP BY site_id, company, title, location
HAVING count( * ) >1
运行此查询后，可以使用服务器端脚本删除重复项。
但是，我想知道这是否只能使用SQL查询。
当前回答
我有这个查询片段的SQLServer，但我认为它可以用在其他DBMS与小的变化:
DELETE
FROM Table
WHERE Table.idTable IN (
SELECT MAX(idTable)
FROM idTable
GROUP BY field1, field2, field3
HAVING COUNT(*) > 1)
我忘了告诉您，这个查询不会删除重复行中id最低的行。如果这对你有用，试试这个查询:
DELETE
FROM jobs
WHERE jobs.id IN (
SELECT MAX(id)
FROM jobs
GROUP BY site_id, company, title, location
HAVING COUNT(*) > 1)
其他回答
我必须对文本字段执行此操作，并且遇到了索引上100字节的限制。
我通过添加一个列来解决这个问题，对字段进行md5哈希，并进行更改。
ALTER TABLE table ADD `merged` VARCHAR( 40 ) NOT NULL ;
UPDATE TABLE SET merged` = MD5(CONCAT(`col1`, `col2`, `col3`))
ALTER IGNORE TABLE table ADD UNIQUE INDEX idx_name (`merged`);
我想更具体地说明我删除了哪些记录，下面是我的解决方案:
delete
from jobs c1
where not c1.location = 'Paris'
and c1.site_id > 64218
and exists
(
select * from jobs c2
where c2.site_id = c1.site_id
and c2.company = c1.company
and c2.location = c1.location
and c2.title = c1.title
and c2.site_id > 63412
and c2.site_id < 64219
)
一种简单易懂且不需要主键的解决方案:
add a new boolean column alter table mytable add tokeep boolean; add a constraint on the duplicated columns AND the new column alter table mytable add constraint preventdupe unique (mycol1, mycol2, tokeep); set the boolean column to true. This will succeed only on one of the duplicated rows because of the new constraint update ignore mytable set tokeep = true; delete rows that have not been marked as tokeep delete from mytable where tokeep is null; drop the added column alter table mytable drop tokeep;
我建议您保留您添加的约束，以便将来防止出现新的重复。
我找到了一个简单的方法。(保持最新的)
DELETE t1 FROM table_name t1 INNER JOIN table_name t2
WHERE t1.primary_id < t2.primary_id
AND t1.check_duplicate_col_1 = t2.check_duplicate_col_1
AND t1.check_duplicate_col_2 = t2.check_duplicate_col_2
...
这将删除标题、公司和网站值相同的重复行。最后一个重复项将被保留，其余重复项将被删除(如果你想保留第一个重复项并删除其他重复项，将id的比较项更改为大于t1。Id > t2.id)
DELETE t1 FROM tablename t1
INNER JOIN tablename t2
WHERE
t1.id < t2.id AND
t1.title = t2.title AND
t1.company=t2.company AND
t1.site_ID=t2.site_ID;
