根据多个条件从表中删除重复项并保留到其他表

Question

我有一个taccounts 表，其中包含account_id(PK)、login_name、password、last_login 等列。现在我必须根据新的业务逻辑删除一些重复的条目。 因此，重复帐户将使用相同的email 或相同（login_name 和password）。必须保留最新登录的帐户。

这是我的尝试（一些电子邮件值为 null 和空白）

DELETE
FROM taccounts
WHERE email is not null and char_length(trim(both ' ' from email))>0 and last_login NOT IN
(
SELECT MAX(last_login)
FROM taccounts
WHERE email is not null and char_length(trim(both ' ' from email))>0 
GROUP BY lower(trim(both ' ' from email)))

login_name 和 password 也是如此

DELETE
FROM taccounts
WHERE last_login NOT IN
(
SELECT MAX(last_login)
FROM taccounts
GROUP BY login_name, password)

有没有更好的方法或任何方法来组合这两个单独的查询？

还有一些其他表有account_id 作为外键。 如何更新这些表的更改？` 我正在使用 PostgreSQL 9.2.1

编辑：一些电子邮件值为空，其中一些为空白（''）。因此，如果两个帐户的登录名和密码不同，并且他们的电子邮件为空或空白，则必须将它们视为两个不同的帐户。

Answer 1

如果大部分行都被删除（大部分是重复的）并且表适合 RAM，请考虑以下路线：

1. SELECT 将幸存的行放入临时表中。
2. 将 FK 引用重新路由到幸存者
3. DELETE 所有 来自基表的行。
4. 回复INSERT幸存者。

1a。提取幸存的行

CREATE TEMP TABLE tmp AS
SELECT DISTINCT ON (login_name, password) *
FROM  (
   SELECT DISTINCT ON (email) *
   FROM   taccounts
   ORDER  BY email, last_login DESC
   ) sub
ORDER  BY login_name, password, last_login DESC;

关于DISTINCT ON：

• Select first row in each GROUP BY group?

要识别两个不同条件的重复项，请使用子查询依次应用这两个规则。第一步保留最新last_login的账号，所以这是“可序列化”的。

检查结果并测试其合理性。

SELECT * FROM tmp;

临时表会在会话结束时自动删除。在 pgAdmin（您似乎正在使用）中，只要编辑器窗口打开，会话就会存在。

1b。 “重复”的更新定义的替代查询

SELECT *
FROM   taccounts t
WHERE  NOT EXISTS (
   SELECT  FROM taccounts t1
   WHERE  ( NULLIF(t1.email, '') = t.email
        OR (NULLIF(t1.login_name, ''), NULLIF(t1.password, '')) = (t.login_name, t.password))
   AND   (t1.last_login, t1.account_id) > (t.last_login, t.account_id)
   );

这不会将 NULL 或空字符串 ('') 在任何“重复”列中视为相同。

行表达式(t1.last_login, t1.account_id) 处理了两个骗子可能共享同一个last_login 的可能性。在这种情况下，会选择具有较大 account_id 的那个 - 这是独一无二的，因为它是 PK。

2a。如何识别所有传入的 FK

SELECT c.confrelid::regclass::text AS referenced_table
     , c.conname AS fk_name
     , pg_get_constraintdef(c.oid) AS fk_definition
FROM   pg_attribute a 
JOIN   pg_constraint c ON (c.conrelid, c.conkey[1]) = (a.attrelid, a.attnum)
WHERE  c.confrelid = 'taccounts'::regclass   -- (schema-qualified) table name
AND    c.contype  = 'f'
ORDER  BY 1, contype DESC;

仅在外键的第一列上构建。更多信息：

• Find the referenced table name using table, field and schema name

或者在选择表taccounts后在pgAdmin的对象浏览器的右侧窗口中检查Dependents骑手。

2b。重新路由到新的主节点

如果您有引用taccounts 的表（传入 外键到 taccounts），您将需要更新所有这些字段，之前 你删除了骗子。
将它们全部重新路由到新的主行：

UPDATE referencing_tbl r
SET    referencing_column = tmp.reference_column
FROM   tmp
JOIN   taccounts t1 USING (email)
WHERE  r.referencing_column = t1.referencing_column
AND    referencing_column IS DISTINCT FROM tmp.reference_column;

UPDATE referencing_tbl r
SET    referencing_column = tmp.reference_column
FROM   tmp
JOIN   taccounts t2 USING (login_name, password)
WHERE  r.referencing_column = t1.referencing_column
AND    referencing_column IS DISTINCT FROM tmp.reference_column;

3。 & 4. 杀戮

现在，不再引用骗子了。去杀戮吧。

ALTER TABLE taccounts DISABLE TRIGGER ALL;
DELETE FROM taccounts;
VACUUM taccounts;
INSERT INTO taccounts
SELECT * FROM tmp;
ALTER TABLE taccounts ENABLE TRIGGER ALL;

在操作期间禁用所有触发器。这避免了在操作期间检查引用完整性。重新激活触发器后，一切都会好起来的。我们处理了上述所有传入 FK。 传出 FK 保证完好无损，因为您没有并发写入访问权限，并且所有值之前都存在。

Answer 2

除了 Erwin 的出色回答之外，在将旧键与新键相关联的中间链接表中创建通常很有用。

DROP SCHEMA tmp CASCADE;
CREATE SCHEMA tmp ;
SET search_path=tmp;
CREATE TABLE taccounts
        ( account_id SERIAL PRIMARY KEY
        , login_name varchar
        , email varchar
        , last_login TIMESTAMP
        );
    -- create some fake data
INSERT INTO taccounts(last_login)
SELECT gs FROM generate_series('2013-03-30 14:00:00' ,'2013-03-30 15:00:00' , '1min'::interval) gs
        ;
UPDATE taccounts
SET login_name = 'User_' || (account_id %10)::text
        , email = 'Joe' || (account_id %9)::text || '@somedomain.tld'
        ;

SELECT * FROM taccounts;

        --
        -- Create (temp) table linking old id <--> new id
        -- After inspection this table can be used as a source for the FK updates
        -- and for the final delete.
        --
CREATE TABLE update_ids AS
WITH pairs AS (
        SELECT one.account_id AS old_id
        , two.account_id AS new_id
        FROM taccounts one
        JOIN taccounts two ON two.last_login > one.last_login
                AND ( two.email = one.email OR two.login_name = one.login_name)
        )
SELECT old_id,new_id
FROM pairs pp
WHERE NOT EXISTS (
        SELECT * FROM pairs nx
        WHERE nx.old_id = pp.old_id
        AND nx.new_id > pp.new_id
        )
        ;

SELECT * FROM update_ids
        ;

UPDATE other_table_with_fk_to_taccounts dst
SET account_id. = ids.new_id
FROM update_ids ids
WHERE account_id. = ids.old_id
        ;
DELETE FROM taccounts del
WHERE EXISTS (
        SELECT * FROM update_ids ex
        WHERE ex.old_id = del.account_id
        );

SELECT * FROM taccounts;

实现相同目的的另一种方法是将带有指向首选键的指针的列添加到表本身，并将其用于更新和删除。

ALTER TABLE taccounts
        ADD COLUMN better_id INTEGER REFERENCES taccounts(account_id)
        ;

   -- find the *better* records for each record.
UPDATE taccounts dst
SET better_id = src.account_id
FROM taccounts src
WHERE src.login_name = dst.login_name
AND src.last_login > dst.last_login
AND src.email IS NOT NULL
AND NOT EXISTS (
        SELECT * FROM taccounts nx
        WHERE nx.login_name = dst.login_name
        AND nx.email IS NOT NULL
        AND nx.last_login > src.last_login
        );

    -- Find records that *do* have an email address
UPDATE taccounts dst
SET better_id = src.account_id
FROM taccounts src
WHERE src.login_name = dst.login_name
AND src.email IS NOT NULL
AND dst.email IS NULL
AND NOT EXISTS (
        SELECT * FROM taccounts nx
        WHERE nx.login_name = dst.login_name
        AND nx.email IS NOT NULL
        AND nx.last_login > src.last_login
        );

SELECT * FROM taccounts ORDER BY account_id;

UPDATE other_table_with_fk_to_taccounts dst
SET account_id = src.better_id
FROM update_ids src
WHERE dst.account_id = src.account_id
AND src.better_id IS NOT NULL
        ;

DELETE FROM taccounts del
WHERE EXISTS (
        SELECT * FROM taccounts ex
        WHERE ex.account_id = del.better_id
        );
SELECT * FROM taccounts ORDER BY account_id;