加快“NOT EXISTS”数据库查询答案

【问题标题】：Speed up "NOT EXISTS" database query加快“NOT EXISTS”数据库查询
【发布时间】：2020-06-03 13:48:40
【问题描述】：

我有以下由 SQLAlchemy 生成的 Postgres 查询。我们的目标是获得与评分作业相关的提交，这些提交属于我们的 3 门课程（=计划）的一部分，但仅限于那些尚未经过审核的提交。

SELECT
    submissions.* # edited for brevity
FROM
    submissions
JOIN
    assignments ON assignments.id = submissions.assignment_id
JOIN
    modules ON modules.id = assignments.module_id
WHERE
    assignments.team IN ('grading') AND
    modules.program_id IN (11, 106, 108) AND
    submissions.gets_review IS true AND
    NOT (
        EXISTS (
            SELECT 1
            FROM reviews
            WHERE submissions.id = reviews.submission_id
        )
    )
ORDER BY submissions.created_at asc
LIMIT 15
OFFSET 0

生成此查询计划：

                                                                            QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=16964.33..16966.08 rows=15 width=167) (actual time=173.618..173.623 rows=3 loops=1)
   ->  Gather Merge  (cost=16964.33..17015.43 rows=438 width=167) (actual time=173.616..173.672 rows=3 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Sort  (cost=15964.30..15964.85 rows=219 width=167) (actual time=166.425..166.426 rows=1 loops=3)
               Sort Key: submissions.created_at
               Sort Method: quicksort  Memory: 25kB
               Worker 0:  Sort Method: quicksort  Memory: 25kB
               Worker 1:  Sort Method: quicksort  Memory: 25kB
               ->  Nested Loop Anti Join  (cost=356.11..15958.93 rows=219 width=167) (actual time=166.358..166.402 rows=1 loops=3)
                     ->  Hash Join  (cost=355.68..8645.36 rows=7373 width=167) (actual time=12.691..152.481 rows=1545 loops=3)
                           Hash Cond: (submissions.assignment_id = assignments.id)
                           ->  Parallel Seq Scan on submissions  (cost=0.00..7976.62 rows=63809 width=167) (actual time=0.008..101.093 rows=50787 loops=3)
                                 Filter: (gets_review IS TRUE)
                                 Rows Removed by Filter: 93127
                           ->  Hash  (cost=346.16..346.16 rows=762 width=4) (actual time=12.585..12.585 rows=675 loops=3)
                                 Buckets: 1024  Batches: 1  Memory Usage: 32kB
                                 ->  Hash Join  (cost=44.69..346.16 rows=762 width=4) (actual time=0.318..12.372 rows=675 loops=3)
                                       Hash Cond: (assignments.module_id = modules.id)
                                       ->  Seq Scan on assignments  (cost=0.00..285.39 rows=6099 width=8) (actual time=0.010..4.978 rows=6099 loops=3)
                                             Filter: (team = 'grading'::teams)
                                             Rows Removed by Filter: 492
                                       ->  Hash  (cost=43.09..43.09 rows=128 width=4) (actual time=0.225..0.226 rows=128 loops=3)
                                             Buckets: 1024  Batches: 1  Memory Usage: 13kB
                                             ->  Seq Scan on modules  (cost=0.00..43.09 rows=128 width=4) (actual time=0.010..0.182 rows=128 loops=3)
                                                   Filter: (program_id = ANY ('{11,106,108}'::integer[]))
                                                   Rows Removed by Filter: 897
                     ->  Index Only Scan using ix_reviews_submission_id on reviews  (cost=0.42..1.00 rows=1 width=4) (actual time=0.008..0.008 rows=1 loops=4636)
                           Index Cond: (submission_id = submissions.id)
                           Heap Fetches: 4633
 Planning Time: 0.763 ms
 Execution Time: 173.760 ms

我可能过度优化，但 174 毫秒似乎太慢了。我们拥有所有基本索引以及我们手动创建的一对。

如何加快查询速度？

【问题讨论】：

"Heap Fetches: 4633" 第 1 步是清理您的表。如果这不起作用，请向我们展示新计划
在任何合理的硬件上，仅对提交的 seq 扫描似乎都太慢了（并且占用了您一半以上的时间）。你能不能打开track_io_timings然后给我们看explain (analyze, buffers) select count(*) from submissions where gets_review ;

标签： database postgresql performance join optimization

【解决方案1】：

您可以尝试将 NOT EXISTS 谓词移动到显式反连接中，看看它是否能为您提供更好的计划：

SELECT
    submissions.* # edited for brevity
FROM
    submissions
JOIN
    assignments ON assignments.id = submissions.assignment_id
JOIN
    modules ON modules.id = assignments.module_id
LEFT JOIN
    reviews ON submissions.id = reviews.submission_id
WHERE
    assignments.team IN ('grading') AND
    modules.program_id IN (11, 106, 108) AND
    submissions.gets_review IS true AND
    reviews.id IS NULL
ORDER BY submissions.created_at asc
LIMIT 15
OFFSET 0

【讨论】：

【解决方案2】：

尝试在提交表上创建部分索引。

CREATE INDEX ix_assignment_id_created_at ON submissions (assignment_id, created_at)
WHERE (gets_review IS true);

【讨论】：