【发布时间】:2019-11-11 15:37:06
【问题描述】:
我有一个包含航空公司时刻表数据的表,目前大约有 4.5 亿行。
我无法弄清楚为什么我的数据库上的以下查询需要这么长时间才能运行(几分钟)。如果我正确阅读了查询计划,它似乎会挂在一个针对完全索引列的特定相等子句上,所以我不确定我还能做些什么来解决这个问题。
其他查询的运行时间似乎没有那么长,因此它可能与查询本身有关,而不是与硬件/其他方面有关。任何帮助将不胜感激!
有问题的表:
Column | Type | Collation | Nullable | Default
-------------+-----------------------------+-----------+----------+---------
pubdate | date | | not null |
flowndate | date | | not null |
carcode | character varying(3) | | not null |
fltno | character varying(5) | | not null |
acfttype | character varying(4) | | not null |
svctype | character varying(5) | | |
traffrest | character varying(1) | | |
depstn | character varying(3) | | not null |
arrstn | character varying(3) | | not null |
depdatetime | timestamp without time zone | | not null |
deputcvar | smallint | | not null |
arrdatetime | timestamp without time zone | | not null |
arrutcvar | smallint | | not null |
seats | integer | | |
acftclass | character varying(3) | | |
Indexes:
"index_dailyflights_arrstn" btree (arrstn)
"index_dailyflights_depstn" btree (depstn)
"index_dailyflights_flowndate" btree (flowndate)
"index_dailyflights_pubdate" btree (pubdate DESC)
查询:
SELECT carCode, COUNT(*) AS departures
FROM truesight_data.oag_dailyflights df
WHERE flowndate <@ '[2016-01-01,2017-10-31]'::daterange
AND (flowndate <= ((DATE_TRUNC('MONTH', pubDate) + INTERVAL '1 MONTH - 1 day')::date) OR PubDate = (SELECT MAX(PubDate) FROM truesight_data.oag_dailyflights))
AND depStn = 'PIT'
AND (svcType IN ('cargo','combi') OR acftClass = 'WB')
GROUP BY carCode
ORDER BY COUNT(*) DESC;
还有这个查询的EXPLAIN (ANALYZE, BUFFERS):
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=140256.14..140256.18 rows=18 width=11) (actual time=22013.297..22013.297 rows=2 loops=1)
Sort Key: (count(*)) DESC
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=985 read=142487
I/O Timings: read=20089.741
-> GroupAggregate (cost=140255.45..140255.76 rows=18 width=11) (actual time=22013.281..22013.284 rows=2 loops=1)
Group Key: carcode
Buffers: shared hit=985 read=142487
I/O Timings: read=20089.741
-> Sort (cost=140255.45..140255.49 rows=18 width=3) (actual time=22013.262..22013.263 rows=27 loops=1)
Sort Key: carcode
Sort Method: quicksort Memory: 26kB
Buffers: shared hit=985 read=142487
I/O Timings: read=20089.741
-> Index Scan using index_dailyflights_depstn on oag_dailyflights df (cost=0.57..140255.07 rows=18 width=3) (actual time=13947.770..22013.193 rows=27 loops=1)
Index Cond: ((depstn)::text = 'PIT'::text)
Filter: ((flowndate <@ '[2016-01-01,2017-11-01)'::daterange) AND (((svctype)::text = ANY ('{cargo,combi}'::text[])) OR ((acftclass)::text = 'WB'::text)) AND current_status(df.*))
Rows Removed by Filter: 616867
Buffers: shared hit=985 read=142487
I/O Timings: read=20089.741
Planning Time: 0.460 ms
Execution Time: 22013.389 ms
(22 rows)
再次感谢您的帮助——很高兴回答您可能遇到的任何其他问题。
【问题讨论】:
-
能否请您添加使用
explain (analyze, buffers)生成的执行计划 -
谢谢——我更新了上面的查询计划以显示完整的
explain (analyze, buffers)输出。 -
Postgres 以大约 55 MB/秒的速度读取索引数据 - 这是一个不错的数据库服务器的较低范围,这限制了您的查询。那是什么类型的服务器/硬盘?
-
您的 EXPLAIN 计划与您的查询不匹配。您的查询中的 current_status 在哪里?您的计划中的 PubDate 在哪里?
-
你有很多选择性,但不清楚它来自哪里。很高兴看到 WHERE 子句中的每个条件分别返回了多少行。这可以告诉你要构建什么索引。
标签: sql postgresql indexing