Postgresql 查询执行得非常糟糕答案

【问题标题】：Postgresql query is performing really badPostgresql 查询执行得非常糟糕
【发布时间】：2016-02-26 22:19:43
【问题描述】：

我的 postgresql 数据库的查询性能存在很大问题。 postgresql的版本是："PostgreSQL 8.4.3 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20070115 (prerelease) (SUSE Linux), 64-bit"

我的配置文件设置如下：

shared_buffers = 8GB 
effective_cache_size = 24GB
work_mem = 419430kB 
maintenance_work_mem = 2GB 
checkpoint_segments = 128 
checkpoint_completion_target = 0.9 
wal_buffers = 16MB 
default_statistics_target = 500
constraint_exclusion = on

我需要执行的查询是：

SELECT events.evt_id, events.evt_time, events.device_evt_time , to_ip_char(events.sip) , evt_agent.port , events.rv40 , events.evt , events.msg , events.sun , events.rv35 , events.dun , events.rv45 , events.fn , events.dp , events.trgt_trust_name , events.trgt_trust_domain , events.rv36 , events.rv43 , events.cv21 , events.cv40 , events.cv41 , events.cv42 , events.cv43 , events.cv44 , events.cv50 , events.cv51 , events.cv52 , events.cv53 , events.cv54 , events.cv55 , events.cv56 , events.cv35 , events.cv60 , events.cv61 , events.cv62
FROM events, evt_agent
WHERE 
    events.agent_id = evt_agent.agent_id AND (evt_agent.port::text = ANY (ARRAY['x'::character varying, 'y'::character varying, 'z'::character varying]::text[])) 
    AND events.evt::text <> 'Internal Message'::text
    AND event_time > '2015-12-31 13:23:55.767+00'::timestamptz limit 10;
ORDER BY events.evt_time;

表事件是一个分区表，每天一个分区。每个分区有两个约束：

Name    events_p_YYYYMMDDHHMISS_events_p_max_pk
Columns evt_time, evt_id
Name    events_p_YYYYMMDDHHMISS_dc  
Definition  evt_time > '2015-12-04 13:24:25.267973+00'::timestamp with time zone AND evt_time <= '2015-12-05 13:24:25.267973+00'::timestamp with time zone

和七个索引：

Name    events_p_YYYYMMDDHHMISS_events_p_max_identity_ix1   
Columns evt_time, init_usr_identity_guid, rid02 
Operator classes    timestamptz_ops, uuid_ops, int8_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_identity_ix2   
Columns evt_time, trgt_usr_identity_guid, rid02 
Operator classes    timestamptz_ops, uuid_ops, int8_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_ix1    
Columns evt_time, sev, agent_id 
Operator classes    timestamptz_ops, int4_ops, int8_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_ix2    
Columns evt_time, dip, sev  
Operator classes    timestamptz_ops, int4_ops, int4_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_ix3    
Columns evt_time, res, sev  
Operator classes    timestamptz_ops, text_ops, int4_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_ix4    
Columns evt_time, sip, sev  
Operator classes    timestamptz_ops, int4_ops, int4_ops

Name    events_p_YYYYMMDDHHMISS_events_p_max_ix5    
Columns evt_time, txnmy_id, agent_id    
Operator classes    timestamptz_ops, int8_ops, int8_ops

表 evt_agent 是一个字典表，只有大约 20-30 行。

约束：

Name    evt_agent_pk    
Columns agent_id

索引：

Name    evt_agent_ak1   
Columns agent, port, rn, pn, sn, st, device_ctgry, src_id, cust_id  
Operator classes    text_ops, text_ops, text_ops, text_ops, text_ops, text_ops, text_ops, uuid_ops, int8_ops

Name    evt_agent_ix1   
Columns device_ctgry, agent_id  
Operator classes    text_ops, int8_ops

当我执行查询时，我得到了这个解释计划： http://explain.depesz.com/s/3xcu

Name    evt_agent_ix2   
Columns st, agent_id    
Operator classes    text_ops, int8_ops

Name    test_ev_ag_indx1    
Columns agent_id    
Operator classes    int8_ops

我认为统计数据有问题，所以我“真空分析”所有相关表，但查询性能没有改善，请解释计划。

我尝试了“内部查询”技巧，解释计划在这里： http://explain.depesz.com/s/FAkH

据我所知，情况变得更糟了。

你知道如何为这个查询获得更好的执行计划吗？现在大约需要 42 分钟才能从查询中获得任何结果。

提前致谢！

【问题讨论】：

ORDER BY 是否是查询的一部分？您只想返回按日期 ASC 排序的前 10 行？
ORDER BY是查询的一部分，LIMIT 10只是一个例子，原来是10000，提供10000行批量的增量报告

标签： performance postgresql

【解决方案1】：

我们将不得不尝试消除散列连接。横向连接可能就是答案：

SELECT events.evt_id, events.evt_time, events.device_evt_time , to_ip_char(events.sip) , evt_agent.port , events.rv40 , events.evt , events.msg , events.sun , events.rv35 , events.dun , events.rv45 , events.fn , events.dp , events.trgt_trust_name , events.trgt_trust_domain , events.rv36 , events.rv43 , events.cv21 , events.cv40 , events.cv41 , events.cv42 , events.cv43 , events.cv44 , events.cv50 , events.cv51 , events.cv52 , events.cv53 , events.cv54 , events.cv55 , events.cv56 , events.cv35 , events.cv60 , events.cv61 , events.cv62
FROM evt_agent,  
  LATERAL (SELECT * 
           FROM events AS e
           WHERE
             e.agent_id = evt_agent.agent_id
             AND e.evt::text <> 'Internal Message'::text
             AND e.event_time > '2015-12-31 13:23:55.767+00'::timestamptz 
             LIMIT 10000) AS events
WHERE  (evt_agent.port::text = ANY (ARRAY['x'::character varying, 'y'::character varying, 'z'::character varying]::text[]))         
ORDER BY events.evt_time
LIMIT 10000;

请尝试使用和不使用内部限制，并发布两者的解释计划。

【讨论】：