DB2 - 虚假的估计运行时间和 SQL0666 问题

Question

我正在尝试根据当前地址和更改日志记录在某个时间点重新创建地址。我们在line1 到line4 字段中有一个客户地址表。以下是表格和数据的近似值：

create table qtemp.customers (
  id int,
  line1 char(40),
  line2 char(40),
  line3 char(40),
  line4 char(40));

insert into qtemp.customers (id, line1, line2, line3, line4)
with cust_temp(id, line1, line2, line3, line4) as(
  select 1, 'Line1', 'Line2', 'Line3', 'Line4'
  from sysibm.sysdummy1
  union all
  select id+1, line1, line2, line3, line4 from cust_temp where id<15000) 
select * from cust_temp;

然后我们有一个记录更改的表，包括各个地址行的更改日志。我感兴趣的更改类型以“Line”和数字开头。它们混合在其他变化中。再次粗略估算一下表格：

create table qtemp.changes (
  seq int,
  dt int,
  cid int,
  change_type char(40),
  change char(40));

insert into qtemp.changes (seq, dt, cid, change_type, change) 
with changes_temp(seq, dt, cid, change_type, change) as(
  select 1, 1, 1, 'not a real change', 'just a bogus line' from sysibm.sysdummy1
  union all
  select seq+1,
         dt + int(rand() + 0.005), --about 175 changes per day on average 
         int(rand() * 15000 + 1), 
         case int(rand() * 13) --little less then 3000 changes to address line
           when 1 then 'Line ' || int(rand() * 4 + 1)
           else trim(TRANSLATE ( CHAR(BIGINT(RAND() * 50 )), 'abcdefghij', '1234567890' )) || ' Some other change'
         end, 
         TRANSLATE ( CHAR(BIGINT(RAND() * 10000000000 )), 'abcdefghij', '1234567890' ) 
  from changes_temp where seq < 35000)
select * from changes_temp;

我的解决方案是只选择 'Line%' 记录，将它们转置到相应的 line1 到 line4 列，然后使用窗口函数填充空值。

with
changes_filtered as (
  select * from changes
  where change_type like 'Line%'),
--only show the last change for any particular customer id, date and line
changes_latest as (
  select a.*
  from changes_filtered a 
  left join changes_filtered b on a.cid = b.cid and a.dt = b.dt and a.change_type = b.change_type and a.seq<b.seq
  where b.seq is null),
changes_pivoted as (
  select cid, dt, 
    max(case when change_type = 'Line 1' then change end) line1, 
    max(case when change_type = 'Line 2' then change end) line2, 
    max(case when change_type = 'Line 3' then change end) line3, 
    max(case when change_type = 'Line 4' then change end) line4
  from changes_latest
  group by cid, dt
  
  union all 
  
  select id, 99999, line1, line2, line3, line4 from customers
  where id in (select cid from changes_filtered)
  ),  
changes_filled as (
  select cid, dt,
     first_value(line1) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line1,
     first_value(line2) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line2,
     first_value(line3) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line3,
     first_value(line4) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line4
  from changes_pivoted
  )  
select * from changes_filled order by cid, dt;

但是，当我尝试运行它时，我立即收到以下错误

[SQL0666] SQL 查询超出指定的时间限制或存储限制。原因 。 . . . . : 即将开始一个数据库查询，其估计运行时间 2147352576 超过了指定的限制 600

注意估计这个词。这是先发制人的打击。 600 秒的限制是通过系统值设置的。如果我用CHGQRYA 覆盖它，查询将在 150 毫秒内运行。所以估计的运行时间完全是假的。当我查看视觉解释时，每个 OLAP 的累积时间呈指数增长。第一个估计时间是1134s，第二个4M s，第三个1400M s，第四个50000000M s。

我发现这个关于 ODBC Query Timeout Property: SQL0666 Estimated Query Processing Time Exceeds Limit 的 IBM 文档说明了这一点

如果没有合适的索引，估计可能会很差 （并且查询可能表现不佳）。

但 150 毫秒 vs 160 万年甚至没有错。查询执行良好，但估计甚至不在这个星系中。

编辑：我想我想问的问题是，在不更改运行时系统值 (QQRYTIMLMT) 并且不为此查询构建专用索引的情况下，是否有解决此问题的方法。

EDIT2：创建索引无效。我尝试自己创建索引以及索引顾问建议的索引，并且在估计的运行时间上我没有遇到任何差异。我有一个 IBM 已立案。

CREATE INDEX QTEMP/CHANGES_IDX
  ON QTEMP/CHANGES (CID ASC, DT ASC) UNIT ANY KEEP IN MEMORY NO;
CREATE INDEX QTEMP/CHANGES_IDX2
  ON QTEMP/CHANGES (CHANGE_TYPE ASC, DT ASC, CID ASC) UNIT ANY KEEP IN MEMORY NO;
CREATE INDEX QTEMP/CUSTOMERS_IDX
  ON QTEMP/CUSTOMERS (ID ASC) UNIT ANY KEEP IN MEMORY NO;
CREATE INDEX QTEMP/CHANGES_IDX3
  ON QTEMP/CHANGES (CID ASC, CHANGE_TYPE ASC) UNIT ANY KEEP IN MEMORY NO;

Answer 1

简短回答，不。

您已要求系统根据估计的运行时间禁止查询。您要么需要更改限制，要么创建索引或收集系统所需的统计信息以进行更好的估计。

话虽如此，您可以检查一下您是否是最新的 PTF，如果不是最新的。

一旦更新，如果您仍然发现与估计有偏差，您可能会向 IBM 开一个案例，让他们查看。这可能是一个错误，或者至少他们可能对您需要哪些索引或统计数据提出建议，以便 Db 能够做出更好的估计。

您也可以尝试将其作为存储过程来执行。有一些计划信息存储在存储过程对象中，可以帮助系统更好地估计运行时间。 （只是一个想法，不知道它是否正确。IBM 可以告诉你）

编辑
我可能提出的一个建议是修改INSERT，以便只输出Line% 记录，从而消除对changes_filtered CTE 的需要。

您也可以尝试将OPTIMIZE FOR xx ROWS 子句添加到您的SELECT 语句中，以便让数据库更好地了解您期望的行数。

Answer 2

我可以通过将 changes_filled CTE 拆分为四个 CTE 来欺骗它，每个 CTE 一个用于窗口函数，然后将它们重新连接在一起。这似乎证明了一次选择中多个窗口的估计是错误的。

with
changes_filtered as (
  select * from changes
  where change_type like 'Line%'),
--only show the last change for any particular customer id, date and line
changes_latest as (
  select a.*
  from changes_filtered a 
  left join changes_filtered b on a.cid = b.cid and a.dt = b.dt and a.change_type = b.change_type and a.seq<b.seq
  where b.seq is null),
changes_pivoted as (
  select cid, dt, 
    max(case when change_type = 'Line 1' then change end) line1, 
    max(case when change_type = 'Line 2' then change end) line2, 
    max(case when change_type = 'Line 3' then change end) line3, 
    max(case when change_type = 'Line 4' then change end) line4
  from changes_latest
  group by cid, dt
  
  union all 
  
  select id, 99999, line1, line2, line3, line4 from customers
  where id in (select cid from changes_filtered)
  ),  
changes_filled1 as (
  select cid, dt,
     first_value(line1) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line1
  from changes_pivoted
  ),  
changes_filled2 as (
  select cid, dt,
     first_value(line2) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line2
  from changes_pivoted
  ),  
changes_filled3 as (
  select cid, dt,
     first_value(line3) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line3
  from changes_pivoted
  ),  
changes_filled4 as (
  select cid, dt,
     first_value(line4) ignore nulls over(partition by cid order by dt rows between current row and unbounded following) line4
  from changes_pivoted
  )  

select t1.cid, t1.dt, line1, line2, line3, line4 
from changes_filled1 t1
join changes_filled2 t2 on t1.cid = t2.cid and t1.dt = t2.dt
join changes_filled3 t3 on t1.cid = t3.cid and t1.dt = t3.dt
join changes_filled4 t4 on t1.cid = t4.cid and t1.dt = t4.dt
order by t1.cid, t1.dt;