【问题标题】:need help in re-writing this query, which uses same data set multiple times, as per explain plan根据解释计划,在重写此查询时需要帮助,该查询多次使用相同的数据集
【发布时间】:2019-07-04 15:46:53
【问题描述】:

我们的开发团队运行了一个查询,该查询占用大量资源,并且查看解释计划,它似乎多次使用相同的数据集。无论如何我们可以重写这个查询。

现在,我尝试将关联查询替换为直接连接,但除了一个细微差别之外,多个关联查询看起来仍然相同。

select tb2.mktg_id, mktg_cd , count(distinct tb2.conf_id) 
  from
(select conf_id, count(distinct c.mktg_id) as num_cpg 
   from acc_latst c, off_latst ot 
  where c.mktg_id = ot.mktg_id and c.bus_eff_dt > '2019-01-01' and to_date(strt_tms) = '2019-01-10'  
  group by conf_id 
 having count(distinct c.mktg_id) >1 
)tb1,
(select distinct conf_id, c.mktg_id, mktg_cd 
   from acc_latst c, off_latst ot 
  where c.mktg_id = ot.mktg_id and c.bus_eff_dt > '2019-01-01' and to_date(strt_tms) = '2019-01-10'
)tb2
  where tb1.conf_id = tb2.conf_id group by tb2.mktg_id, mktg_cd 

【问题讨论】:

    标签: performance hive hadoop-yarn query-tuning apache-tez


    【解决方案1】:

    一种方法是使用 CTE -

    with res1 as 
    (
    select distinct conf_id, c.mktg_id, mktg_cd 
       from acc_latst c, off_latst ot 
      where c.mktg_id = ot.mktg_id and c.bus_eff_dt > '2019-01-01' and to_date(strt_tms) = '2019-01-10'
    )
    ,res2 as
    (
    select conf_id, count(distinct c.mktg_id) as num_cpg
    from res1 group by conf_id having count(distinct c.mktg_id) > 1
    )
    select res1.mktg_id, mktg_cd, count(distinct res1.conf_id)  from res1 t1 inner join res2 t2 on t1.conf_id=t2.conf_id group by res1.mktg_id, mktg_cd;
    

    如果查询仍然很慢,能否提供表和分区的详细信息。

    【讨论】:

    • 先生,原始查询有 --> tb1 的 count(distinct c.mktg_id) >1,这也是您建议的查询。
    猜你喜欢
    • 2019-09-15
    • 2016-03-25
    • 2021-10-05
    • 2011-07-10
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多