避免在 Hive 中自我加入

Question

我正在使用 Hives 内置的 collect_set 函数。该表如下所示：

 cookie, events, keywords,pages 
 1234      1      'dress'  10
 1234      1      'dress'  10
 1235      2      'shoes'  14
 1234      5      'socks'  22

使用 collect_set 我可以得到以下结构

   select cookie, collect_set(events) as ev, collect_set(keywords) as kwords, 
   collect_set(pages)
    from table1 
    group by cookie

我需要做的是多次搜索收集的数组，例如：

 select cookie 
 ,array_contains(collect_set(events),2) as has_2
 ,array_contains(collect_set(keywords),1) as has_4
  from table1
  group by cookie) A 

据我了解，我无法对一个字段进行超过 1 次的投影，最终不得不做类似的事情

 select a.cookie,a.has_2,b.has_4 from ( 
 select cookie 
 ,array_contains(collect_set(events),2) as has_2 
 from table1 group by cookie ) A
 inner join 
 select cookie 
 ,array_contains(collect_set(events),4) as has_4
 from table1 group by cookie) B 
on A.cookie = B. cookie

最终结果如下：

 cookie, has_2, has_4 
 1234     F      F 
 1235     T      T 

有没有办法在没有自我加入的情况下做到这一点？目前，我必须自我加入 30 次才能获得所需的格式。

谢谢

Answer 1

select S.cookie, array_contains(S.events_set,2), array_contains(S.events_set,4) 
from
(select cookie, collect_set(events) as events_set
 from table1 group by cookie ) S

Answer 2

您应该在 SQL 中引入 GROUP BY。

例如

select
    cookie,
    array_contains(collect_set(events),2) as has_2,
    array_contains(collect_set(keywords),1) as has_4
 from
    table1
 group by
    cookie;