在 Athena 中爆炸一个数组

Question

我在 athena 中有一个简单的表，它有一系列事件。我想写一个简单的选择语句，让数组中的每个事件都变成一行。

我尝试了爆炸、变形，但没有成功。我已经在 Spark 和 Hive 中成功完成了它。但是这个雅典娜在欺骗我。请指教

DROP TABLE bi_data_lake.royalty_v4;
CREATE external TABLE bi_data_lake.royalty_v4 (
   KAFKA_ID string,
   KAFKA_TS string,
   deviceUser struct< deviceName:string, devicePlatform:string >,
   consumeReportingEvents array<
                                struct<
                                        consumeEvent: string,
                                        consumeEventAction: string,
                                        entryDateTime: string
                                      >
                               >
   )
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION 's3://XXXXXXXXXXX';

查询不工作

select kafka_id, kafka_ts,deviceuser, 
transform( consumereportingevents, consumereportingevent -> consumereportingevent.consumeevent) as cre
from bi_data_lake.royalty_v4 
where kafka_id = 'events-consumption-0-490565';

不支持 侧视图爆炸（消费报告事件）作为消费者报告事件

回答问题以使用 unnset

找到我问题的答案

WITH samples AS (
 select kafka_id, kafka_ts,deviceuser, consumereportingevent, consumereportingeventPos
 from bi_data_lake.royalty_v4 
 cross join unnest(consumereportingevents)  WITH ORDINALITY AS T (consumereportingevent, consumereportingeventPos)
 where kafka_id = 'events-consumption-0-490565' or kafka_id = 'events-consumption-0-490566'
)
SELECT * FROM samples

Answer 1

在 AWS Athena 中使用 UNNEST 扁平化（'explode'）嵌套数组。

WITH dataset AS (
  SELECT
    'engineering' as department,
    ARRAY['Sharon', 'John', 'Bob', 'Sally'] as users
)
SELECT department, names FROM dataset
CROSS JOIN UNNEST(users) as t(names)

参考：Flattening Nested Arrays