在 BigQuery 的分组依据和分区依据中使用相同的列

Question

我有一个包含 5 列的表 TAB，
col1: 字符串 ('AAA', 'BBB', ...)
col2: 字符串 (null, 'XYZ', ...)
col3：时间戳（'2020-01-01 00:00:00'，...）
col4: 字符串 ('A1A', 'B1B', ...)
col5: 字符串 ('A2A', null, ...)

SELECT
  col1,
  col2,
  DATE(col3) as date,
  COUNTIF(NULLIF(TRIM(col4),"") IS NULL) AS col4_isnull_total,
  COUNTIF(NULLIF(TRIM(col5),"") IS NULL) AS col5_isnull_total,
  COUNTIF(NULLIF(TRIM(col2),"") IS NULL) OVER (PARTITION BY col1, DATE(col3) AS col2_isnull_total
FROM TAB
WHERE DATE(col3) BETWEEN '2020-01-01' AND '2020-01-31'
GROUP BY col1, col2, DATE(col3)

我想要达到的结果是，

col1	col2	date	col4_isnull_total	col5_isnull_total	col2_isnull_total
AAA	null	2020-01-01	11	15	3
AAA	XYZ	2020-01-01	30	45	3
AAA	ABC	2020-01-01	10	5	3
AAA	null	2020-01-02	1	15	1
AAA	XYZ	2020-01-02	3	45	1
AAA	ABC	2020-01-02	10	5	1
BBB	null	2020-01-01	7	5	4
BBB	XYZ	2020-01-01	3	35	4
BBB	ABC	2020-01-01	14	19	4
...	...	...	...	...	...

所以，基本上我想通过 col1、col2 和 DATE(col3) 进行聚合，并使用 group by 查找空值的数量，并使用 partition by 通过 col1 和 DATE(col3) 查找 col2 中的空值数量@

我得到的错误是
PARTITION BY expression references column col3 which is neither grouped nor aggregated

Answer 1

您可能会考虑 beloq 查询。

WITH TAB AS (
  SELECT 'AAA' col1, null col2, TIMESTAMP '2020-01-01 00:00:00' col3, 'A1A' col4, 'A2A' col5
   UNION ALL
  SELECT 'BBB', 'XYZ', TIMESTAMP '2020-01-02 00:00:00', 'B1B', null
)
SELECT col1,
       col2,
       DATE(col3) AS date,
       COUNTIF(NULLIF(TRIM(col4),"") IS NULL) AS col4_isnull_total,
       COUNTIF(NULLIF(TRIM(col5),"") IS NULL) AS col5_isnull_total,
       COUNTIF(NULLIF(TRIM(col2),"") IS NULL) OVER (PARTITION BY col1, ANY_VALUE(DATE(col3))) AS col2_isnull_total
  FROM TAB
 WHERE DATE(col3) BETWEEN '2020-01-01' AND '2020-01-31'
 GROUP BY 1, 2, 3;

查询结果

有时混合分组聚合和窗口函数会导致混淆。在那种情况下，您最好考虑使用子查询将两者分开。

SELECT *,
       COUNTIF(NULLIF(TRIM(col2),"") IS NULL) OVER (PARTITION BY col1, date) AS col2_isnull_total
   FROM (
     SELECT col1,
            col2,
            DATE(col3) AS date,
            COUNTIF(NULLIF(TRIM(col4),"") IS NULL) AS col4_isnull_total,
            COUNTIF(NULLIF(TRIM(col5),"") IS NULL) AS col5_isnull_total,
       FROM TAB
      WHERE DATE(col3) BETWEEN '2020-01-01' AND '2020-01-31'
      GROUP BY 1, 2, 3
   );

更新

WITH TAB AS (
  SELECT 'AAA' col1, null col2, TIMESTAMP '2020-01-01 00:00:00' col3, 'A1A' col4, 'A2A' col5    UNION ALL
  SELECT 'AAA', null, TIMESTAMP '2020-01-01 00:00:00', null, 'A2A'    UNION ALL   
  SELECT 'AAA', 'XYZ', TIMESTAMP '2020-01-01 00:00:00', null, 'A2A'    UNION ALL
  SELECT 'BBB', 'XYZ', TIMESTAMP '2020-01-02 00:00:00', 'B1B', null
)
SELECT col1,
       col2,
       DATE(col3) AS date,
       COUNTIF(NULLIF(TRIM(col4),"") IS NULL) AS col4_isnull_total,
       COUNTIF(NULLIF(TRIM(col5),"") IS NULL) AS col5_isnull_total,
       ANY_VALUE(col2_isnull_total) AS col2_isnull_total,
  FROM (
    SELECT *,
           COUNTIF(NULLIF(TRIM(col2),"") IS NULL) OVER (PARTITION BY col1, DATE(col3)) AS col2_isnull_total
      FROM TAB
     WHERE DATE(col3) BETWEEN '2020-01-01' AND '2020-01-31'
  )
 GROUP BY 1, 2, 3;