Tableau 生成的 BigQuery 查询运行速度比预期慢，并且计费层级较高

Question

我们公司的一位业务分析师抱怨 Tableau 中的仪表板运行缓慢。它在 BigQuery 上的自定义查询中处理了大量数据。

因此我建议他执行一次查询并将输出存储为临时表，然后在 Tableau 中直接处理该表以加快处理速度。临时表现在只包含 3600 万行（而不是 30 亿行），所以它应该相当快。

不幸的是，一段时间后我们又遇到了麻烦。改进仪表板后，Tableau 生成的查询运行了 30 多分钟，并收到 6 的计费层级。我不知道它有什么复杂之处。 COUNT 语句中有一些日期比较，仅此而已。

谁能解释发生了什么以及我们可以改进什么？当然，我们可以将更多计算转移到一次性作业中，但这有点违背了 Tableau 的初衷。

这是作业 ID：vex-production:job_EDKjXtsTbMMb0Knc6ReNjEPaECk

这是 Tableau 生成的查询：

SELECT
  (CASE WHEN (TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) = TIMESTAMP(DATE(TIMESTAMP([month])))) THEN '2.New Members' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) < TIMESTAMP(DATE(TIMESTAMP([month]))))
      AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) IS NULL)
      AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
      OR TIMESTAMP(DATE(TIMESTAMP([month]))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([month])))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END = 1) THEN '1.Repeated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
      OR TIMESTAMP(DATE(TIMESTAMP([month]))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([month])))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END > 1) THEN '3.Reactivated Visitors' ELSE 'Error' END) AS [Calculation_1030479950946398210],
  COUNT((CASE WHEN (((CASE WHEN (TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) = TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) THEN '2.New Members' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) < TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))
              AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) IS NULL)
              AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
              OR TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END = 1) THEN '1.Repeated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
              OR TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END > 1) THEN '3.Reactivated Visitors' ELSE 'Error' END) = '3.Reactivated Visitors')
        OR ((CASE WHEN (TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) = TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) THEN '2.New Members' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) < TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))
              AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN ((TIMESTAMP(DATE(TIMESTAMP([subscription_month]))) IS NULL)
              AND (TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL)) THEN '3.Reactivated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
              OR TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END = 1) THEN '1.Repeated Visitors' WHEN (CASE WHEN TIMESTAMP(DATE(TIMESTAMP([previous_month]))) IS NULL
              OR TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))) IS NULL THEN INTEGER(NULL) ELSE INTEGER(( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month]))))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP(DATE(TIMESTAMP([month])))))) ) -( (12 * YEAR(TIMESTAMP(DATE(TIMESTAMP([previous_month]))))) + MONTH(TIMESTAMP(DATE(TIMESTAMP([previous_month])))) )) END > 1) THEN '3.Reactivated Visitors' ELSE 'Error' END) = '2.New Members')) THEN [user_id] ELSE NULL END)) AS [TEMP_Calculation_1133499793803542545__4248242405__0_],
  COUNT([user_id]) AS [TEMP_Calculation_1133499793848455199__2157635628__0_],
  MONTH(TIMESTAMP(DATE(TIMESTAMP([month])))) AS [mn_month_ok],
  QUARTER(TIMESTAMP(DATE(TIMESTAMP([month])))) AS [qr_month_ok],
  YEAR(TIMESTAMP(DATE(TIMESTAMP([month])))) AS [yr_month_ok]
FROM
  [vex-custom:samyUS.visitors_analysis] [visitors_analysis]
WHERE
  ((TIMESTAMP(DATE(TIMESTAMP([month]))) >= TIMESTAMP('2011-01-01 00:00:00'))
    AND (TIMESTAMP(DATE(TIMESTAMP([month]))) <= TIMESTAMP('2016-09-01 00:00:00')))
GROUP BY
  1,
  4,
  5,
  6

Answer 1

我们已经调查了这个查询 - 它绝对不应该很昂贵。 BigQuery 内部存在一个内部问题。我们正在努力修复，它已经部分应用于您的表，所以现在有问题的查询应该需要大约 25 秒。随着更多代码更改投入生产，它应该会得到更多改进。