如何使用可变滞后窗口函数？

Question

我有一个具有以下架构的表：

CREATE TABLE example (
    userID,
    status, --'SUCCESS' or 'FAIL'
    date -- self explanatory 
   
);
INSERT INTO example
Values(123, 'SUCCESS', 20211010),
(123, 'SUCCESS', 20211011),
(123, 'SUCCESS', 20211028),
(123, 'FAIL', 20211029),
(123, 'SUCCESS', 20211105),
(123, 'SUCCESS', 20211110)

我正在尝试利用滞后或领先功能来评估当前行是否在前一个“成功”的 2 周窗口内。鉴于当前数据，我预计 isWithin2WeeksofSuccessFlag 如下：

123, 'SUCCESS', 20211010,0 --since it is the first instance
123, 'SUCCESS', 20211011,1
123, 'SUCCESS', 20211028,1
123, 'FAIL', 20211029, 1 --failed, but criteria is that it is within 2 weeks of last success, which it is
123, 'SUCCESS', 20211105, 1 --last success is 2 rows back, but it is within 2 weeks
123, 'SUCCESS', 20211128, 0 --outside of 2 weeks

我最初会考虑做这样的事情：

Select userID, status, date, 
case when lag(status,1) over (partition by userid order by date asc) = 'SUCCESS' 
and date_add('day',-14, date) <= lag(date,1) over (partition by userid order by date asc)
then 1 end as isWithin2WeeksofSuccessFlag
from example

如果我没有 'FAIL' 行，这会起作用。为了处理它，我可以将滞后修改为 2（而不是 1），但是如果我连续有 2、3、4、n 'FAIL' 呢？我需要落后 3,4,5,n+1。两者之间的具体失败次数是可变的。如何处理这种可变性？

注意 我正在查询数十亿行。效率并不是真正的问题（因为它是用于分析），但遇到内存分配错误是。因此，无休止地添加更多窗口函数可能会导致查询自动终止，因为内存需求是超过节点限制。

我应该如何处理？

Answer 1

这是一种方法，也使用窗口函数，每个“公共表表达式”一次处理一个步骤。

注意：问题中的预期结果与问题中的数据不匹配。实际数据中不存在“20211128”。我使用了示例 INSERT 语句。

在测试用例中，我将列名更改为 xdate 以避免任何潜在的 SQL 保留字问题。

SQL：

WITH cte1 AS (
         SELECT *
              , SUM(CASE WHEN status = 'SUCCESS' THEN 1 ELSE 0 END) OVER (PARTITION BY userID ORDER BY xdate) AS grp
           FROM example
     )
   , cte2 AS (
         SELECT *
              , MAX(CASE WHEN status = 'SUCCESS' THEN xdate END) OVER (PARTITION BY userID, grp) AS lastdate
           FROM cte1
     )
   , cte3 AS (
         SELECT *
              , CASE WHEN LAG(lastdate) OVER (PARTITION BY userID ORDER BY xdate) > (xdate - INTERVAL '2' WEEK) THEN 1 ELSE 0 END AS isNear
           FROM cte2
     )
SELECT * FROM cte3
 ORDER BY userID, xdate
;

结果：

+--------+---------+------------+------+------------+--------+
| userID | status  | xdate      | grp  | lastdate   | isNear |
+--------+---------+------------+------+------------+--------+
|    123 | SUCCESS | 2021-10-10 |    1 | 2021-10-10 |      0 |
|    123 | SUCCESS | 2021-10-11 |    2 | 2021-10-11 |      1 |
|    123 | SUCCESS | 2021-10-28 |    3 | 2021-10-28 |      0 |
|    123 | FAIL    | 2021-10-29 |    3 | 2021-10-28 |      1 |
|    123 | SUCCESS | 2021-11-05 |    4 | 2021-11-05 |      1 |
|    123 | SUCCESS | 2021-11-10 |    5 | 2021-11-10 |      1 |
+--------+---------+------------+------+------------+--------+

并调整数据以匹配您的预期结果，再加上引入的新用户，结果是这样的：

+--------+---------+------------+------+------------+--------+
| userID | status  | xdate      | grp  | lastdate   | isNear |
+--------+---------+------------+------+------------+--------+
|    123 | SUCCESS | 2021-10-10 |    1 | 2021-10-10 |      0 |
|    123 | SUCCESS | 2021-10-11 |    2 | 2021-10-11 |      1 |
|    123 | SUCCESS | 2021-10-28 |    3 | 2021-10-28 |      0 |
|    123 | FAIL    | 2021-10-29 |    3 | 2021-10-28 |      1 |
|    123 | SUCCESS | 2021-11-05 |    4 | 2021-11-05 |      1 |
|    123 | SUCCESS | 2021-11-28 |    5 | 2021-11-28 |      0 |
|    323 | SUCCESS | 2021-10-10 |    1 | 2021-10-10 |      0 |
|    323 | SUCCESS | 2021-10-11 |    2 | 2021-10-11 |      1 |
|    323 | SUCCESS | 2021-10-28 |    3 | 2021-10-28 |      0 |
|    323 | FAIL    | 2021-10-29 |    3 | 2021-10-28 |      1 |
|    323 | SUCCESS | 2021-11-05 |    4 | 2021-11-05 |      1 |
|    323 | SUCCESS | 2021-11-28 |    5 | 2021-11-28 |      0 |
+--------+---------+------------+------+------------+--------+

这是一个额外的测试用例，它可能会暴露一些解决方案中的问题：

INSERT INTO example VALUES
   (123, 'SUCCESS', '2021-10-11')
 , (123, 'FAIL'   , '2021-10-12')
 , (123, 'FAIL'   , '2021-10-13')
;

结果：

+--------+---------+------------+------+------------+--------+
| userID | status  | xdate      | grp  | lastdate   | isNear |
+--------+---------+------------+------+------------+--------+
|    123 | SUCCESS | 2021-10-11 |    1 | 2021-10-11 |      0 |
|    123 | FAIL    | 2021-10-12 |    1 | 2021-10-11 |      1 |
|    123 | FAIL    | 2021-10-13 |    1 | 2021-10-11 |      1 |
+--------+---------+------------+------+------------+--------+

Answer 2

如果您的 DBMS 不支持窗口函数过滤器，您可以 order by status desc 让“SUCCESS”在“FAIL”之前出现。

select userID, status, date, 
  case when lag(status,1) over (partition by userid order by status desc , date asc) = 'SUCCESS' 
        and dateadd(d, -14, date) <= lag(date,1) over (partition by userid order by status desc , date asc)
  then 1 end as isWithin2WeeksofSuccessFlag
from example
order by date

Sql Server fiddle