MySQL - 在一个请求中获取当月没有订单但之前没有订单的用户

Question

我需要获取用户数量，按用户类型（A、B、C）和当年的每个月（存在于 db 中）分组 - 仅适用于没有支付订单的用户（总数 > 0）在每个月（SQL 返回的每一行），但在任何前几个月（在任何一年，而不仅仅是当前）都有订单（总数 > 0）。换句话说，这是非活跃用户，他们之前下过一些付费订单，但在返回的当前 SQL 请求行中没有下任何新订单。

我期望在结果中得到什么（值只是示例）：

label   user_type   data
 Nov    B           2
 Nov    A           1
 Nov    C           3
 Dec    C           1
 .... other months

这意味着在 12 月有 5 个用户类型为 A 的用户、3 个用户类型为 B 的用户和 0 个用户类型为 C 的用户，他们没有在 2021 年 12 月下订单，但在任何一年的 12 月之前的某个时间下订单.

带有 SQL 的示例数据库（两个表 - 用户和订单），按每个用户类型显示每个月在本月下订单的用户数量。除了这个简单的结果之外，我还需要获取在本月未下订单但之前在某处下过付费订单的用户数。

https://dbfiddle.uk/?rdbms=mysql_5.6&fiddle=4c4fadf67bcdc7cc3443f46c387173df

我需要适用于 MySQL 5.7 的 SQL

Answer 1

尝试此查询以生成所有月份 x 用户类型的计数

SELECT 
  DATE_FORMAT(DATE(CONCAT_WS('-', YEAR(CURDATE()), months.mm, '01')), "%b") as label, 
  users.user_type, 
  SUM(
    EXISTS (
     SELECT 1
     FROM orders
     WHERE orders.user_id = users.userid
     AND orders.`date` < DATE(CONCAT_WS('-', YEAR(CURDATE()), months.mm, '01'))
    ) AND NOT EXISTS (
     SELECT 1
     FROM orders
     WHERE orders.user_id = users.userid
     AND orders.`date` BETWEEN DATE(CONCAT_WS('-', YEAR(CURDATE()), months.mm, '01')) AND LAST_DAY(DATE(CONCAT_WS('-', YEAR(CURDATE()), months.mm, '01')))
    )
  ) counts
FROM (
 SELECT '01' mm
 UNION SELECT '02' UNION SELECT '03' UNION SELECT '04' UNION SELECT '05' 
 UNION SELECT '06' UNION SELECT '07' UNION SELECT '08' UNION SELECT '09' 
 UNION SELECT '10' UNION SELECT '11' UNION SELECT '12'
) months
CROSS JOIN users
GROUP BY months.mm, users.user_type

demo

Answer 2

测试此查询是否符合您的需要

SELECT DATE_FORMAT(o.date, "%b") as label,
UPPER(u.user_type) as user_type,
COUNT(distinct o.user_id) as data FROM orders o 
JOIN users u ON o.user_id = u.userid 
WHERE DATE_FORMAT(o.date, "%Y") = "2021"
AND o.user_id NOT IN 
    (SELECT DISTINCT o1.user_id FROM orders o1 WHERE DATE_FORMAT(o1.date, "%b") = DATE_FORMAT(now(), "%b") AND YEAR(o1.date) = YEAR(now()) )
AND o.user_id IN 
    (SELECT DISTINCT o1.user_id FROM orders o1 WHERE (DATE_FORMAT(o1.date, "%c") < DATE_FORMAT(now(), "%c") OR YEAR(o1.date) < YEAR(now())))

GROUP BY DATE_FORMAT(o.date, "%Y %b"), 
u.user_type HAVING SUM(o.total) > 0 ORDER BY o.date ASC

编辑 下面的查询返回一年中的每个月

SELECT months.MONTH as label,
ifnull(UPPER(u.user_type), '-') as user_type,
COUNT(distinct o.user_id) as data 
FROM (
                      SELECT 1 AS MONTH
                       UNION SELECT 2 AS MONTH
                       UNION SELECT 3 AS MONTH
                       UNION SELECT 4 AS MONTH
                       UNION SELECT 5 AS MONTH
                       UNION SELECT 6 AS MONTH
                       UNION SELECT 7 AS MONTH
                       UNION SELECT 8 AS MONTH
                       UNION SELECT 9 AS MONTH
                       UNION SELECT 10 AS MONTH
                       UNION SELECT 11 AS MONTH
                       UNION SELECT 12 AS MONTH
            ) as months

LEFT JOIN orders o 
 ON DATE_FORMAT(o.date, "%c") = months.MONTH 
LEFT JOIN users u ON o.user_id = u.userid
WHERE (DATE_FORMAT(o.date, "%Y") = "2021" OR o.date IS NULL)
AND ( 
        (
        NOT EXISTS 
            (SELECT DISTINCT o1.user_id 
                FROM orders o1 
                    WHERE 
                    DATE_FORMAT(o1.date, "%b") = DATE_FORMAT(now(), "%b") 
                    AND YEAR(o1.date) = YEAR(now())
                    AND  o1.user_id = o.user_id
            )
        AND EXISTS
            (SELECT DISTINCT o1.user_id 
                FROM orders o1 
                    WHERE 
                    (DATE_FORMAT(o1.date, "%c") < DATE_FORMAT(now(), "%c") OR YEAR(o1.date) < YEAR(now())) AND  o1.user_id = o.user_id
            )
        ) 
        OR o.user_id IS null OR u.userid IS NULL
    )
GROUP BY months.MONTH, u.user_type ORDER BY months.MONTH ASC

Answer 3

这使用了与 VeteranSlayer 类似的方法，但它从月份和用户之间的交叉连接开始，然后是左连接到订单。它还使用范围而不是函数进行日期比较。它可能表现得很糟糕，但它应该给出正确的结果 -

SELECT
    months.month AS `label`,
    u.user_type,
    COUNT(u.userid) AS `data`
FROM (
    SELECT 'Jan' `month`, '2021-01-01' month_start, '2021-01-31' month_end UNION ALL
    SELECT 'Feb', '2021-02-01', '2021-02-28' UNION ALL
    SELECT 'Mar', '2021-03-01', '2021-03-31' UNION ALL
    SELECT 'Apr', '2021-04-01', '2021-04-30' UNION ALL
    SELECT 'May', '2021-05-01', '2021-05-31' UNION ALL
    SELECT 'Jun', '2021-06-01', '2021-06-30' UNION ALL
    SELECT 'Jul', '2021-07-01', '2021-07-31' UNION ALL
    SELECT 'Aug', '2021-08-01', '2021-08-31' UNION ALL
    SELECT 'Sep', '2021-09-01', '2021-09-30' UNION ALL
    SELECT 'Oct', '2021-10-01', '2021-10-31' UNION ALL
    SELECT 'Nov', '2021-11-01', '2021-11-30' UNION ALL
    SELECT 'Dec', '2021-12-01', '2021-12-31'
) months
INNER JOIN users u
LEFT JOIN orders o
    ON o.date BETWEEN months.month_start AND months.month_end
    AND o.user_id = u.userid
WHERE o.user_id IS NULL
AND EXISTS (
    SELECT DISTINCT o1.user_id 
    FROM orders o1 
    WHERE o1.date < months.month_start
    AND  o1.user_id = u.userid
)
GROUP BY months.month, u.user_type
ORDER BY months.month_start ASC, u.user_type ASC;

编辑

这些查询的性能因数据集的规模、数据的分布和索引而有很大差异。我已经使用许多不同的索引变体和以下测试数据集进行了一些测试。请注意，在两个表中创建的随机数据可能会导致截然不同的性能。 INSERT 的 SELECT 中引用的 dummy 表只是一个 1M 行的随机表。

CREATE TABLE `users` (
    `id` int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
    `user_type` char(1) NOT NULL,
    KEY `IDX_user_type` (`user_type`)
);

INSERT INTO users (user_type)
SELECT 
    CASE (FLOOR(RAND() * 3) + 1) WHEN 1 THEN 'A' WHEN 2 THEN 'B' ELSE 'C' END AS `user_type`
FROM dummy
LIMIT 1000;


CREATE TABLE orders (
    `id` int UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
    `user_id` int,
    `date` DATE,
    `total` DECIMAL(6,2),
    KEY `IDX_user_id_date` (`user_id`,`date`)
);

INSERT INTO orders (user_id, date, total)
SELECT
     (FLOOR(RAND() * 1000) + 1) AS `user_id`,
     ('2020-01-01' + INTERVAL FLOOR(RAND() * 685) + 1 DAY) AS `date`,
     ( (FLOOR(RAND() * 10) + 1) * 5) AS `total`
FROM dummy
LIMIT 100000;

查询之间最显着的性能差异来自添加 -

KEY `IDX_user_id_date` (`user_id`,`date`)

添加 user_type 索引带来了小而一致的改进 -

KEY `IDX_user_type` (`user_type`)

ProGu 的查询以 1.466 秒的平均时间始终如一地执行。我的查询在 0.922 秒时同样一致。您的里程会有所不同！

我没有包含 VeteranSlayer 查询的时间，因为它返回的结果完全不同。

编辑 2

用 50k 用户和 100 万订单重新填充两个表

TRUNCATE TABLE orders;
TRUNCATE TABLE users;

INSERT INTO users (user_type)
SELECT 
    CASE (FLOOR(RAND() * 3) + 1) WHEN 1 THEN 'A' WHEN 2 THEN 'B' ELSE 'C' END AS `user_type`
FROM  (SELECT 1 FROM dummy LIMIT 50000) t;

INSERT INTO orders (user_id, date, total)
SELECT
     (FLOOR(RAND() * 50000) + 1),
     TIMESTAMPADD(SECOND, FLOOR(RAND() * TIMESTAMPDIFF(SECOND, '2016-01-01', '2021-12-13')), '2016-01-01'),
     ((FLOOR(RAND() * 50) + 1) * 5)
FROM (SELECT 1 FROM dummy LIMIT 1000000) t
ORDER BY date;

由此产生的订单分布（按时间和 user_id）非常均匀，这不太可能是现实的，因此我认为这个测试数据集严重加剧了任何性能问题。

令我惊讶的是，通过使用我的月份表，ProGu 的查询速度明显更快，从 21.062 秒下降到 9.703 秒，并且使用的临时表少了一个（两个而不是三个）。

SELECT 
  months.month as label, 
  users.user_type, 
  SUM(
    EXISTS (
     SELECT 1
     FROM orders
     WHERE orders.user_id = users.id
     AND orders.`date` < months.month_start
    ) AND NOT EXISTS (
     SELECT 1
     FROM orders
     WHERE orders.user_id = users.id
     AND orders.`date` BETWEEN months.month_start AND months.month_end
    )
  ) counts
FROM (
    SELECT 'Jan' `month`, '2021-01-01' month_start, '2021-01-31' month_end UNION ALL
    SELECT 'Feb', '2021-02-01', '2021-02-28' UNION ALL
    SELECT 'Mar', '2021-03-01', '2021-03-31' UNION ALL
    SELECT 'Apr', '2021-04-01', '2021-04-30' UNION ALL
    SELECT 'May', '2021-05-01', '2021-05-31' UNION ALL
    SELECT 'Jun', '2021-06-01', '2021-06-30' UNION ALL
    SELECT 'Jul', '2021-07-01', '2021-07-31' UNION ALL
    SELECT 'Aug', '2021-08-01', '2021-08-31' UNION ALL
    SELECT 'Sep', '2021-09-01', '2021-09-30' UNION ALL
    SELECT 'Oct', '2021-10-01', '2021-10-31' UNION ALL
    SELECT 'Nov', '2021-11-01', '2021-11-30' UNION ALL
    SELECT 'Dec', '2021-12-01', '2021-12-31'
) months
CROSS JOIN users
GROUP BY months.month, users.user_type
ORDER BY months.month_start ASC, users.user_type ASC

我上面的查询可以通过预先分组当年的订单数据得到显着改善（你的里程会有所不同但值得考虑）-

SELECT
    months.month AS `label`,
    u.user_type,
    COUNT(u.id) AS `data`
FROM (
    SELECT 'Jan' `month`, '2021-01-01' month_start, '2021-01-31' month_end UNION ALL
    SELECT 'Feb', '2021-02-01', '2021-02-28' UNION ALL
    SELECT 'Mar', '2021-03-01', '2021-03-31' UNION ALL
    SELECT 'Apr', '2021-04-01', '2021-04-30' UNION ALL
    SELECT 'May', '2021-05-01', '2021-05-31' UNION ALL
    SELECT 'Jun', '2021-06-01', '2021-06-30' UNION ALL
    SELECT 'Jul', '2021-07-01', '2021-07-31' UNION ALL
    SELECT 'Aug', '2021-08-01', '2021-08-31' UNION ALL
    SELECT 'Sep', '2021-09-01', '2021-09-30' UNION ALL
    SELECT 'Oct', '2021-10-01', '2021-10-31' UNION ALL
    SELECT 'Nov', '2021-11-01', '2021-11-30' UNION ALL
    SELECT 'Dec', '2021-12-01', '2021-12-31'
) months
INNER JOIN users u
LEFT JOIN (
    SELECT `user_id`, DATE_FORMAT(`date`, '%Y-%m-01') AS `m`
    FROM `orders`
    WHERE `date` >= '2021-01-01'
    GROUP BY `user_id`, `m`
) o
    ON o.m = months.month_start
    AND o.user_id = u.id
WHERE o.user_id IS NULL
AND EXISTS (
    SELECT 1
    FROM orders o1 
    WHERE o1.date < months.month_start
    AND  o1.user_id = u.id
)
GROUP BY months.month, u.user_type
ORDER BY months.month_start ASC, u.user_type ASC

执行时间从 12.422 秒降至 6.497 秒

我尝试的最后一个测试是通过将 first_order_date 添加到 users 表来进行反规范化 -

ALTER TABLE `users` ADD COLUMN `first_order_date` DATE NULL AFTER `user_type`;

UPDATE users u
INNER JOIN (SELECT o.user_id, MIN(date) AS `first_o`, MAX(date) AS `last_o` FROM orders o GROUP BY o.user_id) t ON u.id = t.user_id
SET `u`.`first_order_date` = `t`.`first_o`, `u`.`last_order_date` = `t`.`last_o`;

然后我修改了我的查询以使用它而不是 EXISTS 子查询 -

SELECT
    `months`.`month` AS `label`,
    `u`.`user_type`,
    COUNT(`u`.`id`) AS `data`
FROM (
    SELECT 'Jan' `month`, '2021-01-01' month_start, '2021-01-31' month_end UNION ALL
    SELECT 'Feb', '2021-02-01', '2021-02-28' UNION ALL
    SELECT 'Mar', '2021-03-01', '2021-03-31' UNION ALL
    SELECT 'Apr', '2021-04-01', '2021-04-30' UNION ALL
    SELECT 'May', '2021-05-01', '2021-05-31' UNION ALL
    SELECT 'Jun', '2021-06-01', '2021-06-30' UNION ALL
    SELECT 'Jul', '2021-07-01', '2021-07-31' UNION ALL
    SELECT 'Aug', '2021-08-01', '2021-08-31' UNION ALL
    SELECT 'Sep', '2021-09-01', '2021-09-30' UNION ALL
    SELECT 'Oct', '2021-10-01', '2021-10-31' UNION ALL
    SELECT 'Nov', '2021-11-01', '2021-11-30' UNION ALL
    SELECT 'Dec', '2021-12-01', '2021-12-31'
) `months`
INNER JOIN `users` `u`
LEFT JOIN (
    SELECT `user_id`, DATE_FORMAT(`date`, '%Y-%m-01') AS `m`
    FROM `orders`
    WHERE `date` >= '2021-01-01'
    GROUP BY `user_id`, `m`
) o
    ON `o`.`m` =  `months`.`month_start`
    AND `o`.`user_id` = `u`.`id`
WHERE `o`.`user_id` IS NULL
AND `u`.`first_order_date` < `months`.`month_start`
GROUP BY `months`.`month`, `u`.`user_type`
ORDER BY `months`.`month_start` ASC, `u`.`user_type` ASC;

这会在 1.447 秒内返回相同的结果。显然，应该避免这样的反规范化，但我将其包含在此处，因为它显示了这种情况下的性能优势。