Mysql查询执行非常慢

Question

我在 CodeIgniter 模型中使用了一个查询来获取特定日期之间的产品列表计数。当我的表中的项目较少时，这可以正常工作，但我的表中有超过 100,000 个条目，并且仅获得 2 天的输出大约需要 3-4 分钟。 from 和 to 天分开的时间越长，花费的时间就越多。

这里是查询：(Dbfiddle:https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=e7a99f08ecd217cbeb09fe6676cfe645)

with Y as (
  with recursive D (n, day) as (
    select 1 as n, '2021-09-25' my_date
    union
    select n+1, day + interval 1 day from D
      where day + interval 1 day < '2021-10-15'
  ) select * from D
), X as (
  select Y.day,
         l.*,
         (select status_from from logs
            where logs.refno = l.refno
              and logs.logtime >= Y.day
            order by logs.logtime
            limit 1) logstat
    from listings l, Y
    where l.added_date <= Y.day
), Z as (
  select X.day, ifnull(X.logstat,X.status) stat_day, count(*) cnt
    from X
    group by X.day, stat_day
)
select Z.day,
  sum(case when Z.stat_day = 'D' then Z.cnt else 0 end ) Draft,
  sum(case when Z.stat_day = 'A' then Z.cnt else 0 end ) Action,
  sum(case when Z.stat_day = 'Y' then Z.cnt else 0 end ) Publish,
  sum(case when Z.stat_day = 'S' then Z.cnt else 0 end ) Sold,
  sum(case when Z.stat_day = 'L' then Z.cnt else 0 end ) Let
  from Z
  group by Z.day
  order by Z.day;

基本上，此查询所做的是从日期在所选日期范围期间和之后的日志中获取 status_from，并从日期在用户选择的起始日期范围之前的列表中获取 added_date 并计算它。检索到这些记录后，它会检查表中状态所包含的变量并执行sum(case when else 0) 以获取总数。

我认为查询速度慢的一个原因是它必须计算查询本身的状态总和，所以在 php 端执行计数部分可能会更快？如果是这样，那么我如何为其创建一个语句来迭代我的视图类中的计数。

当前视图类：

<?php
            foreach($data_total as $row ){
               $draft = $row->draft ? $row->draft : 0;
               $publish = $row->publish ? $row->publish : 0;
               $action = $row->action ? $row->action : 0;
               $sold = $row->sold ? $row->sold : 0;
               $let = $row->let ? $row->let : 0;                              
          ?>
              <tr>
                    <td><?= $row->day?></td>
                    <td><?= $draft ?></td>
                    <td><?= $publish ?></td>
                    <td><?= $action ?></td>
                    <td><?= $sold ?></td>
                    <td><?= $let ?></td>
              </tr>
          <?php }  ?>

或者如果可能的话，是否有任何方法可以更快地获得此查询的相同输出。

Answer 1

这样更快吗？ 如果您更频繁地调用查询，您可以考虑将ROW_NUMBER 保存到logs 表

with calendar as (
with recursive cal (n, day) as (
    select 1 as n, '2021-09-25' my_date
    union
    select n+1, day + interval 1 day from cal
    where day + interval 1 day < '2021-10-15'
    )select * from cal
), loggs as (
    select
         ROW_NUMBER() OVER (partition by refno order by logtime) as RN
        ,status_from as logstat
        ,refno
        ,logtime
    from logs
),X as (
  select cal.day,
         l.*,
         logs.logstat,
         RN,
         min(RN) over (partition by l.refno, cal.day) as RN_MIN
    from listings l
    join calendar as cal on l.added_date <= cal.day
    left join loggs as logs on logs.refno = l.refno and logs.logtime >= cal.day
), Z as (
  select X.day, ifnull(X.logstat,X.status) stat_day, count(*) cnt
    from X
    where ifnull(RN, 0) = ifnull(RN_min, 0)
    group by X.day, stat_day
)
select Z.day,
  sum(case when Z.stat_day = 'D' then Z.cnt else 0 end ) Draft,
  sum(case when Z.stat_day = 'A' then Z.cnt else 0 end ) Action,
  sum(case when Z.stat_day = 'Y' then Z.cnt else 0 end ) Publish,
  sum(case when Z.stat_day = 'S' then Z.cnt else 0 end ) Sold,
  sum(case when Z.stat_day = 'L' then Z.cnt else 0 end ) Let
  from Z
  group by Z.day
  order by Z.day;

Answer 2

我简化了您的查询，但我不确定您是否会在时间执行方面得到显着改进。您必须定义合适的索引。

请仔细检查，确保输出正确。

WITH RECURSIVE 
  cal AS (SELECT '2021-09-25' AS day
    
          UNION ALL
    
          SELECT day + interval 1 day 
          FROM cal
          WHERE day + interval 1 day < '2021-10-15'),
  
  X AS (SELECT DISTINCT
                  cal.day,
                  l.id,
                  l.status,
                  FIRST_VALUE(status_from) OVER (PARTITION BY logs.refno, cal.day ORDER BY logs.logtime) AS logstat
        FROM listings l
        INNER JOIN cal ON l.added_date <= cal.day
        LEFT JOIN logs ON logs.refno = l.refno AND logs.logtime >= cal.day)

SELECT X.day,
       COUNT(CASE WHEN IFNULL(X.logstat, X.status) = 'D' THEN 1 END) Draft,
       COUNT(CASE WHEN IFNULL(X.logstat, X.status) = 'A' THEN 1 END) Action,
       COUNT(CASE WHEN IFNULL(X.logstat, X.status) = 'Y' THEN 1 END) Publish,
       COUNT(CASE WHEN IFNULL(X.logstat, X.status) = 'S' THEN 1 END) Sold,
       COUNT(CASE WHEN IFNULL(X.logstat, X.status) = 'L' THEN 1 END) Let
FROM X
GROUP BY X.day
ORDER BY X.day;

Answer 3

如果您的最终输出要在网站上，则数据快照通常比过去活动的实时提要更好。过去，我使用存储过程每天使用 Past Activities 更新表，然后使用视图选择与 Current_Activities 联合的 Past_Activities 以减少查看器的加载时间。