如何优化这个复杂的查询？答案

【问题标题】：How can I optimize this complex query?如何优化这个复杂的查询？
【发布时间】：2020-11-23 09:39:33
【问题描述】：

我正在运行具有很多条件的复杂查询。在大型数据库上，查询需要 2 分钟以上。我正在为我的项目使用 Django。我可以优化这个查询吗？或者我应该使用 Django ORM 查询？哪个更快？ LN_ACCOUNT 表的行数超过 300 万行。任何建议和帮助将不胜感激。非常感谢。

SELECT COUNT(*), SUM((select sum(acc.SALDO_EQUIVAL_OUT)
  from LN_ACCOUNT lna, ACCOUNTS acc
  where lna.LOAN_ID = L.LOAN_ID
  and lna.LOAN_TYPE_ACCOUNT in ('1','4','5','8')
  and lna.DATE_NEXT > '21.11.2020'
  and lna.DATE_VALIDATE <= '20.11.2020'
  and acc.CODE = lna.ACCOUNT_CODE
  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
      or (
              select 'Y' from
              V_REP_LN_BAL v
              where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
              and ROWNUM = 1
          ) = 'Y'
      )))/-100
FROM LN_CARD L
and
    L.LOAN_ID IN (
                  select distinct (LOAN_ID) LID
                  from (
                           select (
                                      select LOAN_ID
                                      from LN_ACCOUNT s
                                      where s.ACCOUNT_CODE = ac.CODE
                                        and s.DATE_NEXT > '21.11.2020'
                                        and s.DATE_VALIDATE <= '21.11.2020'
                                        and rownum = 1) as LOAN_ID
                           from ACCOUNTS ac,
                                V_REP_LN_BAL vr
                           where vr.BAL_ACC = ac.CODE_COA
                             and vr.TYPE_BAL in (1, 2, 3, 4)
                       ) lnIDS
                  where LOAN_ID is not null)
and (
  select sum(abs(acc.SALDO_EQUIVAL_OUT))
  from LN_ACCOUNT lna, ACCOUNTS acc
  where lna.LOAN_ID = l.LOAN_ID
  and lna.LOAN_TYPE_ACCOUNT in ('1','4','5','8')
  and lna.DATE_NEXT > '21.11.2020'
  and lna.DATE_VALIDATE <= '21.11.2020'
  and acc.CODE = lna.ACCOUNT_CODE
  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
      or (
              select 'Y' from
              V_REP_LN_BAL v
              where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
              and ROWNUM = 1
          ) = 'Y'
      )
  )
      + abs(
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 3
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                     V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0) +
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 7
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                 and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0) +
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 79
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0)
          )
      + abs(NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 46
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0))
      + abs(
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 6
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0) +
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 52
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0) +
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 53
                 and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0) +
          NVL((select sum(acc.SALDO_EQUIVAL_OUT)
                  from LN_ACCOUNT lna, ACCOUNTS acc
                  where lna.LOAN_ID = l.LOAN_ID
                  and lna.LOAN_TYPE_ACCOUNT = 54
                  and lna.DATE_NEXT > '21.11.2020'
                  and lna.DATE_VALIDATE <= '21.11.2020'
                  and acc.CODE = lna.ACCOUNT_CODE
                  and (substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                  or  (select 'Y' from
                      V_REP_LN_BAL v
                      where v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                      and ROWNUM = 1) = 'Y'
                  )),0)
          ) <> 0;

【问题讨论】：

语法错误...
您在SELECT 子句和WHERE 子句中使用了很多子查询。尝试将其转换为某种 JOINS。

标签： sql database oracle query-performance sqlperformance

【解决方案1】：

您正在执行多个嵌套子查询：

+ abs(
  NVL((select sum(acc.SALDO_EQUIVAL_OUT)
       from   LN_ACCOUNT lna, ACCOUNTS acc
       where lna.LOAN_ID = l.LOAN_ID
       and   lna.LOAN_TYPE_ACCOUNT = 3
       and   lna.DATE_NEXT > '21.11.2020'
       and   lna.DATE_VALIDATE <= '21.11.2020'
       and   acc.CODE = lna.ACCOUNT_CODE
       and   ( substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
               or  (select 'Y'
                    from   V_REP_LN_BAL v
                    where  v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                    and    ROWNUM = 1
                   ) = 'Y'
             )
     ),
     0
   )

唯一的区别似乎是您正在过滤的lna.LOAN_TYPE_ACCOUNT。将所有这些连接起来，这样您只需要查询一次表：

+ COALESCE(
    (
      SELECT SUM( t.abs_total )
      FROM   (
        SELECT lna.LOAN_ID,
               lna.LOAN_TYPE_ACCOUNT,
               ABS( SUM( acc.SALDO_EQUIVAL_OUT ) ) AS abs_total
        FROM   LN_ACCOUNT lna
               INNER JOIN ACCOUNTS acc
               ON ( acc.CODE = lna.ACCOUNT_CODE )
        WHERE  lna.LOAN_TYPE_ACCOUNT IN ( 3, 7, 79, 46, 6, 52, 53, 54 )
        AND    lna.DATE_NEXT > DATE '2020-11-21'
        AND    lna.DATE_VALIDATE <= DATE '2020-11-21'
        AND    (    substr(lna.ACCOUNT_CODE, -20, 3) in ('163','915')
                 OR EXISTS( SELECT 1
                            FROM   V_REP_LN_BAL v
                            WHERE  v.BAL_ACC like substr(lna.ACCOUNT_CODE, -20, 3)||'%'
                          )
               )
        GROUP BY
               lna.LOAN_ID,
               lna.LOAN_ACCOUNT_TYPE
      ) t
      WHERE    t.LOAN_ID = l.LOAN_ID
    ),
    0
  )

在使用ABS 之前，您需要检查以确保您仍然获得与聚合多个LOAN_TYPE_ACCOUNT 相同的结果；但是，如果是这种情况，那么您可以使用：

CASE 
WHEN LOAN_TYPE_ACCOUNT IN (  3,  7, 79     ) THEN 'Group1'
WHEN LOAN_TYPE_ACCOUNT IN ( 46             ) THEN 'Group2'
WHEN LOAN_TYPE_ACCOUNT IN (  6, 52, 53, 54 ) THEN 'Group3'
END

【讨论】：

我还能做些什么来减少执行时间？
@Mika 我不知道。要获得更好的答案，您需要在edit 您的问题中包含一个minimal reproducible example，其中包含：表的 DDL 语句；一些样本数据的 DML 语句；解释代码试图实现的目标；您对该样本数据的预期输出；以及当前查询的执行计划。 ...
否则，我们将继续关注您的查询，并会挑选出容易获得性能的胜利，但可能会错过一些基本的东西（即可能需要完全重写但我们不知道的完全不同的方法因为我们没有您的表格或数据，也不知道您的目标是什么）。

【解决方案2】：

您需要显示您的查询计划。

为此，您可以使用 SQL Developper 并单击顶部工具栏中的“执行计划”图标（快捷键 F10），也可以直接使用 SQL PLus：

解释计划：

explain plan 
for 
select * from table_name where ...;

显示计划：

select * from table(dbms_xplan.display);

这里看看FULL TABLE SCAN 行。那些您查询完整阅读的显示表。所以如果一个表有超过 3M 行，这可能需要很长时间。为避免 FULL TABLE SCAN 在您在 where 子句中指定的列上创建索引。

【讨论】：