【问题标题】:Very slow performance in Django with remote Oracle server使用远程 Oracle 服务器的 Django 性能非常慢
【发布时间】:2013-05-30 01:34:49
【问题描述】:

我正在 Django 中对远程 Oracle 服务器运行原始 SQL 查询。查询很长,需要一分半钟才能完成,但如果我使用 Oracle SQL Server 程序执行相同的查询,查询运行时间不到一秒。

为什么性能差异如此之大?如何在 Django 中加快查询速度?

顺便说一句,我正在使用 Django 的开发服务器和配置文件工具栏 (Django 1.5)。

更新:这是 Django 中的查询

holidays_filter = ''
if filters['holidays'] == 'exclude':
    holidays_filter = 'AND FECHAS.FESTIVO = 0'

hp_inner_join = ''
if filters['hour-mode'] == 'hp-sector-ps':
    hp_inner_join = """
        INNER JOIN
        EGW.RHP_CELLSEC_PS HPCELLPS
        ON UCELL2.DIA = HPCELLPS.DIA
           AND UCELL2.HORA = HPCELLPS.HORA
           AND UCELL2.RNC = HPCELLPS.RNC
           AND UCELL2.UTRANCELL = HPCELLPS.CELLID
        """
elif filters['hour-mode'] == 'hp-rnc-ps':    
    hp_inner_join = """
        INNER JOIN
        EGW.RHP_RNC_PS HPRNCPS
        ON UCELL2.DIA = HPRNCPS.DIA
           AND UCELL2.HORA = HPRNCPS.HORA
           AND UCELL2.RNC = HPRNCPS.RNC
        """

sql = """
  SELECT CUSTOM.DIA, CUSTOM.HORA,
         ROUND(CUSTOM.TRAF_CS57 + CUSTOM.TRAF_CS64 + CUSTOM.TRAF_CS12 + CUSTOM.TRAF_CSAMR12200 + CUSTOM.TRAF_CSAMR7950 + CUSTOM.TRAF_CSAMR5900 + CUSTOM.TRAF_CSAMR4750, 1) AS TRAFICO_CS_ERL,
         ROUND(CUSTOM.A + CUSTOM.B + CUSTOM.C + CUSTOM.D, 1) AS TRAFICO_PS_ERL

    FROM (SELECT TOTAL.DIA, TOTAL.HORA, TOTAL.RNC, TOTAL.UTRANCELL,
                 CASE
                     WHEN TOTAL.PMSAMPLESCS12RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMCS12RABESTABLISH / TOTAL.PMSAMPLESCS12RABESTABLISH
                 END AS TRAF_CS12,
                 CASE
                     WHEN TOTAL.PMSAMPLESBESTCS57RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTCS57RABESTABLISH / TOTAL.PMSAMPLESBESTCS57RABESTABLISH
                 END AS TRAF_CS57,
                 CASE
                     WHEN TOTAL.PMSAMPLESBESTCS64RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTCS64RABESTABLISH / TOTAL.PMSAMPLESBESTCS64RABESTABLISH
                 END AS TRAF_CS64,
                 CASE
                     WHEN TOTAL.PMSAMPLBESTAMR12200RABESTABLIS = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTAMR12200RABESTABLISH / TOTAL.PMSAMPLBESTAMR12200RABESTABLIS
                 END AS TRAF_CSAMR12200,
                 CASE
                     WHEN TOTAL.PMSAMPLBESTAMR7950RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTAMR7950RABESTABLISH / TOTAL.PMSAMPLBESTAMR7950RABESTABLISH
                 END AS TRAF_CSAMR7950,
                 CASE
                     WHEN TOTAL.PMSAMPLBESTAMR5900RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTAMR5900RABESTABLISH / TOTAL.PMSAMPLBESTAMR5900RABESTABLISH
                 END AS TRAF_CSAMR5900,
                 CASE
                     WHEN TOTAL.PMSAMPLBESTAMR4750RABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTAMR4750RABESTABLISH / TOTAL.PMSAMPLBESTAMR4750RABESTABLISH
                 END AS TRAF_CSAMR4750,

                 CASE
                     WHEN TOTAL.PMSAMPLEBESTDCHPSINTRABESTABLI = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTDCHPSINTRABESTABLISH / TOTAL.PMSAMPLEBESTDCHPSINTRABESTABLI
                 END AS A,
                 CASE
                     WHEN TOTAL.PMSAMPLEFACHPSINTRABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMFACHPSINTRABESTABLISH / TOTAL.PMSAMPLEFACHPSINTRABESTABLISH
                 END AS B,
                 CASE
                     WHEN TOTAL.PMSAMPBESTPSHSADCHRABESTABLISH = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTPSHSADCHRABESTABLISH / TOTAL.PMSAMPBESTPSHSADCHRABESTABLISH
                 END AS C,
                 CASE
                     WHEN TOTAL.PMSAMPLBESTPSEULRABESTABLI = 0
                     THEN 0
                     ELSE TOTAL.PMSUMBESTPSEULRABESTABLISH / TOTAL.PMSAMPLBESTPSEULRABESTABLI
                 END AS D

            FROM (SELECT UCELL2.DIA, UCELL2.HORA, UCELL2.RNC, UCELL2.UTRANCELL,
                         SUM(UCELL2.PMSAMPLESCS12RABESTABLISH) AS PMSAMPLESCS12RABESTABLISH,
                         SUM(UCELL2.PMSUMCS12RABESTABLISH) AS PMSUMCS12RABESTABLISH,
                         SUM(UCELL3.PMSAMPLESBESTCS57RABESTABLISH) AS PMSAMPLESBESTCS57RABESTABLISH,
                         SUM(UCELL3.PMSUMBESTCS57RABESTABLISH) AS PMSUMBESTCS57RABESTABLISH,
                         SUM(UCELL3.PMSAMPLESBESTCS64RABESTABLISH) AS PMSAMPLESBESTCS64RABESTABLISH,
                         SUM(UCELL3.PMSUMBESTCS64RABESTABLISH) AS PMSUMBESTCS64RABESTABLISH,
                         SUM(UCELL3.PMSUMBESTDCHPSINTRABESTABLISH) AS PMSUMBESTDCHPSINTRABESTABLISH,
                         SUM(UCELL3.PMSAMPLEBESTDCHPSINTRABESTABLI) AS PMSAMPLEBESTDCHPSINTRABESTABLI,
                         SUM(UCELL3.PMSUMFACHPSINTRABESTABLISH) AS PMSUMFACHPSINTRABESTABLISH,
                         SUM(UCELL3.PMSAMPLEFACHPSINTRABESTABLISH) AS PMSAMPLEFACHPSINTRABESTABLISH,
                         SUM(UCELL3.PMSUMBESTPSHSADCHRABESTABLISH) AS PMSUMBESTPSHSADCHRABESTABLISH,
                         SUM(UCELL3.PMSAMPBESTPSHSADCHRABESTABLISH) AS PMSAMPBESTPSHSADCHRABESTABLISH,
                         SUM(UCELL3.PMSUMBESTPSEULRABESTABLISH) AS PMSUMBESTPSEULRABESTABLISH,
                         SUM(UCELL3.PMSAMPLBESTPSEULRABESTABLI) AS PMSAMPLBESTPSEULRABESTABLI,
                         SUM(UCELL4.PMSAMPLBESTAMR12200RABESTABLIS) AS PMSAMPLBESTAMR12200RABESTABLIS,
                         SUM(UCELL4.PMSUMBESTAMR12200RABESTABLISH) AS PMSUMBESTAMR12200RABESTABLISH,
                         SUM(UCELL4.PMSAMPLBESTAMR7950RABESTABLISH) AS PMSAMPLBESTAMR7950RABESTABLISH,
                         SUM(UCELL4.PMSUMBESTAMR7950RABESTABLISH) AS PMSUMBESTAMR7950RABESTABLISH,
                         SUM(UCELL4.PMSAMPLBESTAMR5900RABESTABLISH) AS PMSAMPLBESTAMR5900RABESTABLISH,
                         SUM(UCELL4.PMSUMBESTAMR5900RABESTABLISH) AS PMSUMBESTAMR5900RABESTABLISH,
                         SUM(UCELL4.PMSAMPLBESTAMR4750RABESTABLISH) AS PMSAMPLBESTAMR4750RABESTABLISH,
                         SUM(UCELL4.PMSUMBESTAMR4750RABESTABLISH) AS PMSUMBESTAMR4750RABESTABLISH

                    FROM EGW.TF_RNC_RAN_UCELL2 UCELL2

                         INNER JOIN
                         EGW.TF_RNC_RAN_UCELL3 UCELL3
                         ON UCELL2.DIA = UCELL3.DIA
                            AND UCELL2.HORA = UCELL3.HORA
                            AND UCELL2.RNC = UCELL3.RNC
                            AND UCELL2.MO = UCELL3.MO
                            AND UCELL2.MINUTO = UCELL3.MINUTO
                            AND UCELL2.UTRANCELL = UCELL3.UTRANCELL

                         INNER JOIN
                         EGW.TF_RNC_RAN_UCELL4 UCELL4
                         ON UCELL2.DIA = UCELL4.DIA
                            AND UCELL2.HORA = UCELL4.HORA
                            AND UCELL2.RNC = UCELL4.RNC
                            AND UCELL2.MO = UCELL4.MO
                            AND UCELL2.MINUTO = UCELL4.MINUTO
                            AND UCELL2.UTRANCELL = UCELL4.UTRANCELL

                         INNER JOIN
                         JANO.FECHAS FECHAS
                         ON UCELL2.DIA = FECHAS.FECHA

                         """ + hp_inner_join + """

                   WHERE UCELL2.DIA BETWEEN TO_DATE(%s, 'YYYY-MM-DD') AND TO_DATE(%s, 'YYYY-MM-DD')
                     AND UCELL2.HORA BETWEEN %s AND %s
                     AND UCELL2.RNC = %s
                     AND UCELL2.UTRANCELL = %s
                     AND FECHAS.DIASEM IN (%s,%s,%s,%s,%s,%s,%s) 
                     """ + holidays_filter + """

                GROUP BY UCELL2.DIA, UCELL2.HORA, UCELL2.RNC, UCELL2.UTRANCELL) TOTAL) CUSTOM

ORDER BY CUSTOM.DIA, CUSTOM.HORA        
"""

【问题讨论】:

  • 你的意思是 sqlplus 吗?返回多少行数据?您确定查询是 EXACT 吗?是否在一个与另一个中使用绑定参数。请显示您的应用服务器中的表结构、sqlplus 运行和 sql 代码。如果您在每个环境中多次运行相同的查询,执行时间是否一致?
  • 好点。查询不完全相同。在 Django 中,我使用绑定参数,而在 Oracle SQL Server 中(我想它在后台使用 sqlplus)在查询中硬编码的“参数”。我做了一个测试,用硬编码参数修改了 Django 查询,性能提高了很多。如何在 Django 中使用绑定参数获得相同的性能?
  • 你的模型是什么样的?
  • 我的应用中没有任何模型。它只是对数据库进行查询以使用图表呈现数据的可视化表示

标签: django oracle


【解决方案1】:

您应该使用解释计划运行这两个查询以查看差异。我的猜测是,在字面参数的情况下,正在使用一个索引,而在绑定参数的情况下,它可能不会被使用。如果您可以看到两者之间的区别,您可能需要在您的选择语句中添加一个“提示”子句以强制选择正确的索引。您可以阅读有关提示here

如果你使用的是sqlplus,你可以打开autotrace来显示解释计划和执行统计,例如:

SQL> set autotrace on
SQL> set linesize 200
SQL> set serveroutput on
SQL> spool foo.log
SQL> select count(*) from ucbcust; -- this is just a test table in my database. replace with your SQL.
SQL>   ... output shows...
SQL>  spool off


Output would look something like this (obviously, different for your query):



  Execution Plan
    ----------------------------------------------------------                                  
    Plan hash value: 1527793343                                                                                           ----------------------------------------------------------------------------------------          
    | Id  | Operation             | Name                   | Rows  | Cost (%CPU)| Time     |            

     ----------------------------------------------------------------------------------------         

| 0 |选择声明 | | 1 | 3 (0)| 00:00:01 | 1 |排序聚合 | | 1 | |
| 2 |索引快速全扫描| UCBCUST_CUST_KEY_INDEX | 2088 | 3 (0)| 00:00:01

|                                                                                                    ----------------------------------------------------------------------------------------
    Statistics

              0  recursive calls                                                                                                                                                                            
              0  db block gets                                                                                                                                                                              
             17  consistent gets                                                                                                                                                                            
              0  physical reads                                                                                                                                                                             
              0  redo size                                                                                                                                                                                  
            343  bytes sent via SQL*Net to client                                                                                                                                                           
            364  bytes received via SQL*Net from client                                                                                                                                                     
              2  SQL*Net roundtrips to/from client                                                                                                                                                          
              0  sorts (memory)                                                                                                                                                                             
              0  sorts (disk)                                                                                                                                                                               
              1  rows processed                                                                                                                                                                             

要在 sqlplus 中以类似的方式执行参数化查询,您需要定义一个变量作为占位符,例如:

SQL> variable foo number
SQL> exec :foo := 1000

PL/SQL procedure successfully completed.

SQL> select :foo from dual
  2  /

      :FOO
----------
      1000

因此,在您的查询中,将所有“%*”参数替换为变量并在 Sqlplus 中运行以查看执行详细信息。希望通过这些工具,您会看到执行计划的不同。

【讨论】:

  • 当我尝试运行 SQL> SET autotrace ON; 时,我得到以下信息:SP2-0613: Unable to verify PLAN_TABLE format or existence SP2-0611: Error enabling EXPLAIN report SP2-0618: Cannot find the Session Identifier. Check PLUSTRACE role is enabled SP2-0611: Error enabling STATISTICS report
  • 您的数据库上没有安装 explain_plan 模式和表。如果有,请咨询您的 DBA。
  • 我发现了问题,请参阅下面的 mi 答案。非常感谢您的帮助
【解决方案2】:

好的,我找到了解决方案。在 Django 中,所有绑定的参数都定义为字符串 (%s),因此当参数是数字时,会进行很多隐式转换。解决方案是使用WHERE 子句中的函数TO_NUMBER(%s) 将它们显式类型转换为数字。

【讨论】:

  • 酷。很高兴你明白了。
猜你喜欢
  • 2013-07-18
  • 1970-01-01
  • 2021-04-20
  • 2014-12-05
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2013-08-26
  • 1970-01-01
相关资源
最近更新 更多