【问题标题】:What should i use instead of IN?我应该用什么代替 IN?
【发布时间】:2014-05-29 21:20:23
【问题描述】:

我有一个这样的查询:

SELECT DISTINCT devices1_.id AS id27_, devices1_.createdTime AS createdT2_27_, devices1_.deletedOn AS deletedOn27_, 
devices1_.deviceAlias AS deviceAl4_27_, devices1_.deviceName AS deviceName27_, devices1_.deviceTypeId AS deviceT21_27_, 
devices1_.equipmentVendor AS equipmen6_27_, devices1_.exceptionDetail AS exceptio7_27_, devices1_.hardwareVersion AS hardware8_27_, 
devices1_.ipAddress AS ipAddress27_, devices1_.isDeleted AS isDeleted27_, devices1_.loopBack AS loopBack27_, 
devices1_.modifiedTime AS modifie12_27_, devices1_.osVersion AS osVersion27_, devices1_.productModel AS product14_27_, 
devices1_.productName AS product15_27_, devices1_.routerType AS routerType27_, devices1_.rundate AS rundate27_, 
devices1_.serialNumber AS serialN18_27_, devices1_.serviceName AS service19_27_, devices1_.siteId AS siteId27_, 
devices1_.siteIdA AS siteIdA27_, devices1_.status AS status27_, devices1_.creator AS creator27_, devices1_.lastModifier AS lastMod25_27_ 
FROM goldenvariation goldenconf0_ 
INNER JOIN devices devices1_ ON goldenconf0_.deviceId=devices1_.id 
CROSS JOIN devices devices2_ 
WHERE goldenconf0_.deviceId=devices2_.id 
AND (goldenconf0_.classType = 'policy-options') 
AND DATE(goldenconf0_.rundate)=DATE('2014-04-14 00:00:00') 
AND devices2_.isDeleted=0 
AND EXISTS (SELECT DISTINCT(deviceId) FROM goldenvariation goldenconf3_ 
        WHERE (goldenconf3_.goldenVariationType = 'MISMATCH') 
        AND (goldenconf3_.classType = 'policy-options') 
        AND DATE(goldenconf3_.rundate)=DATE('2014-04-14 00:00:00')) 
AND EXISTS (SELECT DISTINCT (deviceId) FROM goldenvariation goldenconf4_ 
        WHERE (goldenconf4_.goldenVariationType = 'MISSING') 
        AND (goldenconf4_.classType = 'policy-options') 
        AND DATE(goldenconf4_.rundate)=DATE('2014-04-14 00:00:00'));

它花费了太多时间,我如何重写查询并使其快速?

goldervariation的表结构是:

CREATE TABLE `goldenvariation` (
  `id` BIGINT(20) NOT NULL AUTO_INCREMENT,
  `classType` VARCHAR(255) DEFAULT NULL,
  `createdTime` DATETIME DEFAULT NULL,
  `goldenValue` LONGTEXT,
  `goldenXpath` VARCHAR(255) DEFAULT NULL,
  `isMatched` TINYINT(1) DEFAULT NULL,
  `modifiedTime` DATETIME DEFAULT NULL,
  `pathValue` LONGTEXT,
  `rundate` DATETIME DEFAULT NULL,
  `value` LONGTEXT,
  `xpath` VARCHAR(255) DEFAULT NULL,
  `deviceId` BIGINT(20) DEFAULT NULL,
  `goldenXpathId` BIGINT(20) DEFAULT NULL,
  `creator` INT(10) UNSIGNED DEFAULT NULL,
  `lastModifier` INT(10) UNSIGNED DEFAULT NULL,
  `goldenVariationType` VARCHAR(255) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `FK6804472AD99F2D15` (`deviceId`),
  KEY `FK6804472A98002838` (`goldenXpathId`),
  KEY `FK6804472A27C863B` (`creator`),
  KEY `FK6804472A3617A57C` (`lastModifier`),
  KEY `rundateindex` (`rundate`),
  KEY `varitionidindex` (`id`),
  KEY `classTypeindex` (`classType`),
  CONSTRAINT `FK6804472A27C863B` FOREIGN KEY (`creator`) REFERENCES `users` (`userid`),
  CONSTRAINT `FK6804472A3617A57C` FOREIGN KEY (`lastModifier`) REFERENCES `users` (`userid`),
  CONSTRAINT `FK6804472A98002838` FOREIGN KEY (`goldenXpathId`) REFERENCES `goldenconfigurationxpath` (`id`),
  CONSTRAINT `FK6804472AD99F2D15` FOREIGN KEY (`deviceId`) REFERENCES `devices` (`id`)
) ENGINE=INNODB AUTO_INCREMENT=1868865 DEFAULT CHARSET=latin1;

查询的解释计划是:

"1" "PRIMARY"   "goldenconf0_"  "ref"   "FK6804472AD99F2D15,classTypeindex" "classTypeindex"    "258"   "const" "179223"    "Using where; Using temporary"
"1" "PRIMARY"   "devices2_" "eq_ref"    "PRIMARY,deviceindex"   "PRIMARY"   "8" "cmdb.goldenconf0_.deviceId"    "1" "Using where"
"1" "PRIMARY"   "devices1_" "eq_ref"    "PRIMARY,deviceindex"   "PRIMARY"   "8" "cmdb.goldenconf0_.deviceId"    "1" ""
"3" "DEPENDENT SUBQUERY"    "goldenconf4_"  "index_subquery"    "FK6804472AD99F2D15,classTypeindex" "FK6804472AD99F2D15"    "9" "func"  "19795" "Using where"
"2" "DEPENDENT SUBQUERY"    "goldenconf3_"  "index_subquery"    "FK6804472AD99F2D15,classTypeindex" "FK6804472AD99F2D15"    "9" "func"  "19795" "Using where"

【问题讨论】:

  • 如何将表加入查询而不是使用EXISTS。它应该工作得更快。你也可以从ON中的子查询中指出你的where子句@
  • 你能用一个简单的例子详细说明吗?这将是很大的帮助。
  • 当子查询与外部查询不相关时,为什么要使用exists?这是允许的,但看起来很奇怪。
  • @GordonLinoff 我现在正在尝试加入。这会是一个不错的选择吗?

标签: mysql sql performance join query-optimization


【解决方案1】:
INNER JOIN goldenvariation goldenconf4_ 
ON goldenconf4_.deviceId = goldenconf0_deviceId 
AND (goldenconf4_.goldenVariationType = 'MISSING') 
AND (goldenconf4_.classType = 'policy-options')  
AND DATE(goldenconf4_.rundate)=DATE('2014-04-14 00:00:00'))

以同样的方式更改另一个EXISTS。我认为这个应该工作得更快。还有我的小提示:尝试使用较短的别名。您的查询真的很难阅读。


SELECT DISTINCT 
devices1_.id AS id27_, 
devices1_.createdTime AS createdT2_27_, 
devices1_.deletedOn AS deletedOn27_, 
devices1_.deviceAlias AS deviceAl4_27_,
devices1_.deviceName AS deviceName27_, 
devices1_.deviceTypeId AS deviceT21_27_, 
devices1_.equipmentVendor AS equipmen6_27_,
devices1_.exceptionDetail AS exceptio7_27_,
devices1_.hardwareVersion AS hardware8_27_, 
devices1_.ipAddress AS ipAddress27_, 
devices1_.isDeleted AS isDeleted27_, 
devices1_.loopBack AS loopBack27_, 
devices1_.modifiedTime AS modifie12_27_, 
devices1_.osVersion AS osVersion27_, 
devices1_.productModel AS product14_27_, 
devices1_.productName AS product15_27_, 
devices1_.routerType AS routerType27_, 
devices1_.rundate AS rundate27_, 
devices1_.serialNumber AS serialN18_27_, 
devices1_.serviceName AS service19_27_, 
devices1_.siteId AS siteId27_, 
devices1_.siteIdA AS siteIdA27_, 
devices1_.status AS status27_, 
devices1_.creator AS creator27_, 
devices1_.lastModifier AS lastMod25_27_ 
FROM goldenvariation goldenconf0_ 
INNER JOIN devices devices1_ ON goldenconf0_.deviceId=devices1_.id 
INNER JOIN goldenvariation a on a.deviceId = goldenconf0_.deviceId and a.goldenVariationType = 'MISMATCH'
INNER JOIN goldenvariation b on b.deviceId = goldenconf0_.deviceId and b.goldenVariationType = 'MISSING'
WHERE (goldenconf0_.classType = 'policy-options') 
AND convert(date,goldenconf0_.rundate) = '2014-04-14'
AND devices1_.isDeleted=0 

试试这个。应该比您的查询快得多。您使用CROSS JOIN 加入表,但在SELECT 中甚至没有使用其中的一列。

【讨论】:

  • 我也这样做了。但它的运行速度仍然非常缓慢,需要 10 分钟才能运行。我还需要更多优化吗?
  • 你的桌子上有索引吗?如果不创建一个,它也应该使其更快。此外,从结果中加入子查询应该比加入整个表更快。还有CROSS JOIN devices devices2_这有必要吗?没有更简单的方法来做到这一点?我认为您的联接太多,它们的工作方式相同,并且仅在其他条件下提供相同的信息。也许UNION 会在这里工作得更好?
  • 我注意到你完全无缘无故地加入了goldenconf4_和goldenconf3_,而不是使用(goldenconf4_.goldenVariationType = 'MISSING') 使用(goldenconf4_.goldenVariationType in('MISSING','MISMATCH') )。
  • @glaeran:但构造不一样;原始查询查找“MISSING”行和“MISMATCH”行...
  • @glaeran 为什么会有 WHERE goldenconf0_.deviceId=devices2_.id,如果我们不加入 devices2_
【解决方案2】:

您正在通过 EXISTS 寻找与黄金变化表相关联的元素。我将从该表开始获取不同的 ID,然后加入您的设备表。此外,在转换日期时,您将无法利用 INDEX(如果是索引的一部分)。

INDEX...(classType、rundate、goldenVariationType、deviceID)

将日期子句更改为 >= ?和

此外,您正在对设备表 TWICE 进行交叉联接,该表的匹配“ID”从 GoldenVariations 表到设备 1 和 2 上的相同 ID,这是一种浪费并且没有做任何事情。

您的设备表应该有一个索引 ON (id, isDeleted)

SELECT 
      d1.id AS id27, 
      d1.createdTime AS createdT2_27, 
      d1.deletedOn AS deletedOn27,
      d1.deviceAlias AS deviceAl4_27_, 
      d1.deviceName AS deviceName27_,
      d1.deviceTypeId AS deviceT21_27_,
      d1.equipmentVendor AS equipmen6_27_,
      d1.exceptionDetail AS exceptio7_27_,
      d1.hardwareVersion AS hardware8_27_,
      d1.ipAddress AS ipAddress27_,
      d1.isDeleted AS isDeleted27_,
      d1.loopBack AS loopBack27_,
      d1.modifiedTime AS modifie12_27_,
      d1.osVersion AS osVersion27_,
      d1.productModel AS product14_27_,
      d1.productName AS product15_27_,
      d1.routerType AS routerType27_,
      d1.rundate AS rundate27_,
      d1.serialNumber AS serialN18_27_,
      d1.serviceName AS service19_27_,
      d1.siteId AS siteId27_,
      d1.siteIdA AS siteIdA27_,
      d1.status AS status27_,
      d1.creator AS creator27_,
      d1.lastModifier AS lastMod25_27_ 
   from 
      ( SELECT distinct 
              gv.deviceID
           from
              goldenVariation gv
           where
                  gv.classType =  'policy-options'
              AND gv.runDate >= '2014-04-14' 
              AND gv.runDate < '2014-04-15'
              AND gv.goldenVariationType IN ( 'MISSING', 'MISMATCH' )) PQ
         JOIN devices d1
            ON PQ.deviceId = d1.id 
           AND d1.isDeleted = 0 

【讨论】:

  • @AamirSohail,虽然其他查询可能会通过使用所描述的索引运行,并且简化的预查询应该比您现有的选项执行得更好。没有提到真正优化查询本身的索引。
【解决方案3】:

是的,可以重写查询以提高性能(尽管它看起来像是由 Hibernate 生成的查询,让 Hibernate 使用不同的查询可能是一个挑战。)

您如何确定此查询返回您期望的结果集?因为查询比较奇怪。

就性能而言,美元兑甜甜圈,就性能而言,真正吃掉你的午餐和你的午餐盒的依赖子查询的重复执行。看起来 MySQL 正在使用 deviceId 列上的索引来满足该子查询,这看起来不是最合适的索引。

我们注意到设备表有两个 JOIN 操作;没有理由需要将此表连接两次。两个 JOIN 操作都需要匹配 goldvariation 的 deviceID 列,第二个连接到 devices 表会使用 isDeleted=0 进行额外的过滤。关键字INNERCROSS 对语句完全没有任何影响; devices 表的第二个连接并不是真正的“交叉”连接,它实际上是一个内部连接。 (我们更喜欢在 ON 子句而不是 WHERE 子句中看到连接谓词。

包裹在rundate 列周围的DATE() 函数禁用索引范围扫描操作。可以重写这些谓词以利用适当的索引。

EXISTS 子查询的 SELECT 列表中的 DISTINCT(deviceId) 很奇怪。首先,DISTINCT 是关键字,而不是函数。 deviceId 周围不需要括号。但除此之外,EXISTS 子查询的 SELECT 列表中返回的内容并不重要,它可能只是 SELECT 1

很奇怪看到 EXISTS 谓词的查询没有引用外部查询中的任何表达式(即相关子查询)。这是有效的语法。使用相关子查询,MySQL 对外部查询返回的每一行执行该查询。 EXPLAIN 输出看起来像 MySQL 正在做同样的事情,它没有识别出任何优化。

这些 EXIST 谓词的编写方式,如果没有带有“MISMATCH”的“policy-options”行并且没有带有“MISSING”的“policy-options”行(对于指定的日期,则查询不会返回任何行。如果找到每种类型的行(对于指定的日期,则返回该日期的所有 'policy-options' 行。(它在语法上是有效的,但它很奇怪。)

假设设备表上的id 列是唯一的(即它是主键或该列上有唯一索引,那么在最外层查询中不需要 DISTINCT 关键字。(从 EXPLAIN 输出中,它看起来就像 MySQL 已经优化掉了通常的操作,即 MySQL 认为 DISTINCT 关键字是不必要的。


但归根结底,是依赖子查询在扼杀性能;缺少合适的索引,以及包裹在函数中的日期列上的谓词。

要回答您的问题,是的,可以重写此查询以更有效地返回等效结果集。 (查询是否返回您期望的结果集并不完全清楚。)

SELECT d1.id AS id27_
     , d1.createdTime AS createdT2_27_
     , d1.deletedOn AS deletedOn27_
     , d1.deviceAlias AS deviceAl4_27_
     , d1.deviceName AS deviceName27_
     , d1.deviceTypeId AS deviceT21_27_
     , d1.equipmentVendor AS equipmen6_27_
     , d1.exceptionDetail AS exceptio7_27_
     , d1.hardwareVersion AS hardware8_27_
     , d1.ipAddress AS ipAddress27_
     , d1.isDeleted AS isDeleted27_
     , d1.loopBack AS loopBack27_
     , d1.modifiedTime AS modifie12_27_
     , d1.osVersion AS osVersion27_
     , d1.productModel AS product14_27_
     , d1.productName AS product15_27_
     , d1.routerType AS routerType27_
     , d1.rundate AS rundate27_
     , d1.serialNumber AS serialN18_27_
     , d1.serviceName AS service19_27_
     , d1.siteId AS siteId27_
     , d1.siteIdA AS siteIdA27_
     , d1.status AS status27_
     , d1.creator AS creator27_
     , d1.lastModifier AS lastMod25_27_ 
  FROM devices d1
  JOIN (SELECT g.deviceId
          FROM goldenvariation g
         CROSS 
          JOIN (SELECT 1
                  FROM goldenvariation x3
                 WHERE x3.goldenVariationType = 'MISMATCH'
                   AND x3.classType = 'policy-options' 
                   AND x3.rundate >= '2014-04-14'
                   AND x3.rundate <  '2014-04-14' + INTERVAL 1 DAY
                 LIMIT 1
               ) t3
         CROSS      
          JOIN (SELECT 1
                  FROM goldenvariation x4 
                 WHERE x4.goldenVariationType = 'MISSING'
                   AND x4.classType = 'policy-options'
                   AND x4.rundate >= '2014-04-14'
                   AND x4.rundate <  '2014-04-14' + INTERVAL 1 DAY
                 LIMIT 1
               ) t4
         WHERE g.classType = 'policy-options'
           AND g.rundate >= '2014-04-14'
           AND g.rundate <  '2014-04-14' + INTERVAL 1 DAY
         GROUP BY g.deviceId
       ) t2
    ON t2.device_id = d1.id
 WHERE d1.isDeleted=0

【讨论】:

  • 投票支持完整描述。如果第一个有任何问题,也会尝试这个。
  • glaeren 的答案中的查询返回一个与原始查询不同的结果集。原始查询将返回具有匹配的黄金变化行的设备; glaeren 的答案中的查询将仅返回具有匹配“MISMATCH”和匹配“MISSING”行的设备;这可能是预期的结果,但它与结果集与原始查询有很大不同。
  • 将再次正确测试。如果我遇到问题会回复你。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2010-11-05
  • 2010-12-06
  • 2012-01-16
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多