MySQL 升级后 MySQL NOT IN 查询慢得多答案

【问题标题】：MySQL NOT IN Query much slower after Mysql UpgradeMySQL 升级后 MySQL NOT IN 查询慢得多
【发布时间】：2014-01-12 12:41:29
【问题描述】：

这是一个非常简单的查询：

SELECT * FROM temp_company WHERE number NOT IN (SELECT number FROM company)

之前需要 15 分钟，但这是在缓冲池大小太低的 Mysql 安装上，15 分钟可以，因为这是每月任务。我升级到 Mysql 5.7（从 5.1 或 5.2 之类的东西），因为原始安装是 32 位，我无法将 innodb 缓冲池大小增加到该数据库所需的最低 10gb（我在一台机器上将其设置为 16GB） 32GB RAM。我现在已经在一个月后运行此查询，6 小时后它仍在运行。

上面的解释是：

id | select_type        | table        | partitions | type  | possible_keys | key    | key_len | ref | rows    | filtered | Extra       |
1  | PRIMARY            | temp_company |            | ALL   |               |        |         |     | 3226661 | 100.00   | Using where |
2  | DEPENDENT SUBQUERY | company      |            | index | number        | number | 33      |     | 3383517 | 100.00   | Using where |

company 和 temp_company 上的 PRIMARY 索引是 id，但 number 是它们匹配的值，并且两者都是 KEY，但以上是否表明它没有使用 temp_company 表的索引？

我想尝试的另一个逻辑查询是：

EXPLAIN SELECT tc.* FROM temp_company tc
LEFT JOIN company c on c.number = tc.number
WHERE c.number IS NULL

这同样慢，解释是：

id | select_type | table | partitions | type  | possible_keys | key    | key_len | ref | rows    | filtered | Extra                                                           |
1  | SIMPLE      | tc    |            | ALL   |               |        |         |     | 3226661 | 100.00   |                                                                 |
2  | SIMPLE      | c     |            | index | number        | number | 33      |     | 3383517 | 100.00   | Using where; Ising index; Using join buffer (block nested loop) |

任何帮助将不胜感激。也许 Mysql 改变了它查找索引的方式？

**更新 1--------

显示创建的：公司

   CREATE TABLE `company` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `count_telephone` mediumint(8) unsigned NOT NULL,
  `count_fax` mediumint(8) unsigned NOT NULL,
  `count_person` mediumint(8) unsigned NOT NULL,
  `person_date` date DEFAULT NULL COMMENT 'Date the company_person relation was updated',
  `count_email_address` mediumint(8) unsigned NOT NULL,
  `name` varchar(255) DEFAULT NULL,
  `url` varchar(255) DEFAULT NULL,
  `url_date` date DEFAULT NULL,
  `url_status` smallint(5) unsigned NOT NULL DEFAULT '0' COMMENT 'Failure count for crawling the URL',
  `website_stamp_start` int(10) unsigned DEFAULT NULL,
  `website_stamp` int(10) unsigned DEFAULT NULL,
  `ch_url` varchar(255) DEFAULT NULL COMMENT 'Companies house URL',
  `keywords_stamp_start` int(10) unsigned DEFAULT NULL,
  `keywords_stamp` int(11) DEFAULT NULL,
  `number` varchar(30) CHARACTER SET ascii COLLATE ascii_bin DEFAULT NULL,
  `category` varchar(255) DEFAULT NULL,
  `status` varchar(255) DEFAULT NULL,
  `status_date` date DEFAULT NULL COMMENT 'Date the status field was updated',
  `country_of_origin` varchar(80) DEFAULT NULL,
  `dissolution_date` date DEFAULT NULL,
  `incorporation_date` date DEFAULT NULL,
  `account_ref_day` smallint(5) unsigned DEFAULT NULL,
  `account_ref_month` smallint(5) unsigned DEFAULT NULL,
  `account_next_due_date` date DEFAULT NULL,
  `account_last_made_up_date` date DEFAULT NULL,
  `account_category` varchar(255) DEFAULT NULL,
  `returns_next_due_date` date DEFAULT NULL,
  `returns_last_made_up_date` date DEFAULT NULL,
  `mortgages_num_charges` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_outstanding` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_part_satisfied` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_satisfied` smallint(5) unsigned DEFAULT NULL,
  `partnerships_num_gen_partners` smallint(5) unsigned DEFAULT NULL,
  `partnerships_num_lim_partners` smallint(5) unsigned DEFAULT NULL,
  `ext_name` varchar(255) DEFAULT NULL,
  `turnover` decimal(18,2) DEFAULT NULL,
  `turnover_date` date DEFAULT NULL,
  `trade_debtors` decimal(18,2) DEFAULT NULL,
  `other_debtors` decimal(18,2) DEFAULT NULL,
  `debtors_date` date DEFAULT NULL,
  `real_turnover_band` int(11) DEFAULT NULL,
  `est_turnover_band` int(11) DEFAULT NULL,
  `ext_address_date` date DEFAULT NULL,
  `care_of` varchar(255) DEFAULT NULL,
  `po_box` varchar(60) DEFAULT NULL,
  `line_1` varchar(255) DEFAULT NULL,
  `line_2` varchar(255) DEFAULT NULL,
  `town` varchar(60) DEFAULT NULL,
  `county` varchar(60) DEFAULT NULL,
  `country` varchar(60) DEFAULT NULL,
  `post_code` varchar(20) DEFAULT NULL,
  `DirScrapeID` int(10) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `homepage_keywords_stamp` (`keywords_stamp`),
  KEY `number` (`number`),
  KEY `url` (`url`),
  KEY `town` (`town`),
  KEY `county` (`county`),
  KEY `post_code` (`post_code`),
  KEY `name` (`name`),
  KEY `website_stamp` (`website_stamp`),
  KEY `website_stamp_start` (`website_stamp_start`),
  KEY `keywords_stamp_start` (`keywords_stamp_start`),
  KEY `turnover` (`turnover`),
  KEY `status` (`status`),
  KEY `category` (`category`),
  KEY `incorporation_date` (`incorporation_date`),
  KEY `real_turnover_band` (`real_turnover_band`),
  KEY `est_turnover_band` (`est_turnover_band`)
) ENGINE=InnoDB AUTO_INCREMENT=3706459 DEFAULT CHARSET=utf8

临时公司：

CREATE TABLE `temp_company` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(255) DEFAULT NULL,
  `url` varchar(255) DEFAULT NULL,
  `ch_url` varchar(255) DEFAULT NULL,
  `number` varchar(30) DEFAULT NULL,
  `category` varchar(255) DEFAULT NULL,
  `status` varchar(255) DEFAULT NULL,
  `country_of_origin` varchar(80) DEFAULT NULL,
  `dissolution_date` date DEFAULT NULL,
  `incorporation_date` date DEFAULT NULL,
  `account_ref_day` smallint(5) unsigned DEFAULT NULL,
  `account_ref_month` smallint(5) unsigned DEFAULT NULL,
  `account_next_due_date` date DEFAULT NULL,
  `account_last_made_up_date` date DEFAULT NULL,
  `account_category` varchar(255) DEFAULT NULL,
  `returns_next_due_date` date DEFAULT NULL,
  `returns_last_made_up_date` date DEFAULT NULL,
  `mortgages_num_charges` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_outstanding` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_part_satisfied` smallint(5) unsigned DEFAULT NULL,
  `mortgages_num_satisfied` smallint(5) unsigned DEFAULT NULL,
  `partnerships_num_gen_partners` smallint(5) unsigned DEFAULT NULL,
  `partnerships_num_lim_partners` smallint(5) unsigned DEFAULT NULL,
  `ext_name` varchar(255) DEFAULT NULL,
  `turnover` decimal(18,2) DEFAULT NULL,
  `turnover_date` date DEFAULT NULL,
  `trade_debtors` decimal(18,2) DEFAULT NULL,
  `other_debtors` decimal(18,2) DEFAULT NULL,
  `debtors_date` date DEFAULT NULL,
  `real_turnover_band` int(11) DEFAULT NULL,
  `est_turnover_band` int(11) DEFAULT NULL,
  `ext_address_date` date DEFAULT NULL,
  `care_of` varchar(255) DEFAULT NULL,
  `po_box` varchar(60) DEFAULT NULL,
  `line_1` varchar(255) DEFAULT NULL,
  `line_2` varchar(255) DEFAULT NULL,
  `town` varchar(60) DEFAULT NULL,
  `county` varchar(60) DEFAULT NULL,
  `country` varchar(60) DEFAULT NULL,
  `post_code` varchar(20) DEFAULT NULL,
  `sic_code` varchar(10) DEFAULT NULL,
  `DirScrapeID` int(10) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `number` (`number`),
  KEY `status` (`status`),
  KEY `name` (`name`),
  KEY `sic_code` (`sic_code`)
) ENGINE=InnoDB AUTO_INCREMENT=3297833 DEFAULT CHARSET=utf8

更新 2：查询配置文件（限制为 5）

+-------------------------------+----------+
| Status                        | Duration |
+-------------------------------+----------+
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000111 |
| executing                     | 0.000001 |
| Sending data                  | 0.000110 |
| executing                     | 0.000001 |
| Sending data                  | 0.000110 |
| executing                     | 0.000001 |
| Sending data                  | 0.000110 |
| executing                     | 0.000001 |
| Sending data                  | 0.000111 |
| executing                     | 0.000001 |
| Sending data                  | 0.000111 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000112 |
| executing                     | 0.000001 |
| Sending data                  | 0.000113 |
| executing                     | 0.000001 |
| Sending data                  | 0.000114 |
| executing                     | 0.000001 |
| Sending data                  | 0.000114 |
| executing                     | 0.000001 |
| Sending data                  | 0.000114 |
| executing                     | 0.000001 |
| Sending data                  | 0.000115 |
| executing                     | 0.000001 |
| Sending data                  | 0.000116 |
| executing                     | 0.000001 |
| Sending data                  | 0.000115 |
| executing                     | 0.000001 |
| Sending data                  | 0.000115 |
| executing                     | 0.000001 |
| Sending data                  | 0.000116 |
| executing                     | 0.000001 |
| Sending data                  | 0.000116 |
| executing                     | 0.000001 |
| Sending data                  | 0.000115 |
| executing                     | 0.000001 |
| Sending data                  | 0.000115 |
| executing                     | 0.000001 |
| Sending data                  | 0.000116 |
| executing                     | 0.000001 |
| Sending data                  | 0.000116 |
| executing                     | 0.000001 |
| Sending data                  | 0.000117 |
| executing                     | 0.000001 |
| Sending data                  | 0.000117 |
| executing                     | 0.000001 |
| Sending data                  | 0.000117 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000118 |
| executing                     | 0.000001 |
| Sending data                  | 0.000120 |
| executing                     | 0.000001 |
| Sending data                  | 0.000120 |
| executing                     | 0.000001 |
| Sending data                  | 0.000121 |
| executing                     | 0.000001 |
| Sending data                  | 0.000123 |
| executing                     | 0.000001 |
| Sending data                  | 0.000121 |
| executing                     | 0.000001 |
| Sending data                  | 0.000120 |
| executing                     | 0.000001 |
| Sending data                  | 0.000121 |
| executing                     | 0.000001 |
| Sending data                  | 0.000121 |
| executing                     | 0.000001 |
| Sending data                  | 0.000121 |
| executing                     | 0.000001 |
| Sending data                  | 0.000122 |
| executing                     | 0.000001 |
| Sending data                  | 0.000123 |
| executing                     | 0.000001 |
| Sending data                  | 0.000124 |
| executing                     | 0.000001 |
| Sending data                  | 1.063880 |
| end                           | 0.000009 |
| query end                     | 0.000008 |
| closing tables                | 0.000009 |
| freeing items                 | 0.000007 |
| Waiting for query cache lock  | 0.000002 |
| freeing items                 | 0.000062 |
| Waiting for query cache lock  | 0.000002 |
| freeing items                 | 0.000001 |
| storing result in query cache | 0.000002 |
| cleaning up                   | 0.000028 |
+-------------------------------+----------+

【问题讨论】：

你的表有多少行？
请为您的表提供 SHOW CREATE 语句
@Bojangles 330 万左右
@strawberry 更新了 create's
尝试运行ANALYZE：ANALYZE TABLE company, temp_company

标签： mysql sql mysql-5.7

【解决方案1】：

我不知道为什么它突然运行得更慢了，但我建议转换为 join，它应该会表现更好：

SELECT t.*
FROM temp_company t
LEFT JOIN company c ON c.number = t.number
WHERE c.number is null

这是一种通过连接处理not in (...) 的相当标准的方法，并且之所以有效，是因为不匹配的外部连接在连接表的列中有空值。

【讨论】：

虽然这不是关键问题，我在原始问题中包含了这个查询，但你是对的，它比使用 'not in' 2.7s 和 3.3s 略快

【解决方案2】：

原来问题在于 temp_company 表编号字段没有像 Company 表那样将 ascii_bin 设置为其排序规则。

正如 MySQL 论坛 (http://forums.mysql.com/read.php?24,603620,603732#msg-603732) 上所解释的，具有不同排序规则或字符集的 varchar 字段被视为属于不同类型，因此它们之间无法使用索引。

解决方法是在 temp_company 表的 number 字段上设置相同的排序规则。然后查询耗时 3.3 秒（使用左连接方法为 2.7 秒）。

【讨论】：

【解决方案3】：

与以前不同，MySQL 现在可能未优化。你应该看看配置文件并让您的系统管理员对其进行优化。还有，还有可能是您正在运行的 Web 应用程序的潜在问题和 MySQL 5，例如一些旧插件无法正常工作。

【讨论】：

在 mysql 命令行下的查询速度是一样的——遗憾的是我也是系统管理员，看不到任何有问题的配置变量。我已经更新了可能显示一些亮点的查询配置文件。