为什么 MySQL 5.7 左连接到空集会导致查询执行更差？

Question

我有一个查询将一个表连接到一个子查询，尽管子查询由于 where 条件返回一个空集，但即使我考虑到它所花费的时间，整个查询在连接时仍然比没有它时花费的时间要长得多要运行的子查询。任何想法为什么会发生这种情况？详情如下

询问：

select Address.*
from Address
left join (
    select lotNumber, max(jobId) as id
    from Address
    where jobId is not null
    group by lotNumber
) latestJob on latestJob.lotNumber = Address.lotNumber

架构：

CREATE TABLE `Address` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `streetNumber` varchar(45) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `street` varchar(45) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `lotNumber` varchar(45) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
  `jobId` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `idx_Address_lotNumber` (`lotNumber`)
) ENGINE=InnoDB AUTO_INCREMENT=1032717 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;

解释：

+----+-------------+-----------------+------------+-------+-------------------------------+-------------------------------+---------+---------------------------+---------+----------+-------------+
| id | select_type | table           | partitions | type  | possible_keys                 | key                           | key_len | ref                       | rows    | filtered | Extra       |
+----+-------------+-----------------+------------+-------+-------------------------------+-------------------------------+---------+---------------------------+---------+----------+-------------+
|  1 | PRIMARY     | Address         | NULL       | ALL   | NULL                          | NULL                          | NULL    | NULL                      | 1027850 |   100.00 | NULL        |
|  1 | PRIMARY     | <derived2>      | NULL       | ref   | <auto_key0>                   | <auto_key0>                   | 183     | Address.lotNumber         |      10 |   100.00 | NULL        |
|  2 | DERIVED     | Address         | NULL       | index | idx_Address_lotNumber         | idx_Address_lotNumber         | 183     | NULL                      | 1027850 |    90.00 | Using where |
+----+-------------+-----------------+------------+-------+-------------------------------+-------------------------------+---------+---------------------------+---------+----------+-------------+

目前Address 表有大约1M 条记录，但jobId 对所有这些记录都是空的，因此左连接子查询返回一个空集。

子查询自行运行大约需要 0.07 秒，但整个查询需要大约 2.22 秒。没有子查询的查询大约需要 0.07 秒。似乎当加入一个空集时，整个查询应该只需要 ~0.07 + ~0.07 = ~0.14 秒，那么额外的 2 秒来自哪里？连接操作中似乎发生了一些低效的事情。无论如何要改善这一点？

谢谢

Answer 1

简而言之：为您的jobId 列添加一个索引。我有一个名为proctable 的类似表，它通过一个过程生成了1.4 m 行。虽然表名和列名与您的不同，但结构相似（省略了不必要的列）：

 CREATE TABLE `proctable` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `num` int(11) DEFAULT NULL,
  `jid` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `idx` (`num`)
) ENGINE=InnoDB AUTO_INCREMENT=1429159 DEFAULT CHARSET=latin1

id 的值范围从 1 到 1429158，而 num 是使用函数 rand() 随机生成的，从 1 到 10000。jid 将所有行设置为空。原始查询（没有索引列jid）执行如下：

select * 
from proctable t1
left join (select  num, max(jid) as id
    from proctable
    where jid is not null
    group by num
) t2 on t1.num = t2.num;
-- 1429158 rows in set (7.08 sec)   

explain select * 
from proctable t1
left join (select  num, max(jid) as id
    from proctable
    where jid is not null
    group by num
) t2 on t1.num = t2.num;

-- here is the execution plan :

| id | select_type | table      | partitions | type  | possible_keys | key         | key_len | ref           | rows    | filtered | Extra       |
+----+-------------+------------+------------+-------+---------------+-------------+---------+---------------+---------+----------+-------------+
|  1 | PRIMARY     | t1         | NULL       | ALL   | NULL          | NULL        | NULL    | NULL          | 1426920 |   100.00 | NULL        |
|  1 | PRIMARY     | <derived2> | NULL       | ref   | <auto_key0>   | <auto_key0> | 5       | testdb.t1.num |      10 |   100.00 | NULL        |
|  2 | DERIVED     | proctable  | NULL       | index | idx           | idx         | 5       | NULL          | 1426920 |    90.00 | Using where |

然后我们索引jid 列并再次执行此操作。

create index jidx on proctable(jid);

-- the execution plan has changed:
explain select *  from proctable t1 left join (select  num, max(jid) as id     from proctable     where jid is not null     group by num ) t2 on t1.num = t2.num;
+----+-------------+------------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------------------------------------+
| id | select_type | table      | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                                                  |
+----+-------------+------------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------------------------------------+
|  1 | PRIMARY     | t1         | NULL       | ALL   | NULL          | NULL | NULL    | NULL | 1426920 |   100.00 | NULL                                                   |
|  1 | PRIMARY     | <derived2> | NULL       | ALL   | NULL          | NULL | NULL    | NULL |       2 |   100.00 | Using where; Using join buffer (Block Nested Loop)     |
|  2 | DERIVED     | proctable  | NULL       | range | idx,jidx      | jidx | 5       | NULL |       1 |   100.00 | Using index condition; Using temporary; Using filesort |

通过使用 jidx 作为键，使用范围作为类型而不是索引的 DERIVED 已将预期的行数从 1+ 百万大幅减少到 1。让我们执行查询。

select * 
from proctable t1
left join (select  num, max(jid) as id
    from proctable
    where jid is not null
    group by num
) t2 on t1.num = t2.num;
-- 1429158 rows in set (3.73 sec)

注意：这是在我的实验室虚拟机上完成的，它只有一个 4G 的物理内存。通过为 jid 添加索引，它几乎将响应时间减少了一半。此外，如果将外连接更改为内连接或直连接，差异很大。

select *  from proctable t1   join (select  num, max(jid) as id     from proctable     where jid is not null     group by num ) t2 on t1.num = t2.num;
-- Empty set (0.00 sec)

explain  select *  from proctable t1   join (select  num, max(jid) as id     from proctable     where jid is not null     group by num ) t2 on t1.num = t2.num;
+----+-------------+------------+------------+-------+---------------+------+---------+--------+------+----------+--------------------------------------------------------+
| id | select_type | table      | partitions | type  | possible_keys | key  | key_len | ref    | rows | filtered | Extra                                                  |
+----+-------------+------------+------------+-------+---------------+------+---------+--------+------+----------+--------------------------------------------------------+
|  1 | PRIMARY     | <derived2> | NULL       | ALL   | NULL          | NULL | NULL    | NULL   |    2 |   100.00 | Using where                                            |
|  1 | PRIMARY     | t1         | NULL       | ref   | idx           | idx  | 5       | t2.num |  146 |   100.00 | NULL                                                   |
|  2 | DERIVED     | proctable  | NULL       | range | jidx,idx      | jidx | 5       | NULL   |    1 |   100.00 | Using index condition; Using temporary; Using filesort |

explain  select *  from proctable t1   straight_join (select  num, max(jid) as id     from proctable     where jid is not null     group by num ) t2 on t1.num = t2.num;
+----+-------------+------------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------------------------------------+
| id | select_type | table      | partitions | type  | possible_keys | key  | key_len | ref  | rows    | filtered | Extra                                                  |
+----+-------------+------------+------------+-------+---------------+------+---------+------+---------+----------+--------------------------------------------------------+
|  1 | PRIMARY     | t1         | NULL       | ALL   | idx           | NULL | NULL    | NULL | 1426920 |   100.00 | NULL                                                   |
|  1 | PRIMARY     | <derived2> | NULL       | ALL   | NULL          | NULL | NULL    | NULL |       2 |    50.00 | Using where; Using join buffer (Block Nested Loop)     |
|  2 | DERIVED     | proctable  | NULL       | range | jidx,idx      | jidx | 5       | NULL |       1 |   100.00 | Using index condition; Using temporary; Using filesort |

select *  from proctable t1   straight_join (select  num, max(jid) as id     from proctable     where jid is not null     group by num ) t2 on t1.num = t2.num;
-- Empty set (2.91 sec)