为什么这条 SQL 会生成临时表，而且运行这么慢？答案

【问题标题】：Why is this SQL generating a temporary table and running so slow?为什么这条 SQL 会生成临时表，而且运行这么慢？
【发布时间】：2010-10-14 18:57:43
【问题描述】：

我的 Rails 应用程序生成了以下 SQL，它试图从市场应用程序和 mysql 中获取所有具有实时广告的汽车模型的列表：

SELECT `models`.* FROM `models` 
  INNER JOIN `autos` ON autos.model_id = models.id 
  INNER JOIN `ads` ON `ads`.id = `autos`.ad_id 
WHERE (ads.ad_status_id = 4 AND pub_start_date < NOW() AND pub_end_date > NOW() AND models.manufacturer_id = 50 ) 
GROUP BY models.id ORDER BY models.name;

当我运行解释时，我得到的是：

Id  1   1   1
Select Type SIMPLE  SIMPLE  SIMPLE
Table   models  autos   ads
Type    ref ref eq_ref
Possible Keys   PRIMARY,manufacturer_id model_id,ad_id  PRIMARY,quick_search,ad_status_id
Key manufacturer_id model_id    PRIMARY
Key Length  5   4   4
Ref const   concept_development.models.id   concept_development.autos.ad_id
Rows    70  205 1
Extra   Using where; Using temporary; Using filesort    Using where Using where

我不明白为什么查询会生成一个临时表/使用文件排序 - 所有引用的键都是索引。这几天一直在尝试解决这个问题，但一无所获。

非常感谢任何帮助！

解释模型：

+---------------------+-------------+------+-----+---------+----------------+
| Field               | Type        | Null | Key | Default | Extra          |
+---------------------+-------------+------+-----+---------+----------------+
| id                  | int(11)     | NO   | PRI | NULL    | auto_increment |
| name                | varchar(32) | YES  |     | NULL    |                |
| manufacturer_id     | int(11)     | YES  | MUL | NULL    |                |
| vehicle_category_id | int(11)     | NO   | MUL | 1       |                |
| synonym_names       | longtext    | YES  |     | NULL    |                |
+---------------------+-------------+------+-----+---------+----------------+

显示模型的索引：

+--------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table  | Non_unique | Key_name            | Seq_in_index | Column_name         | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+
| models |          0 | PRIMARY             |            1 | id                  | A         |        2261 |     NULL | NULL   |      | BTREE      |         |
| models |          1 | manufacturer_id     |            1 | manufacturer_id     | A         |         205 |     NULL | NULL   | YES  | BTREE      |         |
| models |          1 | vehicle_category_id |            1 | vehicle_category_id | A         |           7 |     NULL | NULL   |      | BTREE      |         |
+--------+------------+---------------------+--------------+---------------------+-----------+-------------+----------+--------+------+------------+---------+

模型表状态：

+--------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+
| Name   | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time         | Check_time          | Collation         | Checksum | Create_options | Comment |
+--------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+
| models | MyISAM |      10 | Dynamic    | 2261 |             26 |       61000 | 281474976710655 |        84992 |         0 |           2751 | 2010-09-28 18:42:45 | 2010-09-28 18:42:45 | 2010-09-28 18:44:00 | latin1_swedish_ci |     NULL |                |         |
+--------+--------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-------------------+----------+----------------+---------+

解释广告

+------------------+--------------------------+------+-----+---------------------+----------------+
| Field            | Type                     | Null | Key | Default             | Extra          |
+------------------+--------------------------+------+-----+---------------------+----------------+
| id               | int(10)                  | NO   | PRI | NULL                | auto_increment |
| fp_urn           | int(10)                  | NO   | MUL | 0                   |                |
| user_id          | int(10)                  | NO   | MUL | 0                   |                |
| ad_status_id     | int(3) unsigned          | NO   | MUL | 1                   |                |
| style_id         | int(10)                  | NO   |     | 3                   |                |
| search_tags      | varchar(255)             | YES  |     | NULL                |                |
| title            | varchar(255)             | NO   |     |                     |                |
| description      | text                     | YES  |     | NULL                |                |
| currency         | enum('EUR','GBP')        | NO   |     | EUR                 |                |
| price            | decimal(8,2)             | NO   | MUL | 0.00                |                |
| proposal_type    | enum('Offered','Wanted') | NO   |     | Offered             |                |
| category_id      | int(10)                  | YES  |     | 0                   |                |
| contact          | varchar(50)              | NO   | MUL |                     |                |
| area_id          | int(10)                  | NO   |     | 0                   |                |
| origin_id        | int(10)                  | NO   |     | 0                   |                |
| reject_reason_id | int(3)                   | NO   |     | 0                   |                |
| date_created     | timestamp                | NO   |     | 0000-00-00 00:00:00 |                |
| last_modified    | timestamp                | NO   |     | CURRENT_TIMESTAMP   |                |
| pub_start_date   | datetime                 | YES  |     | 0000-00-00 00:00:00 |                |
| pub_end_date     | datetime                 | YES  |     | 0000-00-00 00:00:00 |                |
| bumped_up_date   | datetime                 | YES  |     | 0000-00-00 00:00:00 |                |
| state            | smallint(6)              | YES  |     | NULL                |                |
| eproofed         | tinyint(1)               | NO   |     | 0                   |                |
| is_featured      | int(1)                   | NO   |     | 0                   |                |
| num_featured_imp | int(10)                  | YES  |     | 0                   |                |
| num_direct_imp   | int(10)                  | YES  |     | 0                   |                |
| is_top_listed    | int(1)                   | NO   |     | 0                   |                |
| delta            | tinyint(1)               | NO   |     | 0                   |                |
| ext_ref_id       | varchar(50)              | YES  |     | NULL                |                |
| email_seller     | tinyint(1)               | YES  |     | 1                   |                |
| sort_by          | int(10)                  | YES  |     | 8                   |                |
| permalink        | varchar(500)             | YES  |     | NULL                |                |
| external_url     | varchar(255)             | YES  |     | NULL                |                |
+------------------+--------------------------+------+-----+---------------------+----------------+

显示来自 concept_development 的表格状态，其中名称如“广告”；

+------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+
| Name | Engine | Version | Row_format | Rows   | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation       | Checksum | Create_options                                  | Comment |
+------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+
| ads  | InnoDB |      10 | Compact    | 656350 |            232 |   152748032 |               0 |     87736320 | 340787200 |        1148382 | 2010-09-29 09:55:46 | NULL        | NULL       | utf8_general_ci |     NULL | checksum=1 delay_key_write=1 row_format=DYNAMIC |         |
+------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+

显示来自广告的索引

+-------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name  | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| ads   |          0 | PRIMARY   |            1 | id           | A         |      521391 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | NewIndex1 |            1 | ad_status_id | A         |          15 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | NewIndex1 |            2 | pub_end_date | A         |      260695 |     NULL | NULL   | YES  | BTREE      |         |
| ads   |          1 | NewIndex1 |            3 | category_id  | A         |      521391 |     NULL | NULL   | YES  | BTREE      |         |
| ads   |          1 | NewIndex1 |            4 | style_id     | A         |      521391 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | NewIndex2 |            1 | user_id      | A         |      130347 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | NewIndex3 |            1 | price        | A         |        7667 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | contact   |            1 | contact      | A         |      260695 |     NULL | NULL   |      | BTREE      |         |
| ads   |          1 | fp_urn    |            1 | fp_urn       | A         |      521391 |     NULL | NULL   |      | BTREE      |         |
+-------+------------+-----------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+

解释汽车

+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-----+-------------+----------------+
| Field             | Type                                                                                                                                                                                                                                                                                                                                 | Null | Key | Default     | Extra          |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-----+-------------+----------------+
| id                | int(10)                                                                                                                                                                                                                                                                                                                              | NO   | PRI | NULL        | auto_increment |
| ad_id             | int(10)                                                                                                                                                                                                                                                                                                                              | YES  | MUL | NULL        |                |
| style_id          | int(10)                                                                                                                                                                                                                                                                                                                              | YES  | MUL | NULL        |                |
| manufacturer_id   | int(10)                                                                                                                                                                                                                                                                                                                              | NO   | MUL | NULL        |                |
| model_id          | int(10)                                                                                                                                                                                                                                                                                                                              | NO   | MUL | NULL        |                |
| registration      | varchar(10)                                                                                                                                                                                                                                                                                                                          | YES  |     | NULL        |                |
| year              | int(4)                                                                                                                                                                                                                                                                                                                               | YES  |     | NULL        |                |
| fuel_type         | enum('Petrol','Diesel')                                                                                                                                                                                                                       | NO   |     | Petrol      |                |
| colour            | varchar(75)                                                                                                                                                                                                                                                                                                                          | YES  |     | NULL        |                |
| mileage           | varchar(25)                                                                                                                                                                                                                                                                                                                          | NO   |     | Not Entered |                |
| mileage_units     | enum('mls','kms')                                                                                                                                                                                                                                                                                                                    | NO   |     | mls         |                |
| num_doors         | varchar(25)                                                                                                                                                                                                                                                                                                                          | NO   |     | Not Entered |                |
| num_owners        | int(2)                                                                                                                                                                                                                                                                                                                               | YES  |     | NULL        |                |
| engine_size       | varchar(10)                                                                                                                                                                                                                                                                                                                          | YES  |     | NULL        |                |
| transmission_type | enum('Manual','Automatic')                                                                                                                                                                                                                                                               | NO   |     | Manual      |                |
| body_type         | enum('Saloon','Hatchback')                                                                                                                                                                                                              | NO   |     | Saloon      |                |
| condition         | varchar(75)                                                                                                                                                                                                                                                                                                                          | NO   |     | NA          |                |
| extra_features    | text                                                                                                                                                                                                                                                                                                                                 | YES  |     | NULL        |                |
| tax_expiry        | varchar(7)                                                                                                                                                                                                                                                                                                                           | YES  |     | NULL        |                |
| nct_expiry        | varchar(7)                                                                                                                                                                                                                                                                                                                           | YES  |     | NULL        |                |
| variation         | text                                                                                                                                                                                                                                                                                                                                 | YES  |     | NULL        |                |
| tax_class         | enum('Agricultural','Bus') | NO   |     | Private     |                |
| co2               | int(9)                                                                                                                                                                                                                                                                                                                               | YES  |     | NULL        |                |
+-------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------+-----+-------------+----------------+

显示来自 concept_development 的表格状态，其中名称如“汽车”

+-------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+
| Name  | Engine | Version | Row_format | Rows   | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time         | Update_time | Check_time | Collation       | Checksum | Create_options                                  | Comment |
+-------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+
| autos | InnoDB |      10 | Compact    | 196168 |            136 |    26804224 |               0 |     26279936 | 340787200 |         485405 | 2010-09-17 22:09:45 | NULL        | NULL       | utf8_general_ci |     NULL | checksum=1 delay_key_write=1 row_format=DYNAMIC |         |
+-------+--------+---------+------------+--------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+-------------------------------------------------+---------+

显示来自汽车的索引；

+-------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name        | Seq_in_index | Column_name     | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| autos |          0 | PRIMARY         |            1 | id              | A         |      294937 |     NULL | NULL   |      | BTREE      |         |
| autos |          1 | ad_id           |            1 | ad_id           | A         |      294937 |     NULL | NULL   | YES  | BTREE      |         |
| autos |          1 | style_id        |            1 | style_id        | A         |          10 |     NULL | NULL   | YES  | BTREE      |         |
| autos |          1 | manufacturer_id |            1 | manufacturer_id | A         |         194 |     NULL | NULL   |      | BTREE      |         |
| autos |          1 | model_id        |            1 | model_id        | A         |         830 |     NULL | NULL   |      | BTREE      |         |
+-------+------------+-----------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+

【问题讨论】：

models.name 也被索引了吗？
刚刚添加了，同样的问题 :-(
我知道这可能会很痛苦，但我们确实需要更多信息来帮助解决这个问题。请看这里pastie.org/1221602
你能加入解释广告吗？显示来自广告的索引；解释汽车；并显示来自汽车的索引。如果你不想让你的问题变得太大，你可以粘贴结果
@gregcase: int(10) 和 int(11) 在 MySQL 内部是相同的。括号内的数字仅指输出中显示的位数。在内部，它们都是普通的 32 位整数。

标签： sql mysql ruby-on-rails activerecord

【解决方案1】：

来自 MySQL 文档：

Temporary tables can be created under conditions such as these:
    * If there is an ORDER BY clause and a different GROUP BY clause, or if the ORDER BY or GROUP BY contains columns from tables other than the first table in the join queue, a temporary table is created.

http://dev.mysql.com/doc/refman/5.1/en/internal-temporary-tables.html

【讨论】：

哦，文件排序可能来自 ORDER BY 子句，并且在 models.name 上没有索引
有没有办法重写它，这样它就不会创建 tmp 表？我刚刚在 models.name 上添加了一个索引
我可以尝试使用一些不同的索引，但我需要查看表模式。尝试使用“SHOW CREATE TABLE 模型；” （对其他 2 个表也重复此操作）。这让我更容易重新创建表格。

【解决方案2】：

将所有文本列更改为 varchar。如果您需要将它们维护为“文本”，则需要对架构进行雪花化并从该查询中排除描述表。

如果任何表中的任何列是 text 或 blob，MySQL 会自动创建磁盘临时表，而不是内存临时表。临时表本身并没有杀死你，而是将它写入磁盘的事实。

http://dev.mysql.com/doc/refman/5.1/en/internal-temporary-tables.html

某些情况会阻止使用内存中的临时表，在这种情况下，服务器会使用磁盘表来代替：

表中存在 BLOB 或 TEXT 列

【讨论】：

【解决方案3】：

您在 pub_end_date 上有一个索引，但在 pub_start_date 上没有索引，而且您的 WHERE 子句同时引用了两者。

看起来它没有使用 pub_end_date 索引，但这可能是因为它还需要检查 pub_start_date。

【讨论】：

嗨艾伦，刚刚在 pub_start_date 上添加了一个索引 - 恐怕不高兴

【解决方案4】：

这并不是在解释为什么，但是您如何重写您的查询以不使用 group by？我认为您只是加入这些表格以确保存在感兴趣的广告。那么怎么样：

SELECT `models`.* 
FROM `models` 
WHERE models.manufacturer_id = 50 
AND EXISTS (   SELECT * FROM `autos` 
               INNER JOIN `ads` ON  `ads`.id = `autos`.ad_id 
               WHERE autos.model_id = models.id 
               AND ads.ad_status_id = 4 
               AND ads.pub_start_date < NOW() 
               AND ads.pub_end_date > NOW() 
             )
ORDER BY models.name;

性能问题可能与group by有关，在这种情况下，这会提高性能。

如果你被允许在 mysql 中这样做（并且如果它在边缘情况下按照你想要的方式工作），使用and NOW() between ads.pub_start_date and ads.pub_end_date 可能看起来会更好一些。

【讨论】：