运行缓慢的 MySQL 查询 - 如何优化？答案

【问题标题】：Slow running MySQL query - how to optimize?运行缓慢的 MySQL 查询 - 如何优化？
【发布时间】：2017-09-22 13:44:14
【问题描述】：

我正在运行一个查询，并且很难理解为什么 MySQL 运行它的速度如此之慢。我在以下 3 个场景中包含了实际查询和 EXPLAIN。

第一个查询运行速度非常慢（23 秒），但从 where 子句中仅删除一个字段（查询 2）会导致查询在 0.010 秒内运行。这是一个 TinyInt 字段（本质上存储了一个布尔值 0/1）。

项目 -> hasMany -> 里程碑 -> hasMany -> 任务

task_wishlist 是任务和愿望清单之间的数据透视表

查询 1

SELECT `tasks`.*,
        task_wishlist.description AS item_description,
        task_wishlist.created_at AS item_created_at
FROM `tasks`
LEFT JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id`
LEFT JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id`
LEFT JOIN `projects` ON `milestones`.`project_id` = `projects`.`id`
WHERE `task_wishlist`.`wishlist_id` = '527021'
  AND `tasks`.`active` = '1'
  AND `projects`.`active` = '1'
  AND `milestones`.`active` = '1'
ORDER BY `task_wishlist`.`created_at` DESC
LIMIT 25;
/* Affected rows: 0  Found rows: 25  Warnings: 0  Duration for 1 query: 23.072 sec. (+ 0.040 sec. network) */

查询 1 解释：

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: projects
   partitions: NULL
         type: ALL
possible_keys: PRIMARY
          key: NULL
      key_len: NULL
          ref: NULL
         rows: 997
     filtered: 10.00
        Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: milestones
   partitions: NULL
         type: ref
possible_keys: PRIMARY,milestones_project_id_foreign
          key: milestones_project_id_foreign
      key_len: 4
          ref: fusion.projects.id
         rows: 3
     filtered: 10.00
        Extra: Using index condition; Using where
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: tasks
   partitions: NULL
         type: ref
possible_keys: PRIMARY,tasks_milestone_id_foreign
          key: tasks_milestone_id_foreign
      key_len: 5
          ref: fusion.milestones.id
         rows: 5
     filtered: 10.00
        Extra: Using where
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: task_wishlist
   partitions: NULL
         type: ref
possible_keys: task_wishlist_wishlist_id_foreign,task_wishlist_task_id_foreign
          key: task_wishlist_task_id_foreign
      key_len: 4
          ref: fusion.tasks.id
         rows: 100
     filtered: 0.28
        Extra: Using where
4 rows in set, 1 warning (0.01 sec)

查询 2（删除里程碑.active）

SELECT `tasks`.*,
       task_wishlist.description AS item_description,
       task_wishlist.created_at AS item_created_at
FROM `tasks`
LEFT JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id`
LEFT JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id`
LEFT JOIN `projects` ON `milestones`.`project_id` = `projects`.`id`
WHERE `task_wishlist`.`wishlist_id` = '527021'
  AND `tasks`.`active` = '1'
  AND `projects`.`active` = '1'
  /*AND `milestones`.`active` = '1'*/
ORDER BY `task_wishlist`.`created_at` DESC
LIMIT 25;
/* Affected rows: 0  Found rows: 25  Warnings: 0  Duration for 1 query: 0.028 sec. (+ 0.010 sec. network) */

查询 2 解释：

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: task_wishlist
   partitions: NULL
         type: ref
possible_keys: task_wishlist_wishlist_id_foreign,task_wishlist_task_id_foreign
          key: task_wishlist_wishlist_id_foreign
      key_len: 4
          ref: const
         rows: 7224
     filtered: 100.00
        Extra: Using index condition; Using where; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: tasks
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,tasks_milestone_id_foreign
          key: PRIMARY
      key_len: 4
          ref: fusion.task_wishlist.task_id
         rows: 1
     filtered: 10.00
        Extra: Using where
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: milestones
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,milestones_project_id_foreign
          key: PRIMARY
      key_len: 4
          ref: fusion.tasks.milestone_id
         rows: 1
     filtered: 100.00
        Extra: NULL
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: projects
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: fusion.milestones.project_id
         rows: 1
     filtered: 10.00
        Extra: Using where
4 rows in set, 1 warning (0.00 sec)

查询 3：删除 projects.active = 1

SELECT `tasks`.*,
       task_wishlist.description AS item_description,
       task_wishlist.created_at AS item_created_at
FROM `tasks`
LEFT JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id`
LEFT JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id`
LEFT JOIN `projects` ON `milestones`.`project_id` = `projects`.`id`
WHERE `task_wishlist`.`wishlist_id` = '527021'
  AND `tasks`.`active` = '1'
  /*AND `projects`.`active` = '1'*/
  AND `milestones`.`active` = '1'
ORDER BY `task_wishlist`.`created_at` DESC
LIMIT 25;
/* Affected rows: 0  Found rows: 25  Warnings: 0  Duration for 1 query: 0.027 sec. (+ 4.031 sec. network) */

查询 3 解释

*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: task_wishlist
   partitions: NULL
         type: ref
possible_keys: task_wishlist_wishlist_id_foreign,task_wishlist_task_id_foreign
          key: task_wishlist_wishlist_id_foreign
      key_len: 4
          ref: const
         rows: 7224
     filtered: 100.00
        Extra: Using index condition; Using where; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: tasks
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY,tasks_milestone_id_foreign
          key: PRIMARY
      key_len: 4
          ref: fusion.task_wishlist.task_id
         rows: 1
     filtered: 10.00
        Extra: Using where
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: milestones
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: fusion.tasks.milestone_id
         rows: 1
     filtered: 10.00
        Extra: Using where
*************************** 4. row ***************************
           id: 1
  select_type: SIMPLE
        table: projects
   partitions: NULL
         type: eq_ref
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 4
          ref: fusion.milestones.project_id
         rows: 1
     filtered: 100.00
        Extra: Using index
4 rows in set, 1 warning (0.00 sec)

在包含所有 active 字段的情况下，我该怎么做才能使此查询正常运行？

【问题讨论】：

您是否尝试为这些字段中的每一个创建任何非唯一索引以查看是否有帮助？
这种解释格式很难阅读，但是 MySQL 处理的行数很少。这意味着可能存在 I/O 问题。您是否正在运行机械硬盘和默认 MySQL 配置？ MySQL 中还有一个额外的分析机制，称为profiling。 SET PROFILING = 1; Your query goes here ; SHOW PROFILE FOR QUERY 1; SET PROFILING = 0;。这将向您展示 MySQL 在幕后执行的所有步骤。瞎猜：你的innodb_buffer_pool_size太低了，所以MySQL需要等待HDD完成I/O，比较慢。
表和索引定义是什么？
我的理解是布尔类型字段的索引没有任何好处。没有 I/O 问题 - 这是一个 mysql 优化问题 - 注意第一个解释中的 using temporary，但不是其他的。
@JonoB - 是的，索引布尔值通常是没用的。然而，以布尔开头的“复合”索引可能是有益的。（我在 this 问题中没有看到这样的例子。）

标签： mysql sql query-optimization

【解决方案1】：

您需要了解的一个事实是，您的查询实际上是在执行简单的 INNER JOIN 而不是 LEFT JOINS。
Left outer join 基本上与内部连接的工作方式相同，但如果连接条件在“右”表中找不到任何匹配的行，它会返回 NULL。
但是查询中的所有 NULL 都会被 WHERE 子句中的条件过滤掉，请仔细检查查询：

FROM `tasks`
LEFT JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id`
LEFT JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id`
LEFT JOIN `projects` ON `milestones`.`project_id` = `projects`.`id`
WHERE `task_wishlist`.`wishlist_id` = '527021'  
  AND `tasks`.`active` = '1'  
  AND `milestones`.`active` = '1'
  AND `projects`.`active` = '1'

来自LEFT JOIN task_wishlist 的空值被WHERE task_wishlist.wishlist_id = '527021' 过滤掉
来自LEFT JOIN milestones 的空值被WHERE milestones.active = '1' 过滤掉
来自LEFT JOIN projects 的空值被WHERE projects.active = '1' 过滤掉

MySql 意识到了这一点，它在内部将您的查询重写为 INNER JOIN 查询：

FROM `tasks`
INNER JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id`
INNER JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id`
INNER JOIN `projects` ON `milestones`.`project_id` = `projects`.`id`
WHERE `task_wishlist`.`wishlist_id` = '527021'  
  AND `tasks`.`active` = '1'  
  AND `milestones`.`active` = '1'
  AND `projects`.`active` = '1'

如果您从 WHERE 子句中删除（注释）某些条件，这会“激活”查询的相应 LEFT JOIN 部分，因此查询的语义会发生变化，结果集也会发生变化。比较此类查询的性能没有任何意义，因为坦率地说它们是不同的，实际上您是在比较苹果和橘子。

由于查询是进行简单的 INNER JOIN，您可以将其重写为：

FROM `tasks`
INNER JOIN `task_wishlist` ON `tasks`.`id` = `task_wishlist`.`task_id` 
                                         AND `task_wishlist`.`wishlist_id` = '527021' 

INNER JOIN `milestones` ON `tasks`.`milestone_id` = `milestones`.`id` 
                                                AND `milestones`.`active` = '1'

INNER JOIN `projects` ON `milestones`.`project_id` = `projects`.`id` 
                                                 AND `projects`.`active` = '1'

现在在上述条件下，您可以轻松地看到在连接期间使用的“对”列：

projects.id + projects.active
milestones.id + landmarks.active
task_wishlist.task_id + task_wishlist.wishlist_id

尝试在“对”上添加多列索引，例如projects 表：

CREATE INDEX xxx ON projects( id, active )

如果id 列也是主键，则可能只在单个active 列而不是多列(id, active) 上添加索引，因为in InnoDB, each record in a secondary index contains the primary key columns for the row, as well as the columns specified for the secondary index.
你必须自己试验。

【讨论】：

从哲学上讲，ON 应该只说明这些表是如何相互关联的。 WHERE 应该包含任何过滤。对于INNER JOIN，优化器不在乎。对于LEFT JOIN，存在语义差异。

【解决方案2】：

你说task_wishlist是tasks和wishlists之间的多对多映射表？我敢打赌它没有像样的索引。请按照此处的提示操作：

http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table

如果这还不够，请为每个表格返回SHOW CREATE TABLE。

【讨论】：