我有一个相当大的 SQL 查询被 GROUP BY 和 ORDER BY 语句减慢。优化的最佳方法是什么？

Question

优化此查询的最佳方式是什么，因为它最多需要 12 秒才能执行？

对查询的大小表示歉意，它几乎立即执行，无需 group by 和 order by 语句。

我对 SQL 优化还很陌生，我只写了一年左右的代码。这个查询不是我写的，但我需要修复它。

SELECT
`products`.*,
`product_alternative`.`alternative_product_code`,
`product_alternative`.`id` as `product_alternative_id`,
`vat_rate`.`rate` AS `vat_rate`,
`product_images`.`filename` AS `product_image_file`,
`product_docs`.`filename` AS `product_doc_file`,
`suppliers`.`supplier_code`,
`suppliers`.`name` AS `supplier_name`,
`commission`.`commission` AS `supplier_commission`,
`categories`.`name` AS `category_name`,
`sub_categories`.`name` AS `subcategory_name`,
IF(`products`.`product_doc_id` = 0,
 NULL,
 CONCAT(
"/product-docs/",
`products`.`id`
)) AS `product_doc_url`,IF(`products`.`product_image_id` = 0, NULL, CONCAT(
"/product-images/",
`products`.`id`,
"/original/"
)) AS `original_image_url`,IF(`products`.`product_image_id` = 0, NULL, CONCAT(
"/product-images/",
`products`.`id`,
"/medium/"
)) AS `medium_image_url`,IF(`products`.`product_image_id` = 0, NULL, CONCAT(
"/product-images/",
`products`.`id`,
"/thumb/"
)) AS `thumb_image_url`,CASE
WHEN
    `group_favourite`.`id` IS NOT NULL
    THEN 1 ELSE 0
END
AS `group_favourite`
,CASE
WHEN
    `product_favourite`.`org_id` != 0
    AND `product_favourite`.`org_id` IS NOT NULL
    THEN 1 ELSE 0
END
AS `favourite`
,CASE
WHEN
    `product_favourite`.`org_id` != 0
    AND `product_favourite`.`org_id` IS NOT NULL
    THEN 1 ELSE 0
END
AS `favourite`,
`product_favourite`.`needs_authorisation`,

 IF(`customer_personal_favourite`.`id` IS NOT NULL,1,0) AS `is_personal_favourite`
 FROM
`products` 

LEFT JOIN `vat_rate` ON `products`.`vat_rate_id` = `vat_rate`.`id`
LEFT JOIN `product_images` ON `products`.`product_image_id` = `product_images`.`id`
LEFT JOIN `product_docs` ON `products`.`product_doc_id` = `product_docs`.`id`

LEFT JOIN `orgs` AS `suppliers` ON `products`.`supplier_id` = `suppliers`.`id`

LEFT JOIN `commission` ON
    `suppliers`.`id` = `commission`.`org_id`
    AND `commission`.`status` = 1
    AND DATE(`commission`.`start_date`) <= DATE('2020-07-15')
    AND (
        DATE(`commission`.`end_date`) >= DATE('2020-07-15')
        OR `commission`.`end_date` = '0000-00-00'
    )

LEFT JOIN `categories` AS `categories` ON
    `products`.`category_id` = `categories`.`id`


LEFT JOIN `categories` AS `sub_categories` ON
    `products`.`sub_category_id` = `sub_categories`.`id`

LEFT JOIN `product_attribute_product` ON
    `product_attribute_product`.`product_id` = `products`.`id`
LEFT JOIN `product_alternative` ON
    `product_alternative`.`product_original_id` = `products`.`original_id`
JOIN `orgs` AS `customer` ON
`customer`.`id` IN (320)

LEFT JOIN `product_supplier_account` ON
`product_supplier_account`.`product_id` = `products`.`id`

JOIN `supplier_account` AS `default_supplier_account` ON
`default_supplier_account`.`supplier_id` = `products`.`supplier_id`
AND `default_supplier_account`.`is_default` = 1
AND `default_supplier_account`.`is_deleted` = 0

JOIN `customer_to_supplier` ON
`customer_to_supplier`.`supplier_id` = `products`.`supplier_id`
AND `customer_to_supplier`.`customer_id` IN (`customer`.`id`)
AND `customer_to_supplier`.`allow_access` = 1
AND `customer_to_supplier`.`status` = 1

JOIN `customer_to_supplier_account` ON
    `customer_to_supplier_account`.`customer_to_supplier_id` = `customer_to_supplier`.`id`
    AND (
        `customer_to_supplier_account`.`supplier_account_id` =                                    `product_supplier_account`.`supplier_account_id`
        OR `customer_to_supplier_account`.`supplier_account_id` = `default_supplier_account`.`id`
    )
    AND `customer_to_supplier_account`.`allow_access` = 1
    AND `customer_to_supplier_account`.`is_deleted` = 0
LEFT JOIN `statement_products_categories` ON
`statement_products_categories`.`product_id` = `products`.`original_id`
LEFT JOIN `product_favourite` ON
    `product_favourite`.`product_id` = `products`.`original_id`
    AND `product_favourite`.`org_id` IN (`customer`.`id`)
    LEFT JOIN `group_favourite` AS `group_favourite` ON
`products`.`original_id` = `group_favourite`.`product_id`
AND `group_favourite`.`group_id` IN (37)
LEFT JOIN `budget` AS `category_budget` ON
    `category_budget`.`org_id` IN (`customer`.`id`)
    AND `category_budget`.`budget_type_id` = 1
    AND `category_budget`.`category_id` = `products`.`category_id`
    AND
        `category_budget`.`start_date` <= CURDATE()
        AND (
            `category_budget`.`final_date` >= CURDATE()
            OR `category_budget`.`final_date` IS NULL
        )
LEFT JOIN `budget` AS `supplier_budget` ON
    `supplier_budget`.`org_id` IN (`customer`.`id`)
    AND `supplier_budget`.`budget_type_id` = 3
    AND `supplier_budget`.`supplier_id` = `products`.`supplier_id`
    AND
        `supplier_budget`.`start_date` <= CURDATE()
        AND (
            `supplier_budget`.`final_date` >= CURDATE()
            OR `supplier_budget`.`final_date` IS NULL
        )
LEFT JOIN `budget` AS `product_budget` ON
    `product_budget`.`org_id` IN (`customer`.`id`)
    AND `product_budget`.`budget_type_id` = 2
    AND
        `product_budget`.`start_date` <= CURDATE()
        AND (
            `product_budget`.`final_date` >= CURDATE()
            OR `product_budget`.`final_date` IS NULL
        )
LEFT  JOIN `customer_personal_favourite` ON
`customer_personal_favourite`.`org_id` = `customer`.`id`
AND `customer_personal_favourite`.`product_original_id` = `products`.`original_id`

WHERE
0 OR (1  AND `products`.`status` = 1  AND ((
    `products`.`parent_id` = 0
    OR `products`.`parent_id` IS NULL
)   OR (
    `product_favourite`.`org_id` IS NOT NULL
    AND `product_favourite`.`org_id` != 0
)  )  AND `products`.`id` = `products`.`current_id` )
GROUP BY `products`.`id`
ORDER BY
`favourite` DESC,

`active_date` ASC,
    `products`.`code` ASC,
    `products`.`name` ASC,
    `products`.`parent_id` ASC

 LIMIT 0, 20 

Answer 1

      AND  DATE(`commission`.`start_date`) <= DATE('2020-07-15')
      AND  ( DATE(`commission`.`end_date`) >= DATE('2020-07-15')
              OR  `commission`.`end_date` = '0000-00-00' 

有几个问题：

• OR 很难优化。想办法避免这种情况发生 - 例如删除此类行，或使用遥远的未来而不是过去的结束日期。
• 不要在函数调用中隐藏列。
• 让我们看看SHOW CREATE TABLE。
• 您不需要在文字“2020-07-15”周围使用DATE()。

另一方面，这没问题：product_budget.start_date

ORDER BY `favourite` DESC,
         `active_date` ASC,

除非您有 MySQL 8.0，否则没有索引可以处理混合了 DESC 和 ASC 的 ORDER BY。 （可能无法使用索引出于其他原因。）

WHERE 0 OR (1  AND ...

动态构建查询，而不是使用这样的组合。

LEFT JOIN ... ON ... foo = 1

ON 应该说明这些表是如何关联的。 foo = 1 感觉更像是应该在 WHERE 子句中的东西（用于过滤）。在ON 和WHERE 之间移动它可能会改变生成的行集。但我质疑应该是JOIN（而不是LEFT JOIN）还是测试应该是WHERE，或者它是否真的像你写的那样有效。

请提供EXPLAIN SELECT ...“行”最大的行可能是要关注的表格。

GROUP BY  `products`.`id`

可能会导致输出错误。阅读“only_full_group_by”。

这会导致潜在的加速。

1. 做最少的工作来获得id的列表以满足ORDER BY和LIMIT。

2. 然后 JOIN 到所有其他表，包括返回到products，以获取其他列。这可能会加快速度并摆脱上述错误。

3. 在哪里实际将LEFT JOIN 更改为子查询：

    SELECT ...
        `vat_rate`.`rate` AS `vat_rate`,
        ...
    LEFT JOIN  `vat_rate`  ON `products`.`vat_rate_id` = `vat_rate`.`id`
    ...
   

-->

    SELECT ...
        ( SELECT `rate` FROM vat_rate WHERE id = `products`.`vat_rate_id`
        )  AS `vat_rate`,
        ...
    ...

完成大部分工作，然后回来寻求更多建议。

Answer 2

我曾经遇到过这个问题，当时我决定写一篇关于这个问题的文章，如果有人想看看我是如何实现的，我已经从不同的来源编译并优化了我的查询。请看看这个。 https://junaidtechblog.wordpress.com/2019/09/04/optimize-sql-query-groupby-having/