子查询的使用非常慢答案

【问题标题】：Usage of Subquery very slow子查询的使用非常慢
【发布时间】：2019-12-08 12:38:08
【问题描述】：

我有以下查询，但需要大约 1:20 分钟才能完成：

user_total_revenue = (
    User.objects.annotate(
        total_revenue=Sum("payments__amount"),
        first_order_date=Subquery(
            User.objects.filter(pk=OuterRef("pk"))
            .annotate(first_order_date=Min("shops__orders__created"))
            .values("first_order_date"),
            output_field=DateTimeField(),
        ),
    )
)

原始 SQL 查询：

SELECT
   "users_user"."id",
   "users_user"."created",
   "users_user"."updated",
   "users_user"."user_id",
   "users_user"."name",
   "users_user"."address",
   SUM("payments_payment"."amount") AS "total_revenue",
   (
      SELECT
         MIN(U2."created") AS "first_order_date" 
      FROM
         "users_user" U0 
         LEFT OUTER JOIN
            "shops_shop" U1 
            ON (U0."id" = U1."user_id") 
         LEFT OUTER JOIN
            "orders_order" U2 
            ON (U1."id" = U2."shop_id") 
      WHERE
         U0."id" = 
         (
            "users_user"."id"
         )
      GROUP BY
         U0."id"
   )
   AS "first_order_date" 
FROM
   "users_user" 
   LEFT OUTER JOIN
      "payments_payment" 
      ON ("users_user"."id" = "payments_payment"."user_id") 
GROUP BY
   "users_user"."id",
   (
      SELECT
         MIN(U2."created") AS "first_order_date" 
      FROM
         "users_user" U0 
         LEFT OUTER JOIN
            "shops_shop" U1 
            ON (U0."id" = U1."user_id") 
         LEFT OUTER JOIN
            "orders_order" U2 
            ON (U1."id" = U2."shop_id") 
      WHERE
         U0."id" = 
         (
            "users_user"."id"
         )
      GROUP BY
         U0."id"
   )
   LIMIT 21

因为它太慢了，所以我用一种更快的方式（

def user_total_revenue():
    first_orders = (
        User.objects.annotate(first_order_date=Min("shops__orders__created"))
        .values("pk", "first_order_date")
    )
    first_orders = {
        item["pk"]: item["first_order_date"] for item in first_orders
    }

    user_total_revenue = User.objects.annotate(total_revenue=Sum("payments__amount"))

    return [
        {
            "user": user,
            "first_order_date": first_orders[user.pk],
        }
        for user in user_total_revenue
    ]

我想了解为什么我的第一个版本使用子查询如此缓慢。

【问题讨论】：

您可以编辑您的问题并显示生成的 SQL 吗？
使用 Django-debug-toolbar 或 chrome 扩展来弄清楚发生了什么。我的猜测是 User.objects.filter(pk=... 被重复多次，导致查询速度变慢。

标签： python sql django

【解决方案1】：

这里不是 100%，但我相信由此产生的查询应该更快。

from django.db.models.functions import Coalesce
from django.db.models import Value

user_total_revenue = User.objects.annotate(
    total_revenue=Coalesce(
        Subquery(
            Payment.object.filter(user_id=OuterRef("pk"))
            .values("user_id")
            .annotate(sum=Sum("amount"))
            .values("sum")[:1]
        ),
        Value(0),
    ),
    first_order_date=Subquery(
        User.objects.filter(pk=OuterRef("pk"))
        .annotate(first_order_date=Min("shops__orders__created"))
        .values("first_order_date"),
        output_field=DateTimeField(),
    ),
)

其他尝试：

使用order_by("shops__orders__created") 并选择第一项而不是Min
为订单添加created的索引

【讨论】：