【问题标题】:Django, DRF: How do I use pagination in raw SQL queries?Django,DRF:如何在原始 SQL 查询中使用分页?
【发布时间】:2022-01-06 17:10:58
【问题描述】:

我想在 Django REST 框架的 ListAPIView 中使用以下原始查询。 我不确定如何使用原始 SQL 查询而不是查询集来支持分页。 我已经研究过了,并没有发现很多对我有意义的东西。

如果我想使用查询集,它看起来像这样 如果我想使用原始 SQL 查询,该怎么做?

class VideoListView(generics.ListAPIView):
    queryset = Video.objects.raw(...)
    serializer_class = VideoSerializer
SELECT DISTINCT "t"."id",
                "t"."title",
                "t"."thumbnail_url",
                "t"."preview_url",
                "t"."embed_url",
                "t"."duration",
                "t"."views",
                "t"."is_public",
                "t"."published_at",
                "t"."created_at",
                "t"."updated_at",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_history" U0
   WHERE (U0."user_id" IS NULL
          AND U0."video_id" = "t"."id")
   LIMIT 1) AS "is_viewed",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_favorite" U0
   WHERE (U0."user_id" IS NULL
          AND U0."video_id" = "t"."id")
   LIMIT 1) AS "is_favorited",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_track" U0
   INNER JOIN "videos_playlist" U1 ON (U0."playlist_id" = U1."id")
   WHERE (U1."is_wl"
          AND U1."user_id" IS NULL
          AND U0."video_id" = "t"."id")
   LIMIT 1) AS "is_wl"
FROM (
  (SELECT "videos_video".*
   FROM "videos_video"
   WHERE ("videos_video"."is_public"
      AND "videos_video"."published_at" <= '2022-01-03 05:20:16.725884+00:00'
      AND "videos_video"."title" LIKE '%word')
   ORDER BY "videos_video"."published_at" DESC
   LIMIT 20)
UNION
  (SELECT "videos_video".*
   FROM "videos_video"
   LEFT OUTER JOIN "videos_video_tags" ON ("videos_video"."id" = "videos_video_tags"."video_id")
   LEFT OUTER JOIN "videos_tag" ON ("videos_video_tags"."tag_id" = "videos_tag"."id")
   WHERE ("videos_video"."is_public"
      AND "videos_video"."published_at" <= '2022-01-03 05:20:16.725884+00:00'
      AND "videos_tag"."name" LIKE '%word')
   ORDER BY "videos_video"."published_at" DESC
   LIMIT 20)
) AS t
ORDER BY "t"."published_at" DESC
LIMIT 20;

我需要创建一个自定义分页器还是什么?我更加困惑,因为我正在使用 LIMIT 和其他方法进行子查询。

--- 添加models.py等... ---

# models.py
class Tag(models.Model):
    name = models.CharField(unique=True, max_length=30)
    created_at = models.DateTimeField(default=timezone.now)
    ...


class Video(models.Model):
    title = models.CharField(max_length=300)
    tags = models.ManyToManyField(Tag, blank=True)
    updated_at = models.DateTimeField(auto_now=True)
    ...


class History(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
    video = models.ForeignKey(Video, on_delete=models.CASCADE)
    ...


class Favorite(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE
    video = models.ForeignKey(Video, on_delete=models.CASCADE)
    ...


class Playlist(models.Model):
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
    is_wl = models.BooleanField(default=False, editable=False)
    ...


class Track(models.Model):
    playlist = models.ForeignKey(Playlist, on_delete=models.CASCADE, null=True)
    video = models.ForeignKey(Video, on_delete=models.CASCADE)
    ...

最初使用的慢查询集和 SQL 查询。

Video.objects.annotate(
    is_viewed=Exists(History.objects.filter(user=user, video=OuterRef("pk"))),
    is_favorited=Exists(
        Favorite.objects.filter(user=user, video=OuterRef("pk"))
    ),
    is_wl=Exists(
        Track.objects.filter(
            playlist__user=user, playlist__is_wl=True, video=OuterRef("pk")
        )
    ),
).filter(
    Q(title__contains=value)
    | Q(tags__name__contains=value)),
    is_public=True,
    published_at__lte=timezone.now(),
).order_by("-published_at").distinct()
SELECT DISTINCT "videos_video"."id",
                "videos_video"."published_at",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_history" U0
   WHERE (U0."user_id" IS NULL
          AND U0."video_id" = "videos_video"."id")
   LIMIT 1) AS "is_viewed",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_favorite" U0
   WHERE (U0."user_id" IS NULL
          AND U0."video_id" = "videos_video"."id")
   LIMIT 1) AS "is_favorited",
                EXISTS
  (SELECT (1) AS "a"
   FROM "videos_track" U0
   INNER JOIN "videos_playlist" U1 ON (U0."playlist_id" = U1."id")
   WHERE (U1."is_wl"
          AND U1."user_id" IS NULL
          AND U0."video_id" = "videos_video"."id")
   LIMIT 1) AS "is_wl"
FROM "videos_video"
LEFT OUTER JOIN "videos_video_tags" ON ("videos_video"."id" = "videos_video_tags"."video_id")
WHERE ("videos_video"."is_public"
       AND "videos_video"."published_at" <= '2021-12-27 13:34:29.103369+00:00'
       AND ("videos_video"."title" &@~ 'word'
            OR "videos_video_tags"."tag_id" IN
              (SELECT U0."id"
               FROM "videos_tag" U0
               WHERE U0."name" &@~ 'word')))
ORDER BY "videos_video"."published_at" DESC
LIMIT 20;

【问题讨论】:

  • 为什么首先使用原始查询? Django 的分页将在QuerySet 上使用OFFSETLIMIT,并在列表上工作,但是当使用列表时,它将首先获取 all 记录,因此失去了潜在的效率通过在数据库级别进行分页来获得收益。
  • ORM生成的SQL很慢,所以我自己重写了SQL。但是,我不知道如何在 Queryset 中重现这个 SQL 查询。
  • 能否提供VideoHistory、Favorite`等型号?
  • 我省略了一些东西,但我添加了models.py和我最初使用的慢查询集和SQL查询。
  • 如果您将is_viewed=Exists(History.objects.filter(user=user, video=OuterRef("pk"))[:1]) 用于Exists 查询,然后添加[:20],这样会增加原始查询中的限制。

标签: sql django django-models django-rest-framework django-views


【解决方案1】:

如果您从rest_framework.pagination 询问通用PageNumberPagination,通常是这样完成的。您使用默认值创建自己的分页类:

class VideoPagination(PageNumberPagination):
    max_page_size = 100
    page_size_query_param = 'page_size'
    page_size = 25

然后你只需将它添加到你的视图中:

class VideoListView(generics.ListAPIView):
    queryset = Video.objects.all()
    serializer_class = VideoSerializer
    pagination_class = VideoPagination

就是这样,你可以使用参数pagepage_size

【讨论】:

  • 我为糟糕的解释道歉。我需要使用objects.raw()问题中提到的原始SQL查询来查询集,我不知道如何将objects.raw()与DRF分页结合起来。
  • @Jvn 那么你需要this
  • 谢谢。这个好像很老了,Django目前没有标准这样的功能吗?
  • 据我所知,没有。然而,这对我来说并不奇怪。这项任务非常罕见,因为大多数数据库请求都可以相当可行地使用 ORM 进行。我在这个主题上发现了一件事——this repository。不建议使用它。但对我来说,这证明了这个案子不受欢迎。
猜你喜欢
  • 2011-02-01
  • 2015-11-21
  • 2012-03-16
  • 1970-01-01
  • 2017-04-08
  • 2013-04-17
  • 2017-04-26
  • 2021-01-06
  • 2018-07-15
相关资源
最近更新 更多