什么是N+1查询问题
N+1 查询问题发生在数据访问框架执行 N 个额外的 SQL 语句以获取执行主 SQL 查询时可能检索到的相同数据时。
N值越大,执行的查询越多,对性能的影响越大。而且,与可以帮助您找到运行缓慢的查询的慢查询日志不同,N+1 问题不会被发现,因为每个单独的附加查询运行速度足够快,不会触发慢查询日志。
问题在于执行大量额外查询,总体而言,这些查询需要足够的时间来减慢响应时间。
假设我们有以下 post 和 post_cmets 数据库表,它们形成了一对多的表关系:
我们将创建以下 4 个post 行:
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 1', 1)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 2', 2)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 3', 3)
INSERT INTO post (title, id)
VALUES ('High-Performance Java Persistence - Part 4', 4)
而且,我们还将创建 4 个post_comment 子记录:
INSERT INTO post_comment (post_id, review, id)
VALUES (1, 'Excellent book to understand Java Persistence', 1)
INSERT INTO post_comment (post_id, review, id)
VALUES (2, 'Must-read for Java developers', 2)
INSERT INTO post_comment (post_id, review, id)
VALUES (3, 'Five Stars', 3)
INSERT INTO post_comment (post_id, review, id)
VALUES (4, 'A great reference book', 4)
普通 SQL 的 N+1 查询问题
如果您使用此 SQL 查询选择 post_comments:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
""", Tuple.class)
.getResultList();
然后,您决定为每个post_comment 获取关联的post title:
for (Tuple comment : comments) {
String review = (String) comment.get("review");
Long postId = ((Number) comment.get("postId")).longValue();
String postTitle = (String) entityManager.createNativeQuery("""
SELECT
p.title
FROM post p
WHERE p.id = :postId
""")
.setParameter("postId", postId)
.getSingleResult();
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
您将触发 N+1 查询问题,因为您执行的不是一个 SQL 查询,而是 5 (1 + 4):
SELECT
pc.id AS id,
pc.review AS review,
pc.post_id AS postId
FROM post_comment pc
SELECT p.title FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.title FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.title FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.title FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
解决 N+1 查询问题非常容易。您需要做的就是在原始 SQL 查询中提取您需要的所有数据,如下所示:
List<Tuple> comments = entityManager.createNativeQuery("""
SELECT
pc.id AS id,
pc.review AS review,
p.title AS postTitle
FROM post_comment pc
JOIN post p ON pc.post_id = p.id
""", Tuple.class)
.getResultList();
for (Tuple comment : comments) {
String review = (String) comment.get("review");
String postTitle = (String) comment.get("postTitle");
LOGGER.info(
"The Post '{}' got this review '{}'",
postTitle,
review
);
}
这一次,只执行一个 SQL 查询来获取我们进一步感兴趣使用的所有数据。
JPA 和 Hibernate 的 N+1 查询问题
在使用 JPA 和 Hibernate 时,有几种方法可以触发 N+1 查询问题,因此了解如何避免这些情况非常重要。
对于下一个示例,假设我们将 post 和 post_comments 表映射到以下实体:
JPA 映射如下所示:
@Entity(name = "Post")
@Table(name = "post")
public class Post {
@Id
private Long id;
private String title;
//Getters and setters omitted for brevity
}
@Entity(name = "PostComment")
@Table(name = "post_comment")
public class PostComment {
@Id
private Long id;
@ManyToOne
private Post post;
private String review;
//Getters and setters omitted for brevity
}
FetchType.EAGER
为您的 JPA 关联隐式或显式使用 FetchType.EAGER 是一个坏主意,因为您将获取更多所需的数据。而且FetchType.EAGER策略也容易出现N+1查询问题。
不幸的是,@ManyToOne 和 @OneToOne 关联默认使用 FetchType.EAGER,所以如果您的映射如下所示:
@ManyToOne
private Post post;
您正在使用FetchType.EAGER 策略,并且每次在使用JPQL 或Criteria API 查询加载一些PostComment 实体时忘记使用JOIN FETCH:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
您将触发 N+1 查询问题:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
请注意执行的附加 SELECT 语句,因为在返回 PostComment 实体中的 List 之前必须获取 post 关联。
与调用EnrityManager 的find 方法时使用的默认获取计划不同,JPQL 或标准 API 查询定义了一个显式计划,Hibernate 无法通过自动注入 JOIN FETCH 来更改该计划。所以,你需要手动完成。
如果您根本不需要post 关联,那么您在使用FetchType.EAGER 时就不走运了,因为无法避免获取它。这就是为什么最好默认使用FetchType.LAZY。
但是,如果你想使用post关联,那么你可以使用JOIN FETCH来避免N+1查询问题:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
这一次,Hibernate 将执行一条 SQL 语句:
SELECT
pc.id as id1_1_0_,
pc.post_id as post_id3_1_0_,
pc.review as review2_1_0_,
p.id as id1_0_1_,
p.title as title2_0_1_
FROM
post_comment pc
INNER JOIN
post p ON pc.post_id = p.id
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
FetchType.LAZY
即使您切换到对所有关联显式使用FetchType.LAZY,您仍然会遇到 N+1 问题。
这一次,post 关联映射如下:
@ManyToOne(fetch = FetchType.LAZY)
private Post post;
现在,当您获取 PostComment 实体时:
List<PostComment> comments = entityManager
.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
Hibernate 将执行一条 SQL 语句:
SELECT
pc.id AS id1_1_,
pc.post_id AS post_id3_1_,
pc.review AS review2_1_
FROM
post_comment pc
但是,如果之后,您将引用延迟加载的post 关联:
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
你会得到 N+1 查询问题:
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 1
-- The Post 'High-Performance Java Persistence - Part 1' got this review
-- 'Excellent book to understand Java Persistence'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 2
-- The Post 'High-Performance Java Persistence - Part 2' got this review
-- 'Must-read for Java developers'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 3
-- The Post 'High-Performance Java Persistence - Part 3' got this review
-- 'Five Stars'
SELECT p.id AS id1_0_0_, p.title AS title2_0_0_ FROM post p WHERE p.id = 4
-- The Post 'High-Performance Java Persistence - Part 4' got this review
-- 'A great reference book'
由于post关联是惰性获取的,所以在访问惰性关联时会执行辅助SQL语句以构建日志消息。
同样,修复包括在 JPQL 查询中添加 JOIN FETCH 子句:
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
join fetch pc.post p
""", PostComment.class)
.getResultList();
for(PostComment comment : comments) {
LOGGER.info(
"The Post '{}' got this review '{}'",
comment.getPost().getTitle(),
comment.getReview()
);
}
而且,就像在 FetchType.EAGER 示例中一样,此 JPQL 查询将生成单个 SQL 语句。
即使您使用FetchType.LAZY 并且不引用双向@OneToOne JPA 关系的子关联,您仍然可以触发N+1 查询问题。
如何自动检测N+1查询问题
如果您想在数据访问层自动检测 N+1 查询问题,可以使用db-util 开源项目。
首先,您需要添加以下 Maven 依赖项:
<dependency>
<groupId>com.vladmihalcea</groupId>
<artifactId>db-util</artifactId>
<version>${db-util.version}</version>
</dependency>
之后,您只需使用SQLStatementCountValidator 实用程序来断言生成的底层 SQL 语句:
SQLStatementCountValidator.reset();
List<PostComment> comments = entityManager.createQuery("""
select pc
from PostComment pc
""", PostComment.class)
.getResultList();
SQLStatementCountValidator.assertSelectCount(1);
如果您使用FetchType.EAGER 并运行上述测试用例,您将得到以下测试用例失败:
SELECT
pc.id as id1_1_,
pc.post_id as post_id3_1_,
pc.review as review2_1_
FROM
post_comment pc
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 1
SELECT p.id as id1_0_0_, p.title as title2_0_0_ FROM post p WHERE p.id = 2
-- SQLStatementCountMismatchException: Expected 1 statement(s) but recorded 3 instead!