【问题标题】:MongoDB Aggregation in java: How to get total records count along with pagination result?java中的MongoDB聚合:如何获取总记录数和分页结果?
【发布时间】:2020-03-05 08:06:55
【问题描述】:

我有一个搜索条件,我必须提供搜索结果的分页以及集合中的记录总数。假设在 10 条记录的集合中,我只想获取 5 条记录以及总记录数。结果数据,我想将它们推送到具有 count 和 searchResult 属性的单独对象中。记录总数必须映射到计数,并将分页记录映射到 searchResult。我已经应用了聚合,它运行良好,除了包含 CountOperation 和 ProjectOperation。当我在聚合中添加 countOperation 和 ProjectOperation 时,它给出了无效的引用“_id!”例外。 预期的查询将是这样的。

   db.customer.aggregate([
{
  $facet:{
    searchResult:[{$match:{"name" : { "$regex" : "xyz", "$options" : "i" }}}],
    count: [{ $count: 'count' }]
  }
}

])

输出会是这样的。

   [
    {
      "searchResult":[{...},{...},{...}, ...],
      "count":[{"count":100}]
    }
   ]

搜索逻辑:

public List<SampleSearchResult> findListByRequest(ListRequest queryParams, Class<T> clazz) {

    String collectionName = mongoTemplate.getCollectionName(clazz);
    MatchOperation matchOperation = getMatchOperation(queryParams);
    SortOperation sortOperation = getSortOperation(queryParams);
    SkipOperation skipOperation = Aggregation.skip((long) queryParams.getPageNumber() * queryParams.getSize());
    LimitOperation limitOperation = Aggregation.limit(queryParams.getSize());
            CountOperation countOperation = Aggregation.count().as("count");
    ProjectionOperation projectionOperation = getProjectionOperation();

    AggregationResults<SampleSearchResult> results = mongoTemplate
            .aggregate(Aggregation.newAggregation(matchOperation, sortOperation, skipOperation, limitOperation, countOperation, projectionOperation ), collectionName, SampleSearchResult.class);

    return (List<SampleSearchResult>) results.getMappedResults();
}

投影操作逻辑

    private ProjectionOperation getProjectionOperation() {
    return Aggregation.project("count").and("_id").previousOperation();
}

排序操作逻辑:

private SortOperation getSortOperation(ListRequest listRequest) {
    // setting defaults
    if (StringUtils.isEmpty(listRequest.getSortBy())) {
        listRequest.setSortBy("_id");
        listRequest.setAsc(false);
    }
    Sort sort = listRequest.isAsc() ? new Sort(Direction.ASC, listRequest.getSortBy())
            : new Sort(Direction.DESC, listRequest.getSortBy());
    return Aggregation.sort(sort);
}

比赛操作逻辑:

private MatchOperation getMatchOperation(ListRequest listRequest) {
    Criteria criteria = new Criteria();
    // build match operation logic with listRequest parameters
    return Aggregation.match(criteria);
}

将保存聚合结果的结果对象

    public class SampleSearchResult {
private List<Object> searchResult;
private int count;
public List<Object> getSearchResult() {
    return searchResult;
}
public void setSearchResult(List<Object> searchResult) {
    this.searchResult = searchResult;
}
public int getCount() {
    return count;
}
public void setCount(int count) {
    this.count = count;
}

}

我需要正确编写 CountOperation 和 ProjectionOperation 以将数据映射到 SampleSearchResult 但我不是那么高效,因为我是 MongoDB 操作的新手。

【问题讨论】:

  • 问题:_id! 的感叹号是什么?这真的会出现在标准错误上吗?另外:您是说.aggregate(Aggregation.newAggregation(matchOperation, sortOperation, skipOperation, limitOperation ), collectionName, SampleSearchResult.class); 有效,但添加countOperation, projectionOperation 无效?
  • 是的,在countOperation和projectOperation的时候添加。感叹号出现在 stderr 上。

标签: mongodb spring-boot


【解决方案1】:

好吧,这可能会来得很晚,但是让我在这里放弃我的答案,看看将来谁可能会遇到同样的麻烦。正确的做法是像here 所述真正使用 facet。

我可以在 Java 中使用 facet,但是将查询结果映射到 pojo 类对我来说也是一个问题。但这就是我能够克服这一挑战的方法。

首先查看问题中的预期输出:

   [
    {
      "searchResult":[{...},{...},{...}, ...],
      "count":[{"count":100}]
    }
   ]

一个适合这个输出的 pojo 必须像这样建模:

@Getter //lombok stuff to create getters and setters
@Setter
public class CustomerSearchResult {

    private List<CustomerData> searchResult;
    private List<CountDto> count;
}

所以这里是 CountDto.java

@Setter
@Getter
public class CountDto {
    private Long count;
}

这里是 CustomerData.java

@Getter
@Setter
public class CustomerData {
    
    private Long dateCreated;
    private String Id;
    private String firstName;
    private String lastName;
}

接下来,MongoClient 必须使用自定义或默认代码注册表进行实例化,如下所示:

import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import org.bson.codecs.configuration.CodecRegistry;
import org.bson.codecs.pojo.PojoCodecProvider;

import static org.bson.codecs.configuration.CodecRegistries.fromProviders;
import static org.bson.codecs.configuration.CodecRegistries.fromRegistries;

public class MongoSource {

    private MongoClient mongoClient;

    public MongoClient createClient(){
        String connectionString = "your.database.connection.string";
        ConnectionString connection = new ConnectionString(connectionString);

        CodecRegistry defaultCodec = MongoClientSettings.getDefaultCodecRegistry();
        CodecRegistry fromProvider = fromProviders(PojoCodecProvider.builder().automatic(true).build());
        CodecRegistry pojoCodecRegistry = fromRegistries(defaultCodec, fromProvider);

        MongoClientSettings.Builder builder = MongoClientSettings.builder();
        builder.applyConnectionString(connection);
        MongoClientSettings settings = builder.codecRegistry(pojoCodecRegistry).build();
        mongoClient =  MongoClients.create(settings);
        return mongoClient;
    }


    public MongoClient getMongoClient(){
        return mongoClient;
    }
}

使用 Mongo Java Driver 4.0.X

不确定问题中使用的 mongo 驱动程序或第三方 API 的版本,但我认为标准的使用已被弃用,或者本机从来不是 mongo API 的一部分。但这就是我使用 Mongo Java Driver 4.0.x 实现这一目标的方式

要实现分页结果和总计数的搜索,使用 facet,需要构建两个单独的管道

  • 使用排序、限制和跳过进行搜索

  • 另外做总结果的实际计数没有限制 并跳过

这两个管道最终将用于构建要与聚合 API 一起使用的构面。

考虑下面用于定义相应查询字段的 pojo

import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
public class SearchQuery {
    String firstName;
    String lastName;
    int pageNumber;
    int pageSize;
}

这是执行搜索并返回包含计数和由 pageSizepageNumber 定义的分页结果的 CustomerSearchResult 实例的最终 sn-p。

import java.util.*;
import static com.mongodb.client.model.Aggregates.*;
import static com.mongodb.client.model.Filters.*;
import static com.mongodb.client.model.Projections.fields;
import static com.mongodb.client.model.Projections.include;
import org.apache.commons.lang3.StringUtils;

public CustomerSearchResult findListByRequest(SearchQuery queryParam){

        MongoSource mongoSource = new MongoSource();
        MongoDatabase database = mongoSource.createClient().getDatabase("customers_db");
        MongoCollection<CustomerSearchResult> collection = database.getCollection("customer", CustomerSearchResult.class);


        List<Bson> queryPipeline = new ArrayList<>();
        List<Bson> countPipeline = new ArrayList<>();
        List<Bson> andMatchFilter = new ArrayList<>();

        //you might want to check for null before using the fields in the query param object
        andMatchFilter.add(regex("firstName", Pattern.compile(queryParam.getFirstName(), Pattern.CASE_INSENSITIVE)));
        andMatchFilter.add(regex("lastName", Pattern.compile(queryParam.getLastName(), Pattern.CASE_INSENSITIVE)));

        if(queryParam.getPageNumber() == 0){
            queryParam.setPageNumber(1);
        }

        if(queryParam.getPageSize() == 0){
            queryParam.setPageSize(30);
        }

        queryPipeline.add(match(and(andMatchFilter)));
        queryPipeline.add(sort(eq("dateCreated", -1)));
        queryPipeline.add(skip(queryParam.getPageSize() * (queryParam.getPageNumber() - 1)));
        queryPipeline.add(limit(queryParam.getPageSize()));
        queryPipeline.add(project(fields(include("Id","firstName","lastName"))));

        countPipeline.add(match(and(andMatchFilter)));
        countPipeline.add(count());

        Facet resultFacet = new Facet("searchResult", queryPipeline);
        Facet totalDocFacet = new Facet("count", countPipeline);
        return collection.aggregate(Collections.singletonList(facet(resultFacet, totalDocFacet))).first();
    }

请注意,CustomerSearchResult 字段与分别为每个管道定义的构面的名称相同。这将允许定义的 coderegistry 将输出文档正确映射到您的 pojo 类 (CustomerSearchResult)

然后你可以这样做来访问计数:

    SearchQuery searchQuery = new SearchQuery();
    searchQuery.setPageNumber(1);
    searchQuery.setPageSize(15);
    searchQuery.setFirstName("John");
    searchQuery.setLastName("Doe");
    CustomerSearchResult result = findListByRequest(query);
    long count = result.getCount()!= null? result.getCount().get(0).getCount() : 0;
    List<CustomerData> data = result.getSearchResult();

【讨论】:

    猜你喜欢
    • 2013-12-19
    • 1970-01-01
    • 2020-10-13
    • 1970-01-01
    • 2021-07-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多