ElasticSearch 6.0.1 - SQL DISTINCT 子句 - NEST C#

Question

我需要使用 NEST 从文档中返回所有类别 DISTINCTS（不重复任何内容）。

在 SQL 中是这样的：

SELECT DISTINCT Category  
  FROM Log  
  ORDER BY Category ASC 

在 ElasticSearch 中我是这样做的：

GET log/_search
{
  "size":"0",
  "aggs" : {
  "alias_category" : {
  "terms" : { "field" : "category.keyword" }
  }
  }
}

如何使用 NEST 做到这一点？

public ICollection<string> SelectAllCategoriesDistinct(ElasticClient client)
    {
        var searchResponse = client.Search<LogElasticSearch>(s => s
                                        .Query(q => q
                                            .Terms(t => t
                                                .Field(f => f.Category)
                                            )
                                        )
                                     );
        return (ICollection<string>)searchResponse;
    }

Answer 1

我找到了一种方法。不是一种优雅的方式，但我在 elastico 网站 (https://discuss.elastic.co/t/c-nest-best-way-of-accessing-properties-of-iaggregate-object/85384/2) 中找到并且我是这样做的：

public ICollection<string> SelectAllCategoriesDistinct(ElasticClient client)
    {
        var searchResponse = 
            client.Search<LogElasticSearch>(s => s
                .Size(0)
                .Aggregations(agg => agg
                    .Terms("categories", t => t
                        .Field("category.keyword")
                    )                  
                )
            );
       
        var aggregation = searchResponse.Aggregations.Values;
        var listOfCategories = new List<string>();

        if (searchResponse.Aggregations.Values.FirstOrDefault().GetType() == typeof(BucketAggregate))
        {
            foreach (IBucket bucket in ((BucketAggregate)aggregation.FirstOrDefault()).Items)
            {
                if (bucket.GetType() == typeof(KeyedBucket<object>))
                {
                    var valueKey = ((KeyedBucket<object>)bucket).Key;
                    listOfCategories.Add(valueKey.ToString());
                }
            } 
        }

        return listOfCategories.OrderBy(c => c).ToList();
    }

如果有人知道更好的方法，请帮助我改进，但这样就达到了目标。

Answer 2

我会使用composite aggregation 来获取所有条款。与terms aggregation 相比，复合聚合支持分页，因此如果要获取很多 项，则可以发出多个请求

private static void Main()
{
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    var settings = new ConnectionSettings(pool); 
    var client = new ElasticClient(settings);
    
    var categories = new List<string>();

    GetAllCategories(client, categories);
    
    // do something with the categories
    foreach(var category in categories)
        Console.WriteLine(category);
}

private static void GetAllCategories(IElasticClient client, List<string> categories, CompositeKey after = null)
{
    // number of terms to fetch in each request
    var size = 10_000;
    
    var response = client.Search<LogElasticSearch>(s => s
        .Size(0)
        .Aggregations(a => a
            .Composite("categories", c => c
                .After(after)
                .Size(size)
                .Sources(so => so
                    .Terms("category", t => t
                        .Field("category.keyword")
                        .Order(SortOrder.Ascending)
                    )
                )
            )
        )
    );

    var compositeAgg = response.Aggregations.Composite("categories");
    var buckets = compositeAgg.Buckets;

    foreach (var bucket in buckets)
    {
        if (bucket.Key.TryGetValue("category", out string category))
        {
            categories.Add(category);
        }
    }

    // there may be more
    if (buckets.Count == size)
    {
        GetAllCategories(client, categories, compositeAgg.AfterKey);
    }
}