如何在 Elasticsearch 中对不同的记录进行分组

Question

我的 Elasticsearch 索引中有以下数据：

{
  "title": "Hello from elastic",
  "name": "ABC",
  "j_id": "1",
  "date": '2021-03-02T12:29:31.356514'
},
{
  "title": "Hello from elastic",
  "name": "PQR",
  "j_id": "1",
  "date": '2021-03-02T12:29:31.356514'
},
{
  "title": "Hello from elastic",
  "name": "XYZ",
  "j_id": "2",
  "date": '2021-03-02T12:29:31.356514'
},
{
  "title": "Hello from elastic",
  "name": "MNO",
  "j_id": "3",
  "date": '2021-03-02T12:29:31.356514'
}

现在我想根据id获取唯一记录。

预期的输出是：

{
    "1": [{
      "title": "Hello from elastic",
      "name": "ABC",
      "j_id": "1",
      "date": '2021-03-02T12:29:31.356514'
    },
    {
      "title": "Hello from elastic",
      "name": "PQR",
      "j_id": "1",
      "date": '2021-03-02T12:29:31.356514'
    }],
    "2": [{
      "title": "Hello from elastic",
      "name": "XYZ",
      "j_id": "2",
      "date": '2021-03-02T12:29:31.356514'
    }],
    "3": [{
      "title": "Hello from elastic",
      "name": "MNO",
      "j_id": "3",
      "date": '2021-03-02T12:29:31.356514'
    }]
  }

我尝试了一个聚合查询，但它只给了我计数。 另外，我想在回复中包含最新记录。

1. 如何从 Elasticsearch 中获取按 id 分组的排序的唯一记录？
2. 我想先插入最新的数据

Answer 1

假设一个覆盖date 和j_id 字段的最小映射：

PUT myindex
{
  "mappings": {
    "properties": {
      "j_id": {
        "type": "keyword"
      },
      "date": {
        "type": "date"
      }
    }
  }
}

您可以利用terms aggregation，其sub-aggregation 是有序的top_hits aggregation：

POST myindex/_search?filter_path=aggregations.*.buckets.key,aggregations.*.buckets.sorted_hits.hits.hits._source
{
  "size": 0,
  "aggs": {
    "by_j_id": {
      "terms": {
        "field": "j_id",
        "size": 10,
        "order": {
          "max_date": "desc"
        }
      },
      "aggs": {
        "max_date": {
          "max": {
            "field": "date"
          }
        },
        "sorted_hits": {
          "top_hits": {
            "size": 10,
            "sort": [
              {
                "date": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

URL 参数filter_path 减少了响应正文以紧密模仿您所需的格式：

{
  "aggregations" : {
    "by_j_id" : {
      "buckets" : [
        {
          "key" : "1",        
          "sorted_hits" : {
            "hits" : {
              "hits" : [
                {
                  "_source" : {
                    "title" : "Hello from elastic",
                    "name" : "ABC",
                    "j_id" : "1",
                    "date" : "2021-03-02T12:29:31.356514"
                  }
                },
                {
                  "_source" : {
                    "title" : "Hello from elastic",
                    "name" : "PQR",
                    "j_id" : "1",
                    "date" : "2021-03-02T12:29:31.356514"
                  }
                }
              ]
            }
          }
        },
        {
          "key" : "2",
          "sorted_hits" : {
            "hits" : {
              "hits" : [
                {
                  "_source" : {
                    "title" : "Hello from elastic",
                    "name" : "XYZ",
                    "j_id" : "2",
                    "date" : "2021-03-02T12:29:31.356514"
                  }
                }
              ]
            }
          }
        },
        {
          "key" : "3",
          "sorted_hits" : {
            "hits" : {
              "hits" : [
                {
                  "_source" : {
                    "title" : "Hello from elastic",
                    "name" : "MNO",
                    "j_id" : "3",
                    "date" : "2021-03-02T12:29:31.356514"
                  }
                }
              ]
            }
          }
        }
      ]
    }
  }
}