ElasticSearch - 过滤桶答案

【问题标题】：ElasticSearch - Filter BucketsElasticSearch - 过滤桶
【发布时间】：2020-12-25 15:16:59
【问题描述】：

我的 elasticSearch 查询是这样的：

{
    "size": 0,
    "aggs": {
        "group_by_id": {
            "terms": {
                "field": "Infos.InstanceInfo.ID.keyword",
                "size": 1000
            },
            "aggs": {
                "tops": {
                    "top_hits": {
                        "size": 100,
                        "sort": {
                            "Infos.InstanceInfo.StartTime": "asc"
                        }
                    }
                }
            }
        }
    }
}

效果很好，我有一个这种形式的结果：

aggregations
=========>group_by_id
==============>buckets
                {key:id1}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"KO"}
                
                {key:id2}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"OK"}
                
                {key:id3}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"OK"}

我正在尝试添加一个过滤器，所以当“OK”时，结果必须是这样的：

aggregations
=========>group_by_id
==============>buckets
                {key:id2}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"OK"}
                
                {key:id3}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"OK"}

对于“KO”：

aggregations
=========>group_by_id
==============>buckets
                {key:id1}
                ===============>docs
                {doc1.Status:"KO"}
                {doc2.Status:"KO"}

字段“Startime”和“Status”处于同一级别“Infos.InstanceInfo.[...]” 有什么想法吗？

编辑

示例文档：

{
  "took" : 794,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group_by_id" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 143846,
      "buckets" : [
        {
          "key" : "1000",
          "doc_count" : 6,
          "tops" : {
            "hits" : {
              "total" : {
                "value" : 6,
                "relation" : "eq"
              },
              "max_score" : null,
              "hits" : [
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "vHFvoXYBVWrYChNi7hB7",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.011+01:00",
                        "status" : "KO"
                      }
                    }
                  },
                  "sort" : [
                    1609026236011
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "xHFvoXYBVWrYChNi7xAB",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.145+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609026236145
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "xXFvoXYBVWrYChNi7xAC",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.147+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609026236147
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "x3FvoXYBVWrYChNi7xAs",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.188+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609026236188
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "yHFvoXYBVWrYChNi7xAs",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.19+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609026236190
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "ynFvoXYBVWrYChNi7xBd",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "1000",
                        "StartTime" : "2020-12-27T00:43:56.236+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609026236236
                  ]
                }
              ]
            }
          }
        },
        {
          "key" : "2000",
          "doc_count" : 2,
          "tops" : {
            "hits" : {
              "total" : {
                "value" : 2,
                "relation" : "eq"
              },
              "max_score" : null,
              "hits" : [
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "7HL_onYBVWrYChNij4Is",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "2000",
                        "StartTime" : "2020-12-27T08:00:26.011+01:00",
                        "status" : "KO"
                      }
                    }
                  },
                  "sort" : [
                    1609052426011
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "9HL_onYBVWrYChNij4Kz",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "2000",
                        "StartTime" : "2020-12-27T08:00:26.146+01:00",
                        "status" : "KO"
                      }
                    }
                  },
                  "sort" : [
                    1609052426146
                  ]
                }
              ]
            }
          }
        },
        {
          "key" : "3000",
          "doc_count" : 6,
          "tops" : {
            "hits" : {
              "total" : {
                "value" : 6,
                "relation" : "eq"
              },
              "max_score" : null,
              "hits" : [
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "7nNRpHYBVWrYChNiiruh",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.015+01:00",
                        "status" : "KO"
                      }
                    }
                  },
                  "sort" : [
                    1609074576015
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "9nNRpHYBVWrYChNii7s5",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.166+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609074576166
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "93NRpHYBVWrYChNii7s5",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.166+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609074576166
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "-XNRpHYBVWrYChNii7ti",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.209+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609074576209
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "-nNRpHYBVWrYChNii7ts",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.219+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609074576219
                  ]
                },
                {
                  "_index" : "azerty",
                  "_type" : "_doc",
                  "_id" : "_HNRpHYBVWrYChNii7ud",
                  "_score" : null,
                  "_source" : {
                    "Infos" : {
                      "InstanceInfo" : {
                        "ID" : "3000",
                        "StartTime" : "2020-12-27T14:09:36.269+01:00",
                        "status" : "OK"
                      }
                    }
                  },
                  "sort" : [
                    1609074576269
                  ]
                }
              ]
            }
          }
        }
      ]
    }
  }
}

【问题讨论】：

标签： elasticsearch elasticsearch-aggregation

【解决方案1】：

假设状态字段在Infos.InstanceInfo 下并且属于keyword 映射，您可以使用filter 聚合：

{
  "size": 0,
  "aggs": {
    "status_KO_only": {
      "filter": {                                  <--
        "term": {
          "Infos.InstanceInfo.Status": "KO"
        }
      },
      "aggs": {
        "group_by_id": {
          "terms": {
            "field": "Infos.InstanceInfo.ID.keyword",
            "size": 1000
          },
          "aggs": {
            "tops": {
              "top_hits": {
                "size": 100,
                "sort": {
                  "Infos.InstanceInfo.StartTime": "asc"
                }
              }
            }
          }
        }
      }
    }
  }
}

在这种特殊情况下，您可以在搜索请求的 query 部分应用相同的 term 查询，而不必使用 filter 聚合。

如果您想在同一个请求中同时获得 OK 和 KO，您可以复制/粘贴整个 status_KO_only 聚合，重命名第二个聚合，然后瞧——您现在在一个请求中拥有两个组.当然，您可以拥有任意多个不同命名的（顶级）过滤器聚合。

现在，当您确实需要同时使用多个过滤器聚合时，有一种更优雅的方式不需要复制粘贴——输入filters 聚合：

{
  "size": 0,
  "aggs": {
    "by_statuses": {
      "filters": {                                     <--
        "filters": {
          "status_KO": {
            "term": {
              "Infos.InstanceInfo.Status": "KO"
            }
          },
          "status_OK": {
            "term": {
              "Infos.InstanceInfo.Status": "OK"
            }
          }
        }
      },
      "aggs": {
        "group_by_id": {
          "terms": {
            "field": "Infos.InstanceInfo.ID.keyword",
            "size": 1000
          },
          "aggs": {
            "tops": {
              "top_hits": {
                "size": 100,
                "sort": {
                  "Infos.InstanceInfo.StartTime": "asc"
                }
              }
            }
          }
        }
      }
    }
  }
}

任何子子聚合都将自动成为显式声明的术语过滤器的存储桶。

我个人认为复制/粘贴方法更具可读性，尤其是。动态构建此类请求时（基于 UI 下拉列表等）。

【讨论】：

我之前试过你的第一个建议，但它不符合预期的结果，因为它也会从id2返回doc1，从id3返回doc1。对于我们的逻辑业务，它应该只返回 id1 的文档，因为那里都是 KO。并且用于搜索 OK 的第二个查询必须返回所有文档（甚至是具有 KO 的文档），其 id 至少有一个文档具有 OK（id2 和 id3）..
对。您能用一些示例文档更新您的问题吗？
完成。有 3 个 ID：1000、2000 和 3000。
嗨，乔，你认为有可能达到这个结果吗？
只需将 filter 聚合移动到您需要的位置即可。