如何在 query_string 搜索中聚合匹配的术语？答案

【问题标题】：How to aggregate matched terms in a query_string search?如何在 query_string 搜索中聚合匹配的术语？
【发布时间】：2023-01-13 03:57:38
【问题描述】：

我希望在 dict 的嵌套列表中搜索通配符术语，然后获取术语列表及其按匹配通配符分组的 uuid。

我的索引中有以下映射：

"mappings": {
    "properties": {
        "uuid": {
            "type": "keyword"
        },
        "urls": {
            "type": "nested",
            "properties": {
                "url": {
                    "type": "keyword"
                },
                "is_visited": {
                    "type": "boolean"
                }
            }
        }           
    }
}

还有很多这样的数据：

{
    "uuid":"afa9ac03-0723-4d66-ae18-08a51e2973bd"
    "urls": [
        {
            "is_visited": true,
            "url": "https://www.google.com"
        },
        {
            "is_visited": false,
            "url": "https://www.facebook.com"
        },
        {
            "is_visited": true,
            "url": "https://www.twitter.com"
        },              
    ]
},
{
    "uuid":"4a1c695d-756b-4d9d-b3a0-cf524d955884"
    "urls": [
        {
            "is_visited": true,
            "url": "https://www.stackoverflow.com"
        },
        {
            "is_visited": false,
            "url": "https://www.facebook.com"
        },
        {
            "is_visited": false,
            "url": "https://drive.google.com"
        },
        {
            "is_visited": false,
            "url": "https://maps.google.com"
        },                      
    ]
}
...

我希望通过通配符 "*google.com OR *twitter.com" 搜索并获得如下内容：

"hits": [
    "*google.com": [
        {
            "uuid": "4a1c695d-756b-4d9d-b3a0-cf524d955884",
            "_source": {
                "is_visited": false,
                "url": "https://drive.google.com"
            }
        },
        {
            "id": "4a1c695d-756b-4d9d-b3a0-cf524d955884",
            "_source": {
                "is_visited": false,
                "url": "https://maps.google.com"
            }
        },
        {
            "uuid":"afa9ac03-0723-4d66-ae18-08a51e2973bd",
            "_source": {
                "is_visited": true,
                "url": "https://www.google.com"
            }
        }
    ]
    "*twitter.com": [
        {
            "uuid":"afa9ac03-0723-4d66-ae18-08a51e2973bd",
            "_source": {
                "is_visited": true,
                "url": "https://www.twitter.com"
            },  
        },
    ]
]

这是我的（python）搜索查询：

body = {
  #"_source": False,
  "size": 100,
  "query": {
        "nested": {
            "path": "urls",
            "query":{
                "query_string":{
                    "query": f"urls.url:{urlToSearch}",
                }
            }
            ,"inner_hits": {
                "size":100 # returns top 100 results
            }
        }
    }
}

但它会为每个匹配的术语返回一个匹配项，而不是将它们聚合在一个类似于我想要得到的列表中。

【问题讨论】：

标签： elasticsearch elasticsearch-aggregation elasticsearch-dsl

【解决方案1】：

Elasticsearch 不会按照您设置查询的方式提供您想要的输出。这个场景是一个聚合。我的建议是应用嵌套查询并对结果使用聚合。

关注点wildcard query：

避免以 * 或 ? 开头的模式。这可以增加迭代需要找到匹配的术语和缓慢的搜索性能。

{
  "size": 0,
  "query": {
    "nested": {
      "path": "urls",
      "query": {
        "bool": {
          "should": [
            {
              "wildcard": {
                "urls.url": {
                  "value": "*google.com"
                }
              }
            },
            {
              "wildcard": {
                "urls.url": {
                  "value": "*twitter.com"
                }
              }
            }
          ]
        }
      }
    }
  },
  "aggs": {
    "agg_providers": {
      "nested": {
        "path": "urls"
      },
      "aggs": {
        "google.com": {
          "terms": {
            "field": "urls.url",
            "include": ".*google.com",
            "size": 10
          }
        },
        "twitter.com": {
          "terms": {
            "field": "urls.url",
            "include": ".*twitter.com",
            "size": 10
          }
        }
      }
    }
  }
}

结果：

"aggregations": {
    "agg_providers": {
      "doc_count": 7,
      "twitter.com": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "https://www.twitter.com",
            "doc_count": 1
          }
        ]
      },
      "google.com": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "https://drive.google.com",
            "doc_count": 1
          },
          {
            "key": "https://maps.google.com",
            "doc_count": 1
          },
          {
            "key": "https://www.google.com",
            "doc_count": 1
          }
        ]
      }
    }
  }

【讨论】：