使用 ElasticSearch Python 客户端的聚合查询失败

Question

Here is an aggregation query that works as expected when I use dev tools in on Elastic Search :  

   search_query = {
      "aggs": {
        "SHAID": {
          "terms": {
            "field": "identiferid",
            "order": {
              "sort": "desc"
            },
    #         "size": 100000
          },
          "aggs": {
            "update": {
              "date_histogram": {
                "field": "endTime",
                "calendar_interval": "1d"
              },
              "aggs": {
                "update1": {
                      "sum": {
                        "script": {
                          "lang": "painless",
                          "source":"""
                              if (doc['distanceIndex.att'].size()!=0) { 
                                  return doc['distanceIndex.att'].value;
                              } 
                              else { 
                                  if (doc['distanceIndex.att2'].size()!=0) { 
                                  return doc['distanceIndex.att2'].value;
                              }
                              return null;
                              }
                              """
                        }
                      }
                    },
                "update2": {
                         "sum": {
                        "script": {
                          "lang": "painless",
                          "source":"""
                              if (doc['distanceIndex.att3'].size()!=0) { 
                                  return doc['distanceIndex.att3'].value;
                              } 
                              else { 
                                  if (doc['distanceIndex.at4'].size()!=0) { 
                                  return doc['distanceIndex.att4'].value;
                              }
                              return null;
                              }
                              """
                        }
                      }
                  },
              }
            },
            "sort": {
              "sum": {
                "field": "time2"
              }
            }
          }
        }
      },
    "size": 0,
      "query": {
        "bool": {
          "filter": [
            {
              "match_all": {}
            },
            {
              "range": {
                "endTime": {
                  "gte": "2021-11-01T00:00:00Z",
                  "lt": "2021-11-03T00:00:00Z"
                }
              }
            }
          ]
        }
      }
    }

当我尝试使用 Python ElasticSearch 客户端 (https://elasticsearch-py.readthedocs.io/en/v7.15.1/) 执行此聚合时，我收到异常：

exception search() got multiple values for keyword argument 'size'

如果我删除该属性：

"size": 0,

从查询中不会引发异常，但聚合不会运行，因为聚合需要 "size": 0,。

在使用 Python ElasticSearch 客户端执行聚合时，我应该使用不同的查询格式吗？

更新：

这是用于调用查询的代码：

import elasticsearch
from elasticsearch import Elasticsearch, helpers

es_client = Elasticsearch(
    ["https://test-elastic.com"],
    scheme="https",
    port=443,
    http_auth=("test-user", "test-password"),
    maxsize=400,
    timeout=120,
    max_retries=10,
    retry_on_timeout=True
)

query_response = helpers.scan(client=es_client,
                                     query=search_query,
                                     index="test_index",
                                     clear_scroll=False,
                                     request_timeout=1500)

rows = []
try:
    for row in query_response:
        rows.append(row)
except Exception as e:
    print('exception' , e)
        

使用es_client：

es_client.search(index="test_index", query=search_query)

导致错误：

/opt/oss/conda3/lib/python3.7/site-packages/elasticsearch/connection/base.py in _raise_error(self, status_code, raw_data)
    336 
    337         raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
--> 338             status_code, error_message, additional_info
    339         )
    340 

RequestError: RequestError(400, 'parsing_exception', 'unknown query [aggs]')

aggs 对搜索 api 有效吗？

Answer 1

helpers.scan 是一个

scroll() api 之上的简单抽象 - 一个简单的迭代器，它产生所有命中，并通过下划线滚动请求返回。

它旨在遍历大型结果集，并带有 size=1000 的默认关键字参数

要运行聚合，请直接使用es_client.search() method，将查询作为body 传递，并在查询中包含"size": 0 应该没问题。