【问题标题】:SearchTemplate with sorting and paging variables in Elasticsearch 6.6 and NEST 6.6在 Elasticsearch 6.6 和 NEST 6.6 中具有排序和分页变量的 SearchTemplate
【发布时间】:2019-02-11 00:40:03
【问题描述】:

所有, 我正在尝试调用 ES 6.6 中定义的 SearchTemplate。该模板具有分页变量(来自和大小)和我在数组中传递的电子邮件。这也具有自定义脚本逻辑的排序。当我在 kibana 中运行它时,我看不到分页和排序不起作用。我将不胜感激任何帮助使其发挥作用。请参阅下面的详细信息。我使用索引别名搜索了两个索引。

人员和访客索引的映射是相同的(只是为了简化示例)

索引映射

PUT _template/person_guest_template
{
  "order": 0,
  "index_patterns": ["person*","guest*"],
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "autoComplete_filter": {
            "type": "edge_ngram",
            "min_gram": "2",
            "max_gram": "20"
          }
        },
        "analyzer": {
          "autoComplete": {
            "filter": ["lowercase", "asciifolding","autoComplete_filter"],
            "type": "custom",
            "tokenizer": "whitespace"
          },
          "default": {
            "filter": ["lowercase", "asciifolding"],
            "type": "custom",
            "tokenizer": "whitespace"
          }
        }
      },
      "number_of_shards": "3",
      "number_of_replicas": "1"
    }
  },
  "mappings": {
    "_doc": {
      "dynamic": false,
      "properties": {
        "firstName": {
          "type": "keyword",
          "fields": {
            "search": {
              "type": "text",
              "analyzer": "autoComplete",
              "search_analyzer": "default"
            }
          }
        },
        "lastName": {
          "type": "keyword",
          "fields": {
            "search": {
              "type": "text",
              "analyzer": "autoComplete",
              "search_analyzer": "default"
            }
          }
        },
        "email": {
          "type": "keyword"
        },"email": {
      "type": "keyword"
    }
      }
    }
  }
}

搜索模板定义

POST _scripts/guest_person_by_email
{
  "script": {
    "from": "{{from}}{{^from}}0{{/from}}",
    "size": "{{size}}{{^size}}5{{/size}}",
    "sort": [
      {
        "_script": {
          "order": "asc",
          "type": "number",
          "script": "return (doc['type'].value == 'person')? 0 : 1;"
        }
      },
      {
        "firstName": {
          "order": "asc"
        }
      },
      {
        "lastName": {
          "order": "asc"
        }
      }
    ],
    "lang": "mustache",
    "source": """
    {
      "query":{
        "bool":{
          "filter":{
            "terms":{
              "email":
              {{#toJson}}emails{{/toJson}}
            }
          }
        }
      }
    }
"""
  }
}

使用 SearchTemplate 搜索

GET guest-person/_search/template
{
  "id":"guest_person_by_email",
  "params": {
    "emails":["rennishj@test.com"]
  }
}

样本数据

PUT person/_doc/1
{
  "firstName": "Rennish",
  "lastName": "Joseph",
  "email": [
    "rennishj@test.com"
  ],
  "type":"person"
}

使用 NEST 6.6 调用搜索模板

List<string> emails = new List<string>(){"rennishj@test.com"};
var searchResponse = client.SearchTemplate<object>(st => st
    .Index("guest-person")
    .Id("guest_person_by_email")
    .Params(p => p
        .Add("emails", emails.ToArray())
        .Add("from", 0)     
        .Add("size", 50)
    )
);

观察

  1. 当我从 searchtemplate 中删除 from、size 和 sort 逻辑时,它可以工作
  2. 好像我将 sort 和 from/size 变量放在了错误的位置?

我在 https://discuss.elastic.co/t/c-nest-5-search-with-template/104074/2 找到了类似的帖子,但似乎 GetSearchTemplate 和 PutSearchTemplate 在 NEST 6.x 上已停用

这可以使用搜索模板来完成吗?我们使用一些非常复杂的 NEST 查询,并且正在远离 NEST 并使用搜索模板。

【问题讨论】:

    标签: elasticsearch nest elasticsearch-template


    【解决方案1】:

    有几个问题

    1. 索引模板定义"email"字段映射两次
    2. 索引模板将"dynamic"设置为false,但不包含"type"字段映射,因此脚本排序会失败
    3. 需要在 "source" 中为 Put Script API 调用定义整个搜索请求

    NEST 有助于构建正确的搜索请求并将它们用作搜索模板的基础,此外还有其他一大堆使用客户端的原因,例如循环请求、自动故障转移和重试等。

    这是一个完整的例子

    private static void Main()
    {
        var defaultIndex = "person";
        var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    
        var settings = new ConnectionSettings(pool)
            .DefaultIndex(defaultIndex)
            .DefaultTypeName("_doc");
    
        var client = new ElasticClient(settings);
    
        // WARNING: This deletes the index to make this code repeatable.
        // You probably want to remove this if copying verbatim
        if (client.IndexExists(defaultIndex).Exists)
            client.DeleteIndex(defaultIndex);
    
        var indexTemplateResponse = client.LowLevel.IndicesPutTemplateForAll<PutIndexTemplateResponse>(
            "person_guest_template",
            @"{
              ""order"": 0,
              ""index_patterns"": [""person*"",""guest*""],
              ""settings"": {
                ""index"": {
                  ""analysis"": {
                    ""filter"": {
                      ""autoComplete_filter"": {
                        ""type"": ""edge_ngram"",
                        ""min_gram"": ""2"",
                        ""max_gram"": ""20""
                      }
                    },
                    ""analyzer"": {
                      ""autoComplete"": {
                        ""filter"": [""lowercase"", ""asciifolding"",""autoComplete_filter""],
                        ""type"": ""custom"",
                        ""tokenizer"": ""whitespace""
                      },
                      ""default"": {
                        ""filter"": [""lowercase"", ""asciifolding""],
                        ""type"": ""custom"",
                        ""tokenizer"": ""whitespace""
                      }
                    }
                  },
                  ""number_of_shards"": ""3"",
                  ""number_of_replicas"": ""1""
                }
              },
              ""mappings"": {
                ""_doc"": {
                  ""dynamic"": false,
                  ""properties"": {
                    ""firstName"": {
                      ""type"": ""keyword"",
                      ""fields"": {
                        ""search"": {
                          ""type"": ""text"",
                          ""analyzer"": ""autoComplete"",
                          ""search_analyzer"": ""default""
                        }
                      }
                    },
                    ""lastName"": {
                      ""type"": ""keyword"",
                      ""fields"": {
                        ""search"": {
                          ""type"": ""text"",
                          ""analyzer"": ""autoComplete"",
                          ""search_analyzer"": ""default""
                        }
                      }
                    },
                    ""email"": {
                      ""type"": ""keyword""
                    },
                    ""type"": {
                      ""type"": ""keyword""
                    }
                  }
                }
              }
            }");
    
        // build a prototype search request     
        var searchRequest = new SearchRequest
        {
            From = 0,
            Size = 0,
            Sort = new List<ISort> 
            {
                new ScriptSort
                {
                    Order = Nest.SortOrder.Ascending,
                    Type = "number",
                    Script = new InlineScript("return (doc['type'].value == 'person')? 0 : 1;")
                },
                new SortField
                {
                    Field = "firstName",
                    Order = Nest.SortOrder.Ascending
                },
                new SortField
                {
                    Field = "lastName",
                    Order = Nest.SortOrder.Ascending
                }
            },
            Query = new BoolQuery
            {
                Filter = new QueryContainer[] 
                {
                    new TermsQuery
                    {
                        Field = "email",
                        Terms = new[] { "emails" }
                    }
                }
            }
        };
    
        var json = client.RequestResponseSerializer.SerializeToString(searchRequest);
        // create template from prototype search request
        var jObject = JsonConvert.DeserializeObject<JObject>(json); 
        jObject["from"] = "{{from}}{{^from}}0{{/from}}";
        jObject["size"] = "{{size}}{{^size}}5{{/size}}";    
        json = jObject.ToString(Newtonsoft.Json.Formatting.None);
        // below is invalid JSON, so can only be constructed with replacement
        json = json.Replace("[\"emails\"]", "{{#toJson}}emails{{/toJson}}");
    
        // add search template
        var putScriptResponse = client.PutScript("guest_person_by_email", s => s
            .Script(sc => sc
                .Lang(ScriptLang.Mustache)
                .Source(json)
            )
        );
    
        var person = new Person
        {
            FirstName = "Rennish",
            LastName = "Joseph",
            Email = new[] { "rennishj@test.com" }
        };
    
        // index document
        var indexResponse = client.Index(person, i => i.Id(1).Refresh(Refresh.WaitFor));
    
        // search
        var searchResponse = client.SearchTemplate<Person>(s => s
            .Id("guest_person_by_email")
            .Params(p => p
                .Add("emails", person.Email)
                .Add("from", 0)
                .Add("size", 50)
            )
        );
    }
    
    public class Person 
    {
        public string FirstName {get;set;}
        public string LastName { get; set; }
        public string[] Email {get;set;}
        public string Type {get; set;} = "person";
    }
    

    搜索模板请求的结果是

    {
      "took" : 47,
      "timed_out" : false,
      "_shards" : {
        "total" : 3,
        "successful" : 3,
        "skipped" : 0,
        "failed" : 0
      },
      "hits" : {
        "total" : 1,
        "max_score" : null,
        "hits" : [
          {
            "_index" : "person",
            "_type" : "_doc",
            "_id" : "1",
            "_score" : null,
            "_source" : {
              "firstName" : "Rennish",
              "lastName" : "Joseph",
              "email" : [
                "rennishj@test.com"
              ],
              "type" : "person"
            },
            "sort" : [
              0.0,
              "Rennish",
              "Joseph"
            ]
          }
        ]
      }
    }
    

    【讨论】:

    • 感谢 Russ Cam 抽出时间详细介绍。在我们的例子中,我们将创建搜索模板并对 NEST 之外的文档进行索引,并且只会从 NEST 中调用它。我们还能在模板中定义 size/from 和脚本排序并且只调用它吗?我们的查询有太多的逻辑,用 NEST 组合它们是一项相当困难的任务。
    • 是的,您可以定义模板并从任何 HTTP 客户端使用 Put Script API :)
    【解决方案2】:

    添加正确的 SearchTemplate(如 Russ Cam 指出的那样,在“source”下移动分页和排序)以防将来有人需要它。

    POST _scripts/guest_person_by_email
    {
      "script": {    
        "lang": "mustache",
        "source": """
         {
           "from": "{{from}}{{^from}}0{{/from}}",
           "size": "{{size}}{{^size}}5{{/size}}",
          "sort": [
          {
            "_script": {
              "order": "asc",
              "type": "number",
              "script": "return (doc['type'].value == 'person')? 0 : 1;"
            }
          },
          {
            "firstName": {
              "order": "asc"
            }
          },
          {
            "lastName": {
              "order": "asc"
            }
          }
        ],
          "query":{
            "bool":{
              "filter":{
                "terms":{
                  "email":
                  {{#toJson}}emails{{/toJson}}
                }
              }
            }
          }
        }
    """
      }
    }
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2012-12-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2012-04-05
      相关资源
      最近更新 更多