带有通配符和转义字符的弹性搜索查询

Question

我们在 freebsd 11 上使用 elasticsearch。 我们在表中有很多数据 ipv4、ipv6 格式。

客户希望使用通配符进行搜索。 例如

*192* -> no problem
*192.168.* -> no problem
*2001:db8* -> take error
*2001\:db8 -> take error....

我没有从 Elastic 中获取正确的数据。特别是“：”是非常有问题的。

我的系统信息和查询结果是

弹性信息这个

{
  "name": "WxaxEg6",
  "cluster_name": "elasticsearch",
  "cluster_uuid": "o-7IPcD3RjODelTyPYUBJw",
  "version": {
    "number": "5.6.8",
    "build_hash": "688ecce",
    "build_date": "2018-02-16T16:46:30.010Z",
    "build_snapshot": false,
    "lucene_version": "6.6.1"
  },
  "tagline": "You Know, for Search"
}

我的测试表是

{
  "ip_test2": {
    "aliases": {},
    "mappings": {
      "doc": {
        "properties": {
          "ip_addr": {
            "type": "text"
          }
        }
      }
    },
    "settings": {
      "index": {
        "creation_date": "1549119687946",
        "number_of_shards": "5",
        "number_of_replicas": "1",
        "uuid": "Aljv_81nQDKx3B3Fs2nVOA",
        "version": {
          "created": "5060899"
        },
        "provided_name": "ip_test2"
      }
    }
  }
}

查询 1：

{
    "query": {
        "query_string" : {
            "fields" : ["ip_addr"],
            "query": "*192.*",
            "analyze_wildcard": true
       }
    }
}'

结果是

{"took":3,"timed_out":false,"_shards": 
{"total":5,"successful":5,"skipped":0,"failed":0},"hits": 
{"total":255,"max_score":1.0,"hits": 
 ...:{"ip_addr": "192.168.1.4"}},

没问题。

查询 2

 "query": "*2001*",

结果是

{"took":5,"timed_out":false,"_shards": 
{"total":5,"successful":5,"skipped":0,"failed":0},"hits": 
{"total":100,"max_score":1.0,"hits": 
...:{"ip_addr": "2001:db8:100:0:2359:8a17:17c6:e316"}},

没问题。 现在问题开始。 查询

"query": "*2001:*",

结果

"error":{"root_cause": 
[{"type":"query_shard_exception","reason":"Failed to parse query 
[*2001:*]","index_uuid":"Aljv_81nQDKx3B3Fs2nVOA","index":"ip_test2"}]

查询

"query": "\"*2001:db*\"",

结果是错误，有很多2001:db8开头的数据

"took":1,"timed_out":false,"_shards": 
{"total":5,"successful":5,"skipped":0,"failed":0},"hits": 
{"total":0,"max_score":null,"hits":[]}}

查询是

"query": "\"*2001:db8*\"",

结果正确，太棒了....WHYYYY

{"took":2,"timed_out":false,"_shards": 
{"total":5,"successful":5,"skipped":0,"failed":0},"hits": 
{"total":100,"max_score":1.8449252,

字段类型不是 ip ，我不明白不同的结果。

谁能给我解释一下

我最后的解决方案是

{"from":0,"size":100,"sort":[{"start_time": 
{"order":"desc","unmapped_type":"boolean"}}],
"query":{"bool":{"must":[{"range":{"start_time": 
{"gte":1546678703407,"lte":1549270703407,"format":"epoch_millis"}}},
            {"bool":{"should":[{"multi_match": 
{"query":"2001:db","fields":["ip_dst_saddr"],"type":"phrase_prefix"}},
                       {"query_string":{"query":"*2001\\:db*","fields": 
["ip_dst_saddr"],"analyze_wildcard":true}}]}}]}}}

Answer 1

对于查询

"query": "*2001:*" 您需要转义冒号（更多示例请参见here），因此请尝试使用"query": "*2001\\:*"

那么对于其他查询，您不能在词组匹配中使用通配符（有关详细信息，请参阅here）

如果您在查询中使用验证 API：

POST <your_index>/_validate/query?explain=true
{
    "query": {
        "query_string" : {
            "fields" : ["ip_addr"],
            "query": "\"*2001:db*\"",
            "analyze_wildcard": true
       }
    }
}

你会看到这个查询被解释为

"explanations": [
    {
      "index": "<your_index>",
      "valid": true,
      "explanation": """ip_addr:"2001 db""""
    }
  ]

和"query": "\"*2001:db8*\"" 到

"explanations": [
    {
      "index": "test_so",
      "valid": true,
      "explanation": """ip_addr:"2001 db8""""
    }
  ]

所以查询"query": "\"*2001:db*\"" 将只匹配具有相同顺序标记“2001”和“db”的文档（完整标记“db”，而不是“db8”或其他任何内容） 并且查询"query": "\"*2001:db8*\"" 将以相同的顺序匹配包含“2001”和“db8”的任何文档。

您应该真正使用 elasticsearch 的 IP 数据类型。