如何让 Elasticsearch 布尔匹配适用于多个字段

Question

我需要一些专家指导来尝试使布尔匹配正常工作。如果 both 'message' 匹配 'Failed password for'， 和 'path' 匹配 '/var/log/，我希望查询仅返回成功的搜索结果安全”。

这是我的查询：

curl -s -XGET 'http://localhost:9200/logstash-2015.05.07/syslog/_search?pretty=true' -d '{
    "filter" : { "range" : { "@timestamp" : { "gte" : "now-1h" } } },
    "query" : {
        "bool" : {
            "must" : [
                {  "match_phrase" : { "message" : "Failed password for" } },
                {  "match_phrase" : { "path"    : "/var/log/secure"     } }
            ]
        }
    }
} '

这是搜索输出的开始：

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 46,
    "max_score" : 13.308596,
    "hits" : [ {
      "_index" : "logstash-2015.05.07",
      "_type" : "syslog",
      "_id" : "AU0wzLEqqCKq_IPSp_8k",
      "_score" : 13.308596,
      "_source":{"message":"May  7 16:53:50 s_local@logstash-02 sshd[17970]: Failed password for fred from 172.28.111.200 port 43487 ssh2","@version":"1","@timestamp":"2015-05-07T16:53:50.554-07:00","type":"syslog","host":"logstash-02","path":"/var/log/secure"}
    }, ...

问题是，如果我将“/var/log/secure”更改为“var”，然后运行查询，我仍然会得到结果，只是分数较低。我理解 bool...must 构造意味着这里的两个匹配项都需要成功。如果“路径”与“/var/log/secure”不完全匹配，我想要的是 no 结果...

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 46,
    "max_score" : 10.354593,
    "hits" : [ {
      "_index" : "logstash-2015.05.07",
      "_type" : "syslog",
      "_id" : "AU0wzLEqqCKq_IPSp_8k",
      "_score" : 10.354593,
      "_source":{"message":"May  7 16:53:50 s_local@logstash-02 sshd[17970]: Failed password for fred from 172.28.111.200 port 43487 ssh2","@version":"1","@timestamp":"2015-05-07T16:53:50.554-07:00","type":"syslog","host":"logstash-02","path":"/var/log/secure"}
    },...

我检查了这些字段的映射以检查它们是否未被分析：

curl -X GET 'http://localhost:9200/logstash-2015.05.07/_mapping?pretty=true'

我认为这些字段未被分析，因此我相信搜索也不会被分析（基于我最近从 elasticsearch 阅读的一些培训文档）。这是下面此索引的输出 _mapping 的 sn-p。

      ....
      "message" : {
        "type" : "string",
        "norms" : {
          "enabled" : false
        },
        "fields" : {
          "raw" : {
            "type" : "string",
            "index" : "not_analyzed",
            "ignore_above" : 256
          }
        }
      },
      "path" : {
        "type" : "string",
        "norms" : {
          "enabled" : false
        },
        "fields" : {
          "raw" : {
            "type" : "string",
            "index" : "not_analyzed",
            "ignore_above" : 256
          }
        }
      },
      ....

我哪里错了，或者我在这里误解了什么？

Answer 1

正如 OP 中所述，您需要使用字段的“not_analyzed” 视图，但根据 OP 映射，字段的非分析版本是 message.raw， path.raw 示例：

{
    "filter" : { "range" : { "@timestamp" : { "gte" : "now-1h" } } },
    "query" : {
        "bool" : {
            "must" : [
                {  "match_phrase" : { "message.raw" : "Failed password for" } },
                {  "match_phrase" : { "path.raw"    : "/var/log/secure"     } }
            ]
        }
    }
}

.旁边的链接可以让您更深入地了解multi-fields

.进一步扩展

OP中path的映射如下：

"path" : {
        "type" : "string",
        "norms" : {
          "enabled" : false
        },
        "fields" : {
          "raw" : {
            "type" : "string",
            "index" : "not_analyzed",
            "ignore_above" : 256
          }
        }
      }

这指定路径字段使用默认分析器并且不分析field.raw。

如果您想将路径字段设置为不分析而不是原始的，则将是以下几行：

"path" : {
            "type" : "string",
            "index" : "not_analyzed",
            "norms" : {
              "enabled" : false
            },
            "fields" : {
              "raw" : {
                "type" : "string",
                "index" : <whatever analyzer you want>,
                "ignore_above" : 256
              }
            }
          }