ElasticSearch 查询以排除某些结果

Question

我正在尝试编写一个允许过滤结果集的 ElasticSearch 查询。该应用程序为职位提供了一个过滤器，也为相同的职位提供了一个排除过滤器。所以例如在下面的数据集中，我想过滤Engineer，但也排除Software Engineer。问题是现在查询也排除了Principal Software Engineer，它不应该。

这是我正在使用的数据：

{
  "data": [
    {
      "email": "chris@example.com",
      "job_title": "Industrial Electrical Engineer"
    },
    {
      "email": "esther@example.com",
      "job_title": "Chief Revenue Officer"
    },
    {
      "email": "george@example.com",
      "job_title": "Principal Software Engineer"
    },
    {
      "email": "helena@example.com",
      "job_title": "Software Engineer"
    },
    {
      "email": "john@example.com",
      "job_title": "Engineer"
    },
    {
      "email": "judith@example.com",
      "job_title": "Design Engineer"
    },
    {
      "email": "katy@example.com",
      "job_title": "Software Designer"
    },
    {
      "email": "mark@example.com",
      "job_title": "Engineer"
    },
    {
      "email": "mary@example.com",
      "job_title": "Mechanical Design Engineer"
    },
    {
      "email": "tom@example.com",
      "job_title": "Electrical Engineer"
    },
    {
      "email": "tony@example.com",
      "job_title": "Chief Executive Officer"
    }
  ]
}

这里是 ElasticSearch 查询：

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "job_title": {
                    "query": "Engineer",
                    "operator": "and"
                  }
                }
              }
            ]
          }
        }
      ],
      "filter": [
        {
          "term": {
            "user_id": 1
          }
        },
        {
          "bool": {
            "must_not": [
              {
                "match": {
                  "job_title": "Software Engineer"
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Answer 1

假设job_title 是text 类型。如果未指定分析器，Elasticsearch 将 standard analyzer 用于 text 类型字段。这会将“软件工程师”分解为

{
  "tokens": [
    {
      "token": "software",
      "start_offset": 0,
      "end_offset": 8,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "engineer",
      "start_offset": 9,
      "end_offset": 17,
      "type": "<ALPHANUM>",
      "position": 1
    }
  ]
}

因此，当使用must_not 进行查询并匹配"Software Engineer" 的查询时，它不会包含任何包含software 或engineer 的结果

---

如果您没有明确定义任何映射，那么您需要将 .keyword 添加到 job_title 字段。这使用关键字分析器而不是标准分析器（注意 job_title 字段后的“.keyword”）。

将您的查询修改为

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "job_title": {
                    "query": "Engineer",
                    "operator": "and"
                  }
                }
              }
            ]
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must_not": [
              {
                "term": {
                  "job_title.keyword": "Software Engineer"
                }
              }
            ]
          }
        }
      ]
    }
  }
}

更新 1：

如果您使用的是 elasticsearch 7.10 或更高版本，并且您希望搜索不区分大小写以及搜索确切的术语，那么您可以使用case_insensitive param with the term query.

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              {
                "match": {
                  "job_title": {
                    "query": "Engineer",
                    "operator": "and"
                  }
                }
              }
            ]
          }
        }
      ],
      "filter": [
        {
          "bool": {
            "must_not": [
              {
                "term": {
                  "job_title.keyword": {
                    "value": "software engineer",
                    "case_insensitive": true
                  }
                }
              }
            ]
          }
        }
      ]
    }
  }
}

否则，如果你使用的是7.10以下的版本，那么你需要修改你的索引映射，如下图，然后重新索引数据

{
  "settings": {
    "analysis": {
      "normalizer": {
        "my_normalizer": {
          "type": "custom",
          "filter": [
            "lowercase"
          ]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "job_title": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "normalizer": "my_normalizer"
          }
        }
      }
    }
  }
}

Answer 2

您可以在 'must_not' 子句中使用 match phrase 来仅排除确切的短语 'Software Engineer'。