【发布时间】：2020-04-02 08:48:09
【问题描述】：
我尝试使用Elastic Search（6.8 版）从文本中查找最相似的标签，我希望得到相似标签的得分总和，而不是默认弹性搜索的计算（公式）。
例如，我创建 my_test_index 并插入三个文档：
POST my_test_index/_doc/17
{
"id": 17,
"tags": ["devops", "server", "hardware"]
}
POST my_test_index/_doc/20
{
"id": 20,
"tags": ["software", "application", "developer", "develop"]
}
POST my_test_index/_doc/21
{
"id": 21,
"tags": ["electronic", "electric"]
}
没有映射，默认如下：
{
"my_test_index" : {
"aliases" : { },
"mappings" : {
"_doc" : {
"properties" : {
"id" : {
"type" : "long"
},
"tags" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
},
"settings" : {
"index" : {
"creation_date" : "1585820383702",
"number_of_shards" : "5",
"number_of_replicas" : "1",
"uuid" : "05SgLog6S-GTSShTatrvQw",
"version" : {
"created" : "6080199"
},
"provided_name" : "my_test_index"
}
}
}
}
所以，我请求以下查询：
GET my_test_index/_search
{
"query": {
"more_like_this": {
"fields": [
"tags"
],
"like": [
"i like electric devices and develop some softwares."
],
"min_term_freq": 1,
"min_doc_freq": 1
}
}
}
得到这个响应：
{
"took" : 4,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 0.2876821,
"hits" : [
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "21",
"_score" : 0.2876821,
"_source" : {
"id" : 21,
"tags" : [
"electronic",
"electric"
]
}
},
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "20",
"_score" : 0.2876821,
"_source" : {
"id" : 20,
"tags" : [
"software",
"application",
"developer",
"develop"
]
}
}
]
}
}
如果我设置解释：true，结果是：
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 0.2876821,
"hits" : [
{
"_shard" : "[my_test_index][1]",
"_node" : "maQL1REnQHaff51ekrqMxA",
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "21",
"_score" : 0.2876821,
"_source" : {
"id" : 21,
"tags" : [
"electronic",
"electric"
]
},
"_explanation" : {
"value" : 0.2876821,
"description" : "weight(tags:electric in 0) [PerFieldSimilarity], result of:",
"details" : [
{
"value" : 0.2876821,
"description" : "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
"details" : [
{
"value" : 0.2876821,
"description" : "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
"details" : [
{
"value" : 1.0,
"description" : "docFreq",
"details" : [ ]
},
{
"value" : 1.0,
"description" : "docCount",
"details" : [ ]
}
]
},
{
"value" : 1.0,
"description" : "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
"details" : [
{
"value" : 1.0,
"description" : "termFreq=1.0",
"details" : [ ]
},
{
"value" : 1.2,
"description" : "parameter k1",
"details" : [ ]
},
{
"value" : 0.75,
"description" : "parameter b",
"details" : [ ]
},
{
"value" : 2.0,
"description" : "avgFieldLength",
"details" : [ ]
},
{
"value" : 2.0,
"description" : "fieldLength",
"details" : [ ]
}
]
}
]
}
]
}
},
{
"_shard" : "[my_test_index][2]",
"_node" : "maQL1REnQHaff51ekrqMxA",
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "20",
"_score" : 0.2876821,
"_source" : {
"id" : 20,
"tags" : [
"software",
"application",
"developer",
"develop"
]
},
"_explanation" : {
"value" : 0.2876821,
"description" : "weight(tags:develop in 0) [PerFieldSimilarity], result of:",
"details" : [
{
"value" : 0.2876821,
"description" : "score(doc=0,freq=1.0 = termFreq=1.0\n), product of:",
"details" : [
{
"value" : 0.2876821,
"description" : "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",
"details" : [
{
"value" : 1.0,
"description" : "docFreq",
"details" : [ ]
},
{
"value" : 1.0,
"description" : "docCount",
"details" : [ ]
}
]
},
{
"value" : 1.0,
"description" : "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
"details" : [
{
"value" : 1.0,
"description" : "termFreq=1.0",
"details" : [ ]
},
{
"value" : 1.2,
"description" : "parameter k1",
"details" : [ ]
},
{
"value" : 0.75,
"description" : "parameter b",
"details" : [ ]
},
{
"value" : 4.0,
"description" : "avgFieldLength",
"details" : [ ]
},
{
"value" : 4.0,
"description" : "fieldLength",
"details" : [ ]
}
]
}
]
}
]
}
}
]
}
}
但是，这对我来说不合适，我想计算类似标签的得分总和，如下所示： 我在文本和标签中有“electric”字样，等于“electric”标签，它得到 1.0 分并且与“electrical”标签相似，它得到〜0.7分。 而文本和标签中的“develop”字，等于“develop”标签，得1.0分，与“developer”标签相似，它获得 ~0.8 分，与“softwares”的相似度约为 0.9 分，依此类推...
所以，我希望这个结果==> _id:20 的总分= ~2.7, _id:21= ~1.7 和 ....
我希望有人可以举例说明如何做到这一点，或者至少为我指明正确的方向。
谢谢。
【问题讨论】：
-
感谢您提供大部分信息，如果您可以添加映射，我可以快速复制并提供答案
-
@OpsterElasticsearchNinja 没有映射，是默认的。
-
不确定，为什么在您的结果中，两个 id 的分数相同，我尝试为带有
text字段和 id 21 的标签创建自己的映射，正如预期的那样，分数比 20 高很多
标签： elasticsearch lucene text-mining scoring elasticsearch-query