更新 Elasticsearch _mapping 中的字符串参数

Question

我在Elasticsearch6.8中有这样一个_mapping：

{
  "grch38_test__wes__grch38__variants__20210222" : {
    "mappings" : {
      "variant" : {
        "_meta" : {
          "gencodeVersion" : "25",
          "hail_version" : "0.2.20",
          "genomeVersion" : "38",
          "sampleType" : "WES",
          "sourceFilePath" : "s3://my_folder/my_vcf.vcf"
        },
    ...

我的目标是在Kibana 中发出查询以修改variant._meta.sourceFilePath。跟帖：

Elastic search mapping for nested json objects

我能够提出查询：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "variant": {
      "type": "nested",
      "properties": {
        "_meta": {
          "type": "nested",
          "properties": {
            "type": "text",
            "sourceFilePath": "s3://my_folder/my_vcf.vcf"
          }
        }
      }
    }
  }
}

但它给了我一个错误：

elasticsearch mapping Expected map for property [fields] on field [name] but got a class java.lang.String

完整的错误信息：

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Expected map for property [fields] on field [type] but got a class java.lang.String"
  },
  "status": 400
}

我也试过了：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "variant": {
      "type": "nested",
      "properties": {
        "_meta": {
          "type": "nested",
          "properties": {
            "sourceFilePath": {
              "type": "text",
              "value":"s3://my_folder/my_vcf.vcf"
            }
          }
        }
      }
    }
  }
}

但它告诉我 value 不受支持：

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Mapping definition for [sourceFilePath] has unsupported parameters:  [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Mapping definition for [sourceFilePath] has unsupported parameters:  [value : s3://seqr-dp-data--prod/vcf/dev/grch38_test_contracted.vcf]"
  },
  "status": 400
}

我做错了什么？如何修改字段？

Answer 1

_meta 是storing application-specific metadata 的保留字段。它不是可搜索的，只能通过GET Mapping API 检索。

这意味着，如果您的 _meta 内容旨在与 _meta 字段的设计用途一致，则您不能对其应用任何映射。它是具体值的“最终”哈希图，需要在更新映射负载的顶层定义：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "_meta": {
    "variant": {            <-- shared index-level metadata
      "gencodeVersion": "25",
      "hail_version": "0.2.20",
      "genomeVersion": "38",
      "sampleType": "WES",
      "sourceFilePath": "s3://my_folder/my_vcf.vcf"
    }
  },
  "properties": {
    "some_text_field": {    <-- actual document properties
      "type": "text" 
    }
  }
}

另一方面，如果您的 _meta 字段是一个不幸的命名巧合，您可以像这样声明它的映射：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant
{
  "properties": {
    "_meta": {
      "properties": {
        "variant": {
          "properties": {
            "gencodeVersion": {
              "type": "text"
            },
            "genomeVersion": {
              "type": "text"
            },
            "hail_version": {
              "type": "text"
            },
            "sampleType": {
              "type": "text"
            },
            "sourceFilePath": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}

并摄取表单的文档：

POST grch38_test__wes__grch38__variants__20210222/variant/_doc
{
  "_meta": {
    "variant": {
      "gencodeVersion": "25",
      "hail_version": "0.2.20",
      "genomeVersion": "38",
      "sampleType": "WES",
      "sourceFilePath": "s3://my_folder/my_vcf.vcf"
    }
  }
}

但同样，_meta 内容将是文档特定的，而不是索引范围的！

顺便说一句，nested 映射仅在您处理arrays of objects 时才有意义，而不是对象的对象。 但如果你坚持想要它，你会这样做：

PUT /grch38_test__wes__grch38__variants__20210222/_mapping/variant?include_type_name
{
  "properties": {
    "_meta": {
      "type": "nested",            <---
      "properties": {
        "variant": {
          "type": "nested",        <---
          "properties": {
            "gencodeVersion": {
              "type": "text"
            },
            "genomeVersion": {
              "type": "text"
            },
            "hail_version": {
              "type": "text"
            },
            "sampleType": {
              "type": "text"
            },
            "sourceFilePath": {
              "type": "text"
            }
          }
        }
      }
    }
  }
}