禁用动态映射在 ElasticSearch 中不起作用

Question

当我尝试在 ElasticSearch 设置中禁用动态映射时遇到错误。我正在使用 ElasticSearch 1.7 版本来实现。

堆栈跟踪：

8151 [main] WARN org.apache.hadoop.mapred.YarnChild - Exception running child : org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: Found unrecoverable error [10.74.51.71:9200] returned Not Found(404) - [TypeMissingException[[test_2017051222] type[[vehicle, trying to auto create mapping, but dynamic mapping is disabled]] missing]]; Bailing out..
    at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:207)
    at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:170)
    at org.elasticsearch.hadoop.rest.RestRepository.tryFlush(RestRepository.java:225)
    at org.elasticsearch.hadoop.rest.RestRepository.flush(RestRepository.java:248)
    at org.elasticsearch.hadoop.rest.RestRepository.doWriteToIndex(RestRepository.java:187)
    at org.elasticsearch.hadoop.rest.RestRepository.writeToIndex(RestRepository.java:163)
    at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:151)
    at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:566)
    at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
    at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
    at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150)
    at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
    at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:635)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

设置sn-p：

  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1,
    "index.query.default_field":"test",
    "index.refresh_interval" : "5s",
    "index.mapper.dynamic": false ,
    "analysis": {
         "filter": {
            "ngram_filter": {
               "type": "ngram",
               "min_gram": 2,
               "max_gram": 18,
               "token_chars": [
                  "letter",
                  "digit"
               ]
            }
         },
         "analyzer": {
            "ngram_analyzer": {
               "type": "custom",
               "tokenizer": "standard",
               "filter": [
                  "lowercase",
                  "ngram_filter"
               ]
            }
         }
      }
  }

我看到动态映射在 ES 端点的设置中被禁用，但作业失败。我有一个 avro json 映射文件和 es json 映射文件，其中 avro json 映射文件是超集，而 es json 映射文件是子集。我不希望超集映射文件中的所有字段都反映在 ES 索引上，而只转储子映射文件中的字段。是我做错了还是有其他方法。

谢谢。

Answer 1

那是因为您设置了"index.mapper.dynamic": false，这意味着如果不先声明新类型，则不会自动创建它们。

您想要做的是在您的类型映射中设置"dynamic": "false"。 PUT /test_index { "mappings": { "test_type": { "dynamic": "false" } } } 欲了解更多信息： https://www.elastic.co/guide/en/elasticsearch/guide/1.x/dynamic-mapping.html

例子：

1. 运行映射 PUT /my_index { "mappings": { "testing": { "dynamic": "false", "properties": { "field1": { "type": "string", "index": "analyzed" } } } } }

2. 以testing 类型为文档编制索引 POST /my_index/testing/1 { "field1":"demo", "field99":"anotherDemo" }

3. GET /my_index/testing/_mapping 的回复 { "my_index": { "mappings": { "testing": { "dynamic": "false", "properties": { "field1": { "type": "string" } } } } } }

正如您将看到的那样，field99 没有映射。