【问题标题】:Disabling dynamic mapping per index in Elastic Search 5.2在 Elastic Search 5.2 中禁用每个索引的动态映射
【发布时间】:2017-08-29 03:09:31
【问题描述】:

我正在尝试禁用 Elastic Search 索引的动态映射器。下面是我通过 Kibana 控制台运行的一系列查询以对其进行测试。

我认为添加 'index.mapper.dynamic' 应该在索引级别禁用它,对吗?

在这里编辑: 我还尝试将动态属性设置为'falsein the type. Strangely, if I post the document then it doesn't actually update the mapping, so if check_mapping`“filteredField”未列出,但是当我搜索文档时,filteredField 已被索引并且存在! -

弹性版:

{
  "name": "DpVBoAZ",
  "cluster_name": "elasticsearch",
  "cluster_uuid": "RX6axsbOTMatorw7s5AOXQ",
  "version": {
    "number": "5.2.0",
    "build_hash": "24e05b9",
    "build_date": "2017-01-24T19:52:35.800Z",
    "build_snapshot": false,
    "lucene_version": "6.4.0"
  },
  "tagline": "You Know, for Search"
}

但最终的 GET 会返回此文档,请注意 filteredField 是我添加到已发布文档中的一个字段,只是为了测试 - 它不应该出现在结果中。我还检查了映射,它已被动态添加到映射中。

我的创建索引查询有什么问题??

{
  "_index": "meu_locations",
  "_type": "location",
  "_id": "12345",
  "_version": 2,
  "found": true,
  "_source": {
    "__v": 0,
    "address1": "30 Mort Street",
    "email": "Braddon.Manager@stationeryhub.com.au",
    "faxNumber": "(02) 6122 0070",
    "geo": {
      "location": {
        "lon": 149.1317,
        "lat": -35.27433
      },
      "autogeocode": false
    },
    "identifier": "sample-store",
    "phoneNumber": "(02) 6122 0000",
    "postCode": "2612",
    "state": "ACT",
    "suburb": "Braddon",
    "title": "Hello New Title",
    "filteredField": "Hello There",
    "urlToken": "sample-store",
    "status": "Active",
    "country": "AU",
    "id": "Skcyxox6x"
  }
}

搜索查询

DELETE meu_locations

PUT meu_locations
{
  "settings": {
    "index.mapper.dynamic": false,
    "index.mapping.total_fields.limit": 2000,
    "analysis": {
      "filter": {
        "email": {
          "type": "pattern_capture",
          "preserve_original": 1,
          "patterns": [
            "([^@]+)",
            "(\\p{L}+)",
            "(\\d+)",
            "@(.+)",
            "([^-@]+)"
          ]
        }
      },
      "analyzer": {
        "case_insensitive_sort": {
          "tokenizer": "keyword",
          "filter": [
            "lowercase"
          ]
        },
        "email": {
          "tokenizer": "uax_url_email",
          "filter": [
            "email",
            "lowercase",
            "unique"
          ]
        }
      }
    }
  },
  "mappings": {
    "location": {
      "dynamic": false,
      "properties": {
        "title": {
          "type": "keyword"
        },
        "identifier": {
          "type": "keyword",
          "index": "not_analyzed"
        },
        "address1": {
          "type": "text"
        },
        "address2": {
          "type": "text"
        },
        "state": {
          "type": "keyword"
        },
        "suburb": {
          "type": "text"
        },
        "postCode": {
          "type": "keyword"
        },
        "country": {
          "type": "keyword"
        },
        "phoneNumber": {
          "type": "keyword"
        },
        "type": {
          "type": "keyword",
          "index": "not_analyzed"
        },
        "geo": {
          "properties": {
            "location": {
              "type": "geo_point"
            },
            "autogeocode": {
              "type": "boolean"
            }
          }
        },
        "services": {
          "properties": {
            "specialOrder": {
              "type": "boolean"
            },
            "assembleIt": {
              "type": "boolean"
            },
            "wifi": {
              "type": "boolean"
            },
            "officeFitout": {
              "type": "boolean"
            },
            "deliverIt": {
              "type": "boolean"
            },
            "techServices": {
              "type": "boolean"
            },
            "parking": {
              "type": "boolean"
            },
            "courtesyTrailer": {
              "type": "boolean"
            },
            "extraCover": {
              "type": "boolean"
            }
          }
        },
        "updatedAt": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "createdAt": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "publishDate": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "expiryDate": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "status": {
          "type": "keyword"
        }
      }
    }
  }
}

PUT meu_locations/location/12345
{
  "__v": 0,
  "address1": "30 Mort Street",
  "email": "Braddon.Manager@stationeryhub.com.au",
  "faxNumber": "(02) 6122 0070",
  "geo": {
    "location": {
      "lon": 149.1317,
      "lat": -35.27433
    },
    "autogeocode": false
  },
  "identifier": "sample-store",
  "phoneNumber": "(02) 6122 0000",
  "postCode": "2612",
  "state": "ACT",
  "suburb": "Braddon",
  "title": "Hello New Title",
  "filteredField": "Hello There",
  "urlToken": "sample-store",
  "status": "Active",
  "country": "AU",
  "id": "Skcyxox6x"
}

GET meu_locations/location/12345

【问题讨论】:

  • 我开始认为这是我对弹性映射工作原理的误解。我(有点明智地)假设如果一个字段没有被映射,那么该字段将不会存储在_source 中并且只会被删除。但是,弹性是否仍会将字段存储在 _source 中,但不会将字段映射保存到索引?这是我正在观察的。我保留字段的唯一选择是从源头进行过滤,然后发布一个完美的文档——真是太痛苦了。至少我可以通过将 strict 设置为 true 来阻止错误。但是如果该字段被丢弃会更好。谁能证实这一点?

标签: elasticsearch


【解决方案1】:

您必须在映射中"dynamic": "strict" 才能为您的索引禁用动态映射。

您需要修改设置中的映射部分。

PUT meu_locations2
{
  "settings": {
    "index.mapping.total_fields.limit": 2000,
    "analysis": {
      "filter": {
        "email": {
          "type": "pattern_capture",
          "preserve_original": 1,
          "patterns": [
            "([^@]+)",
            "(\\p{L}+)",
            "(\\d+)",
            "@(.+)",
            "([^-@]+)"
          ]
        }
      },
      "analyzer": {
        "case_insensitive_sort": {
          "tokenizer": "keyword",
          "filter": [
            "lowercase"
          ]
        },
        "email": {
          "tokenizer": "uax_url_email",
          "filter": [
            "email",
            "lowercase",
            "unique"
          ]
        }
      }
    }
  },
  "mappings": {
    "location": {
      "dynamic": false,
      "properties": {
        "title": {
          "type": "keyword"
        },
        "identifier": {
          "type": "keyword",
          "index": "not_analyzed"
        },
        "address1": {
          "type": "text"
        },
        "address2": {
          "type": "text"
        },
        "state": {
          "type": "keyword"
        },
        "suburb": {
          "type": "text"
        },
        "postCode": {
          "type": "keyword"
        },
        "country": {
          "type": "keyword"
        },
        "phoneNumber": {
          "type": "keyword"
        },
        "type": {
          "type": "keyword",
          "index": "not_analyzed"
        },
        "geo": {
          "properties": {
            "location": {
              "type": "geo_point"
            },
            "autogeocode": {
              "type": "boolean"
            }
          }
        },
        "services": {
          "properties": {
            "specialOrder": {
              "type": "boolean"
            },
            "assembleIt": {
              "type": "boolean"
            },
            "wifi": {
              "type": "boolean"
            },
            "officeFitout": {
              "type": "boolean"
            },
            "deliverIt": {
              "type": "boolean"
            },
            "techServices": {
              "type": "boolean"
            },
            "parking": {
              "type": "boolean"
            },
            "courtesyTrailer": {
              "type": "boolean"
            },
            "extraCover": {
              "type": "boolean"
            }
          }
        },
        "updatedAt": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "createdAt": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "publishDate": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "expiryDate": {
          "type": "date",
          "format": "strict_date_optional_time||epoch_millis"
        },
        "status": {
          "type": "keyword"
        }
      }
    }
  }
}

谢谢

【讨论】:

  • 我已经试过了。我不想抛出异常。我想忽略这些字段。我尝试在类型级别设置为 false,但无济于事。
  • 我已经编辑了我的答案,如果您不想抛出异常,请将其设置为 false。github.com/elastic/elasticsearch/issues/17880github.com/elastic/elasticsearch/issues/12358
  • 检查更新的问题。我也尝试过,即使它没有被映射,该字段也会被索引 - 并且是可搜索的。我确定这一定是一个错误......我更新了脚本以反映最新情况。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2013-08-15
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多