【发布时间】：2017-03-17 07:50:00
【问题描述】：
我有一个运行 Elasticsearch 1.4.4 的旧集群。
我的集群包含约 110 亿个文档，所有初选的大小约为 4TB。
我现在正在升级到 Elasticsearch 5.2.2，这当然意味着要重新索引我的数据。我有一个单独的集群，目前正在发生这种情况。我正在从我的源数据库重新索引，因为我在原始索引上禁用了 _all 和 _source。
我现在已经重新索引了大约 7.5 亿个文档，并注意到我的新索引大小已经是 350GB。我做了一些数学运算，看起来索引在完全索引时会增长到 5.5TB 左右。这比1.4.4 索引多1.5TB。我没想到会这样。相反，我预计尺寸会减小，因为我已经删除了几个属性。这是正常的事情还是我做错了什么？ 5.2.2 中是否有不同的默认设置可以促进这种增长？
1.4.4 索引设置：
{
"index": {
"refresh_interval": "30s",
"number_of_shards": "20",
"creation_date": "1426251049131",
"analysis": {
"analyzer": {
"default": {
"filter": [
"icu_folding",
"icu_normalizer"
],
"type": "custom",
"tokenizer": "icu_tokenizer"
}
}
},
"uuid": "WdgnCLyITgmpb4DROegV3Q",
"version": {
"created": "1040499"
},
"number_of_replicas": "1"
}
}
1.4.4 索引映射：
{
"article": {
"_source": {
"enabled": false
},
"_all": {
"enabled": false
},
"properties": {
"date": {
"format": "dateOptionalTime",
"type": "date",
"doc_values": true
},
"has_enclosures": {
"type": "boolean"
},
"feed_subscribers": {
"type": "integer",
"doc_values": true
},
"feed_language": {
"index": "not_analyzed",
"type": "string"
},
"author": {
"norms": {
"enabled": false
},
"analyzer": "keyword",
"type": "string"
},
"has_pictures": {
"type": "boolean"
},
"title": {
"norms": {
"enabled": false
},
"type": "string"
},
"content": {
"norms": {
"enabled": false
},
"type": "string"
},
"has_video": {
"type": "boolean"
},
"url": {
"index": "not_analyzed",
"type": "string"
},
"feed_canonical": {
"type": "boolean"
},
"feed_id": {
"type": "integer",
"doc_values": true
}
}
}
}
5.2.2 索引设置：
{
"articles": {
"settings": {
"index": {
"refresh_interval": "-1",
"number_of_shards": "40",
"provided_name": "articles",
"creation_date": "1489604158595",
"analysis": {
"analyzer": {
"default": {
"filter": [
"icu_folding",
"icu_normalizer"
],
"type": "custom",
"tokenizer": "icu_tokenizer"
}
}
},
"number_of_replicas": "0",
"uuid": "LOeOcZb_TMCX6E_86uMyXQ",
"version": {
"created": "5020299"
}
}
}
}
}
5.2.2 索引映射：
{
"articles": {
"mappings": {
"article": {
"_all": {
"enabled": false
},
"_source": {
"enabled": false
},
"properties": {
"author": {
"type": "text",
"norms": false,
"analyzer": "keyword"
},
"content": {
"type": "text",
"norms": false
},
"date": {
"type": "date"
},
"feed_canonical": {
"type": "boolean"
},
"feed_id": {
"type": "integer"
},
"feed_subscribers": {
"type": "integer"
},
"title": {
"type": "text",
"norms": false
},
"url": {
"type": "keyword"
}
}
}
}
}
}
任何帮助都将不胜感激，因为在此集群上完全重新索引大约需要 30 天...谢谢！
【问题讨论】：
标签： elasticsearch