Elastic Search 何时添加动态映射

Question

我一直在使用 Elastic Search (ES) 动态映射时遇到问题。好像我在catch-22中。 https://www.elastic.co/guide/en/elasticsearch/guide/current/custom-dynamic-mapping.html

主要目标是将进入 ES 的所有内容都存储为字符串。

我尝试过的：

1. 在 ES 中，在索引创建完成之前，您无法创建动态映射 创建的。好吧，有道理。

2. 我无法创建空索引，所以如果 发送到索引的第一项不是字符串，我不能 重新分配它...我不知道什么类型的对象是第一个 索引中的项目，它可以是任何类型，这取决于应用如何接受各种对象/事件。

所以如果不能提前创建mapping，又不能插入空索引来创建mapping，又不能事后改变mapping，第一项怎么处理如果它不是字符串？？？

这是我目前正在做的事情（使用 Javascript 客户端）。

createESIndex = function (esClient){
    esClient.index({
        index: 'timeline-2015-11-21',
        type: 'event',
        body: event
    },function (error, response) {
        if (error) {
            logger.log(logger.SEVERITY.ERROR, 'acceptEvent elasticsearch create failed with: '+ error + " req:" + JSON.stringify(event));
            console.log(logger.SEVERITY.ERROR, 'acceptEvent elasticsearch create failed with: '+ error + " req:" + JSON.stringify(event));
            res.status(500).send('Error saving document');
        } else {
            res.status(200).send('Accepted');
        }
    });
}

esClientLookup.getClient( function(esClient) {

    esClient.indices.putTemplate({
        name: "timeline-mapping-template",
        body:{
            "template": "timeline-*",
            "mappings": {
                "event": {
                    "dynamic_templates": [
                        { "timestamp-only": {
                              "match":              "@timestamp",
                              "match_mapping_type": "date",
                              "mapping": {
                                  "type":           "date",
                              }
                        }},
                        { "all-others": {
                              "match":              "*",
                              "match_mapping_type": "string",
                              "mapping": {
                                  "type":           "string",
                              }
                            }
                        }
                    ]
                }
            }
        }
    }).then(function(res){
        console.log("put template response: " + JSON.stringify(res));
        createESIndex(esClient);

    }, function(error){
        console.log(error);
        res.status(500).send('Error saving document');
    });
});

Answer 1

Index templates 来救援！！这正是您所需要的，其想法是为您的索引创建一个模板，一旦您希望在该索引中存储一个文档，ES 就会使用您提供的映射（甚至是动态映射）为您创建它

curl -XPUT localhost:9200/_template/my_template -d '{
  "template": "index_name_*",
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "type_name": {
      "dynamic_templates": [
        {
          "strings": {
            "match": "*",
            "match_mapping_type": "*",
            "mapping": {
              "type": "string"
            }
          }
        }
      ],
      "properties": {}
    }
  }
}'

然后，当您索引名称与index_name_* 匹配的索引中的任何内容时，将使用上面的动态映射创建索引。

例如：

curl -XPUT localhost:9200/index_name_1/type_name/1 -d '{
  "one": 1,
  "two": "two", 
  "three": true
}'

这将创建一个名为index_name_1 的新索引，其映射类型为type_name，其中所有属性均为string。您可以使用

进行验证

curl -XGET localhost:9200/index_name_1/_mapping/type_name

回复：

{
  "index_name_1" : {
    "mappings" : {
      "type_name" : {
        "dynamic_templates" : [ {
          "strings" : {
            "mapping" : {
              "type" : "string"
            },
            "match" : "*",
            "match_mapping_type" : "*"
          }
        } ],
        "properties" : {
          "one" : {
            "type" : "string"
          },
          "three" : {
            "type" : "string"
          },
          "two" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

请注意，如果您愿意通过 Javascript API 执行此操作，可以使用 indices.putTemplate 调用。

Answer 2

export const user = {
  email: {
    type: 'text',
  },
};
export const activity = {
  date: {
    type: 'text',
  },
};
export const common = {
  name: {
    type: 'text',
  },
};

import { Client } from '@elastic/elasticsearch';
import { user } from './user';
import { activity } from './activity';
import { common } from './common';

export class UserDataFactory {
  private schema = {
    ...user,
    ...activity,
    ...common,
    relation_type: {
      type: 'join',
      eager_global_ordinals: true,
      relations: {
        parent: ['activity'],
      },
    },
  };
  constructor(private client: Client) {
    Object.setPrototypeOf(this, UserDataFactory.prototype);
  }
  async create() {
    const settings = {
      settings: {
        analysis: {
          normalizer: {
            useLowercase: {
              filter: ['lowercase'],
            },
          },
        },
      },
      mappings: {
        properties: this.schema,
      },
    };

    const { body } = await this.client.indices.exists({
      index: ElasticIndex.UserDataFactory,
    });

    await Promise.all([
      await (async (client) => {
        await new Promise(async function (resolve, reject) {
          if (!body) {
            await client.indices.create({
              index: ElasticIndex.UserDataFactory,
            });
          }
          resolve({ body });
        });
      })(this.client),
    ]);

    await this.client.indices.close({ index: ElasticIndex.UserDataFactory });

    await this.client.indices.putSettings({
      index: ElasticIndex.UserDataFactory,
      body: settings,
    });

    await this.client.indices.open({
      index: ElasticIndex.UserDataFactory,
    });

    await this.client.indices.putMapping({
      index: ElasticIndex.UserDataFactory,
      body: {
        dynamic: 'strict',
        properties: {
          ...this.schema,
        },
      },
    });
  }
}

包装器.ts

class ElasticWrapper {
  private _client: Client = new Client({
    node: process.env.elasticsearch_node,
    auth: {
      username: 'elastic',
      password: process.env.elasticsearch_password || 'changeme',
    },
    ssl: {
      ca: process.env.elasticsearch_certificate,
      rejectUnauthorized: false,
    },
  });
  get client() {
    return this._client;
  }
}

export const elasticWrapper = new ElasticWrapper();

index.ts

new UserDataFactory(elasticWrapper.client).create();