当在 Logstash 中定义 Elasticsearch 索引模板时，Logstash 抛出 [message=>"undefined method `update' for nil:NilClass"]

Question

我正在尝试使用 logstash 将数据从 sql server 加载到弹性搜索。对于索引创建，我使用在 logstash 的配置文件中指定的索引模板。

logstash.config 的输出字段：

 output{ 
    stdout { codec => json_lines } 
    elasticsearch { 
                    hosts => ["localhost:9200"] 
                                             document_id => "%{[id]}" 
                                             manage_template => true                     
                                             template => "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json" 
                                             template_name => "logstash_test" 
                                             template_overwrite => true 
            } 
    } 

运行 logstash 时出错：

    [2021-01-25T19:48:28,232][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]} 
[2021-01-25T19:48:28,298][INFO ][logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"} 
[2021-01-25T19:48:28,370][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/sampleconf.conf"], :thread=>"#<Thread:0x797ce1a8 run>"} 
[2021-01-25T19:48:28,509][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:42:in `add_ilm_settings_to_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]} 
[2021-01-25T19:48:28,546][INFO ][logstash.outputs.elasticsearch][main] Creating rollover alias <logstash-{now/d}-000001> 
[2021-01-25T19:48:29,463][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.08} 
[2021-01-25T19:48:29,802][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"} 
[2021-01-25T19:48:29,888][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]} 

索引模板配置：

{ 
   "index_patterns":"*-logstash_test-*", 
   "priority":1, 
   "template":{ 
      "settings":{ 
            "analysis":{ 
               "analyzer":{ 
                  "test_autocomplete":{ 
                     "tokenizer":"test_tokenizer" 
                  } 
               }, 
               "tokenizer":{ 
                  "test_tokenizer":{ 
                     "custom_token_chars":"'-", 
                     "max_gram":"3", 
                     "min_gram":"3", 
                     "token_chars":[ 
                         
                     ], 
                     "type":"ngram" 
                  } 
               } 
            }, 
         "mappings":{ 
            "properties":{ 
               "test_id":{ 
                  "type":"text", 
                  "analyzer":"test_autocomplete" 
               } 
            } 
         } 
      } 
   } 
} 

虽然使用数据创建索引失败，但settings{..} 中指定的分析器不会反映。我是 ELK 堆栈的新手，不知道为什么会发生这种情况。

Elasticsearch 版本：7.10.1

Logstash 版本：7.10.1

Answer 1

确保您的logstash_test.json 文件具有以下格式：

{
  "index_patterns": "*-logstash_test-*",
  "order": 1,
  "settings": {
    "analysis": {
      "analyzer": {
        "test_autocomplete": {
          "tokenizer": "test_tokenizer"
        }
      },
      "tokenizer": {
        "test_tokenizer": {
          "custom_token_chars": "'-",
          "max_gram": "3",
          "min_gram": "3",
          "token_chars": [],
          "type": "ngram"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "test_id": {
        "type": "text",
        "analyzer": "test_autocomplete"
      }
    }
  }
}

mappings 和 settings 包含在 template 部分中，但这仅适用于 elasticsearch Logstash 输出尚不支持的新 index templates。您需要使用legacy index templates。