【问题标题】:Logstash not writing to Elasticsearch with ShieldLogstash 不使用 Shield 写入 Elasticsearch
【发布时间】:2015-08-27 04:21:12
【问题描述】:

我一直试图让logstash用屏蔽写到elasticseach,但没有成功。

在将屏蔽插件安装到 elasticsearch 之前,我的设置工作正常。 我关注了this guide from elastic.co 并为logstash user role 创建了一个新用户:

esusers useradd logstashadmin -r logstash

我还按照指南中的建议更新了logstash output configuration 并添加了protocoluserpassword

重新启动 logstash 和 elasticsearch 后,我仍然没有收到来自 logstash 的弹性搜索的任何内容。我错过了什么吗?

这是我的设置:

$ esusers roles logstashadmin                         
logstashadmin  : logstash

$ cat shield/roles.yml
...
# The required role for logstash users
logstash:
  cluster: indices:admin/template/get, indices:admin/template/put
  indices:
    'logstash-*': indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index
...


$ cat logstash/output.conf

output {
  elasticsearch {
    protocol => "http"
    cluster => "logstash"
    user => "logstashadmin"
    password => "logstashadmin123"
  }
}

注意:我还在 logstash 中安装了 transport 插件,并尝试了它而不是 protocol => "http",结果相同。

如果您需要更多信息,请告诉我。谢谢

编辑 1:

弹性搜索日志:

[2015-06-12 05:59:16,952][INFO ][node                     ] [Silver Sable] stopping ...
[2015-06-12 05:59:17,087][INFO ][shield.license           ] [Silver Sable] DISABLING LICENSE FOR [shield]
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] stopped
[2015-06-12 05:59:17,088][INFO ][node                     ] [Silver Sable] closing ...
[2015-06-12 05:59:17,104][INFO ][node                     ] [Silver Sable] closed
[2015-06-12 05:59:20,479][INFO ][node                     ] [Lionheart] version[1.4.5], pid[28662], build[2aaf797/2015-04-27T08:06:06Z]
[2015-06-12 05:59:20,480][INFO ][node                     ] [Lionheart] initializing ...
[2015-06-12 05:59:20,586][INFO ][plugins                  ] [Lionheart] loaded [license, shield], sites []
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.ShieldServerTransportService] as transport service, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][transport                ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyTransport] as transport, overridden by [shield]
[2015-06-12 05:59:21,301][INFO ][http                     ] [Lionheart] Using [org.elasticsearch.shield.transport.netty.ShieldNettyHttpServerTransport] as http transport, overridden by [shield]
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] initialized
[2015-06-12 05:59:27,166][INFO ][node                     ] [Lionheart] starting ...
[2015-06-12 05:59:28,148][INFO ][shield.transport         ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/10.1.0.25:9300]}
[2015-06-12 05:59:28,209][INFO ][discovery                ] [Lionheart] logstash/uuDCpM6hTKKvLNd2oFGYpA
[2015-06-12 05:59:32,032][INFO ][cluster.service          ] [Lionheart] new_master [Lionheart][uuDCpM6hTKKvLNd2oFGYpA][0ba2a1c6e1de][inet[/10.1.0.25:9300]], reason: zen-disco-join (elected_as_master)
[2015-06-12 05:59:32,119][INFO ][http                     ] [Lionheart] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/10.1.0.25:9200]}
[2015-06-12 05:59:32,119][INFO ][node                     ] [Lionheart] started
[2015-06-12 05:59:33,007][INFO ][shield.license           ] [Lionheart] enabling license for [shield]
[2015-06-12 05:59:33,013][INFO ][license.plugin.core      ] [Lionheart] license for [shield] - valid
[2015-06-12 05:59:33,028][ERROR][shield.license           ] [Lionheart] 
#
# Shield license will expire on [Saturday, July 11, 2015]. Cluster health, cluster stats and indices stats operations are
# blocked on Shield license expiration. All data operations (read and write) continue to work. If you
# have a new license, please update it. Otherwise, please reach out to your support contact.
#
[2015-06-12 05:59:33,162][INFO ][gateway                  ] [Lionheart] recovered [2] indices into cluster_state

Logstash 日志:(这部分重复了很多次)

NotImplementedError: block device detection unsupported or native support failed to load
       blockdev? at org/jruby/RubyFileTest.java:67
         device? at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/helper.rb:67
  _sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:230
   sincedb_write at /opt/logstash/vendor/bundle/jruby/1.9/gems/filewatch-0.6.2/lib/filewatch/tail.rb:203
        teardown at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.10/lib/logstash/inputs/file.rb:151
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     synchronize at org/jruby/ext/thread/Mutex.java:149
     inputworker at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:203
     start_input at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/pipeline.rb:171

【问题讨论】:

  • 任何日志/日志存储跟踪?
  • @JettroCoenradie,请参阅我编辑的帖子。谢谢

标签: elasticsearch logstash elasticsearch-plugin logstash-configuration elasticsearch-shield


【解决方案1】:

我认为这是一个与屏蔽无关的问题。检查这个问题: https://github.com/elastic/logstash/issues/3127

就像帖子提到的那样,执行以下操作对我有用:

ln -s /lib/x86_64-linux-gnu/libcrypt.so.1 /usr/lib/x86_64-linux-gnu/libcrypt.so

【讨论】:

  • 哇,感谢您为我解决了这个问题。这是一个奇怪的错误。我不知道为什么在安装防护罩之前相同的设置对我有用。这就是为什么我认为它是问题的根源。谢谢!!
猜你喜欢
  • 1970-01-01
  • 2017-05-18
  • 1970-01-01
  • 1970-01-01
  • 2015-10-29
  • 2018-01-10
  • 1970-01-01
  • 2014-06-10
  • 2016-03-09
相关资源
最近更新 更多