如何配置 logstash 和 filebeat SSL 通信

Question

问题：

谁能帮我弄清楚为什么我不能让 filebeats 通过 TLS/SSL 与 logstash 通信？

错误：

我可以在禁用 TLS/SSL 的情况下让 filebeat 和 logstash 与每个人交谈，但是当我启用它并使用下面的设置/配置时，我收到以下错误（在 logstash.log 中观察到）：

{:timestamp=>"2016-10-28T17:21:44.445000+0100", :message=>"Pipeline aborted due to error",
 :exception=>java.lang.NullPointerException, :backtrace=>["org.logstash.netty.PrivateKeyCo
nverter.generatePkcs8(org/logstash/netty/PrivateKeyConverter.java:43)", "org.logstash.nett
y.PrivateKeyConverter.convert(org/logstash/netty/PrivateKeyConverter.java:39)", "java.lang
.reflect.Method.invoke(java/lang/reflect/Method.java:498)", "RUBY.create_server(/usr/share
/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.0.beta4-java/lib/logstash/
inputs/beats.rb:139)", "RUBY.register(/usr/share/logstash/vendor/bundle/jruby/1.9/gems/log
stash-input-beats-3.1.0.beta4-java/lib/logstash/inputs/beats.rb:132)", "RUBY.start_inputs(
/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:311)", "org.jruby.RubyArray.eac
h(org/jruby/RubyArray.java:1613)", "RUBY.start_inputs(/usr/share/logstash/logstash-core/li
b/logstash/pipeline.rb:310)", "RUBY.start_workers(/usr/share/logstash/logstash-core/lib/lo
gstash/pipeline.rb:187)", "RUBY.run(/usr/share/logstash/logstash-core/lib/logstash/pipelin
e.rb:145)", "RUBY.start_pipeline(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:2
40)", "java.lang.Thread.run(java/lang/Thread.java:745)"], :level=>:error}
{:timestamp=>"2016-10-28T17:21:47.452000+0100", :message=>"stopping pipeline", :id=>"main"
, :level=>:warn}
{:timestamp=>"2016-10-28T17:21:47.456000+0100", :message=>"An unexpected error occurred!",
:error=>#<NoMethodError: undefined method `stop' for nil:NilClass>, :backtrace=>["/us
r/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-beats-3.1.0.beta4-java/lib/lo
gstash/inputs/beats.rb:173:in `stop'", "/usr/share/logstash/logstash-core/lib/logstash/inp
uts/base.rb:88:in `do_stop'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logst
ash/logstash-core/lib/logstash/pipeline.rb:366:in `shutdown'", "/usr/share/logstash/logsta
sh-core/lib/logstash/agent.rb:252:in `stop_pipeline'", "/usr/share/logstash/logstash-core/
lib/logstash/agent.rb:261:in `shutdown_pipelines'", "org/jruby/RubyHash.java:1342:in `each
'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:261:in `shutdown_pipelines'",
 "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:123:in `shutdown'", "/usr/share/
logstash/logstash-core/lib/logstash/runner.rb:237:in `execute'", "/usr/share/logstash/vend
or/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/usr/share/logsta
sh/logstash-core/lib/logstash/runner.rb:157:in `run'", "/usr/share/logstash/vendor/bundle/
jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in `run'", "/usr/share/logstash/lib/bo
otstrap/environment.rb:66:in `(root)'"], :level=>:fatal}

设置：

服务器

• 2 个服务器。

  $> uname -a Linux elkserver 3.10.0-327.36.2.el7.x86_64 #1 SMP Mon Oct 10 23:08:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $> cat /etc/*-release CentOS Linux 版本 7.2.1511（核心）

• SELinux 是许可的 (soz)。

• 防火墙属于。 (mazza soz)。
• 一台服务器运行elasticsearch和logstash；一个运行filebeat。

弹性搜索

$> /usr/share/elasticsearch/bin/elasticsearch -version 版本：2.4.1，构建：c67dc32/2016-09-27T18:57:55Z，JVM：1.8.0_111

Logstash

$> /usr/share/logstash/bin/logstash -V logstash 5.0.0-alpha5

菲尔贝特

$> /usr/share/filebeat/bin/filebeat -version filebeat 版本 5.0.0 (amd64)，libbeat 5.0.0

配置：

• Lo​​gstash

input {
  beats {
    port => 5044
  ssl => true
  ssl_certificate => "/etc/pki/tls/certs/filebeat-forwarder.crt"
  ssl_key => "/etc/pki/tls/private/filebeat-forwarder.key"
  }
}
output {
  elasticsearch {
  hosts => "localhost:9200"
  manage_template => false
  index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
  document_type => "%{[@metadata][type]}"
  }
}

• Filebeat.yml

output:
 logstash:
   enabled: true
   hosts:
     - "<my ip address>:5044"
   timeout: 15
   tls:
     certificate_authorities:
     - /etc/pki/tls/certs/filebeat-forwarder.crt
filebeat:
 prospectors:
   -
     paths:
       - /var/log/syslog
       - /var/log/auth.log
     document_type: syslog
   -
     paths:
       - /var/log/nginx/access.log
     document_type: nginx-access

• 文件：openssl_extras.cnf：

  [req]    
  distinguished_name = req_distinguished_name    
  x509_extensions = v3_req    
  prompt = no    
  [req_distinguished_name]    
  C = TG    
  ST = Togo    
  L =  Lome    
  O = Private company    
  CN = *    
  [v3_req]    
  subjectKeyIdentifier = hash    
  authorityKeyIdentifier = keyid,issuer    
  basicConstraints = CA:TRUE    
  subjectAltName = @alt_names        
  [alt_names]    
  DNS.1 = *    
  DNS.2 = *.*    
  DNS.3 = *.*.*    
  DNS.4 = *.*.*.*    
  DNS.5 = *.*.*.*.*    
  DNS.6 = *.*.*.*.*.*    
  DNS.7 = *.*.*.*.*.*.*    
  IP.1 = <my ip address>
  

用于创建证书的命令：

$> openssl req -subj '/CN=elkserver.system.local/' -config /etc/pki/tls/openssl_extras.cnf \ -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout /etc/pki/tls/private/filebeat-forwarder.key \ -out /etc/pki/tls/certs/filebeat-forwarder.crt

Answer 1

在 Filebeat 5.0 中，tls 配置设置更改为 ssl，以与 Logstash 和 Elasticsearch 中使用的配置设置保持一致。尝试更新您的 Filebeat 配置。

参考资料：

• Securing Communication With Logstash by Using SSL
• Breaking Changes in 5.0