我找不到此警告（logstash）的原因：“beats 协议版本无效”

Question

尊敬的 Stackoverflow 社区，

我想知道我为什么要解决这个 logstash 的问题：

2021-01-20T01:02:33,444][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2021-01-20T01:02:41,603][INFO ][org.logstash.beats.BeatsHandler][synlite_suricata][input_beats] [local: 10.0.100.12:5044, remote: 10.0.100.1:39666] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
[2021-01-20T01:02:41,614][WARN ][io.netty.channel.DefaultChannelPipeline][synlite_suricata][input_beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.
io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:353) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
    at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 22
    at org.logstash.beats.Protocol.version(Protocol.java:22) ~[logstash-input-beats-6.0.9.jar:?]
    at org.logstash.beats.BeatsParser.decode(BeatsParser.java:62) ~[logstash-input-beats-6.0.9.jar:?]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
    ... 8 more
[2021-01-20T01:02:41,637][INFO ][org.logstash.beats.BeatsHandler][synlite_suricata][input_beats] [local: 10.0.100.12:5044, remote: 10.0.100.1:39666] Handling exception: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 3
[2021-01-20T01:02:41,639][WARN ][io.netty.channel.DefaultChannelPipeline][synlite_suricata][input_beats] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception.

我找到了类似的主题，但我找不到明确的解释...... Logstash beats input "invalid version of beats protocol" Mock an ELK Beat output to Logstash with Postman

我可以分享我的配置：

pipelines.yml：

- pipeline.id: synlite_suricata
  path.config: "/etc/logstash/synlite_suricata/conf.d/*.conf"

- pipeline.id: fallback
  path.config: "/etc/logstash/fallback/conf.d/*.conf"

synlite_suricata 输入：

input {
  # Beats
  beats {
    id => "input_beats"
    host => "${SYNLITE_SURICATA_BEATS_HOST}"
    port => "${SYNLITE_SURICATA_BEATS_PORT}"
    client_inactivity_timeout => 180
    ssl => false
    ssl_certificate_authorities => "${SYNLITE_SURICATA_CACERT}"
    ssl_certificate => "${SYNLITE_SURICATA_BEATS_CERT}"
    ssl_key => "${SYNLITE_SURICATA_BEATS_KEY}"
    ssl_verify_mode => "peer"
    ssl_peer_metadata => true
    tls_min_version => 1.2
    cipher_suites => [ "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384" ]

  }
}

以及 /etc/systemd/system/logstash.service.d/synlite_suricata.conf 中的 systemd 变量：

[Service]
# Synesis Lite for Suricata global configuration
Environment="SYNLITE_SURICATA_DICT_PATH=/etc/logstash/synlite_suricata/dictionaries"
Environment="SYNLITE_SURICATA_TEMPLATE_PATH=/etc/logstash/synlite_suricata/templates"
Environment="SYNLITE_SURICATA_GEOIP_DB_PATH=/etc/logstash/synlite_suricata/geoipdbs"
Environment="SYNLITE_SURICATA_GEOIP_CACHE_SIZE=8192"
Environment="SYNLITE_SURICATA_GEOIP_LOOKUP=true"
Environment="SYNLITE_SURICATA_ASN_LOOKUP=true"
Environment="SYNLITE_SURICATA_CLEANUP_SIGS=false"

# Name resolution option
Environment="SYNLITE_SURICATA_RESOLVE_IP2HOST=false"
Environment="SYNLITE_SURICATA_NAMESERVER=127.0.0.1"
Environment="SYNLITE_SURICATA_DNS_HIT_CACHE_SIZE=25000"
Environment="SYNLITE_SURICATA_DNS_HIT_CACHE_TTL=900"
Environment="SYNLITE_SURICATA_DNS_FAILED_CACHE_SIZE=75000"
Environment="SYNLITE_SURICATA_DNS_FAILED_CACHE_TTL=3600"

# Elasticsearch connection settings
Environment="SYNLITE_SURICATA_ES_HOST=10.0.100.11"
Environment="SYNLITE_SURICATA_ES_USER=logstash"
Environment="SYNLITE_SURICATA_ES_PASSWD=password"

# Beats input
Environment="SYNLITE_SURICATA_BEATS_HOST=10.0.100.12"
Environment="SYNLITE_SURICATA_BEATS_PORT=5044"

# Certs config
Environment="SYNLITE_SURICATA_CACERT=/etc/logstash/tls/root-ca.crt"
Environment="SYNLITE_SURICATA_BEATS_CERT=/etc/logstash/tls/logstash-input-server.crt"
Environment="SYNLITE_SURICATA_BEATS_KEY=/etc/logstash/tls/logstash-input-server.pk8"
Environment="SYNLITE_SURICATA_ES_KEYSTORE=/etc/logstash/tls/logstash-elasticsearch-output-client.p12"
Environment="SYNLITE_SURICATA_ES_KEYSTORE_PASSWORD=password"

我不知道如何找到和“beat version”相关的信息... Logstash-oss 7.8.0

谢谢，期待您的关注

Answer 1

这通常是由连接到节拍输入的设备不使用节拍（伐木工人）协议引起的。输入基本上是说字节流中某个位置的字节具有它无法理解的值。这可能有很多原因。

要确定原因，您需要找到有问题的程序

远程：1​​0.0.100.1:39666

打开并检查它发送给您的内容。 tcpdump 可能会有所帮助。可能的原因包括（但当然不限于）：

• 一个节拍被配置为使用 SSL，但输入不是

• beat 输入需要 SSL，但 filebeat 没有 已配置

• 如果您在公司网络上，则公司安全人员可能是 对每个内部 IP 地址运行端口扫描并尝试 连接到每个 TCP 端口（可能是数亿个端口） 每周）并检查它是否响应 HTTP 或其他一些 如果没有得到适当的保护，可能会成为安全漏洞的协议。

• 如果该地址在互联网上（不太可能，因为它是私人 地址，但您可能混淆了消息）然后我可以 保证 Shodan 和许多其他人正在对您进行端口扫描。

• 另一种可能性是在 filebeat.yml 中注释掉 elasticsearch hosts: entry（但不是 elasticsearch: entry），然后取消注释 logstash hosts: entry（但不是 logstash: entry）。这会导致节拍尝试与节拍输入通信 HTTP，这将引发此异常。

Answer 2

非常有帮助，谢谢！ 我将使用 tcpdump 进行探索。我在使用 Wireguard Tunel 的专用网络下。