启动 Logstash，将其连接到 kafka 生产者答案

【问题标题】：Starting Logstash, connecting it to kafka producer启动 Logstash，将其连接到 kafka 生产者
【发布时间】：2021-05-07 10:10:43
【问题描述】：

我已将 Elasticsearch 和 Logstash 安装为主要组件。

启动了每 4 秒创建一次记录的 Kafka 生产者
systemctl status logstash，表示 logstash 正在运行
从logstash的bin文件夹中，当我运行以下命令监听一个主题时，它给出了一个错误bash：“localhost:9092”：权限被拒绝

/usr/share/logstash/bin$ logstash -e ‘input { kafka { bootstrap_servers => “localhost:9092” topics => “Customer_demo” } } filter { json { source => “message”} } output { elasticsearch { hosts => [“localhost:9200”] index => “customer-demo” } stdout {} }

【问题讨论】：

您可以在运行 Logstash 时添加 --debug 并分享您在日志中获得的输出吗？
用tail -f /var/log/logstash/logstash-plain.log 检查错误日志
如果你在根文件夹中连接（jar）文件，logstash无法读取它...创建/home/me这样的目录
转发完整日志
普通日志如下-

标签： elasticsearch apache-kafka logstash

【解决方案1】：

生产者现在没有运行，但是，普通日志是- 我会通过让生产者起来拍摄快照。

[2021-02-03T15:57:25,387][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null) is unavailable or invalid, will attempt rediscovery
[2021-02-03T15:57:25,387][INFO ][org.apache.kafka.clients.FetchSessionHandler][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Error sending fetch request (sessionId=786233676, epoch=9143) to node 0: {}.
org.apache.kafka.common.errors.DisconnectException: null
[2021-02-03T15:57:25,827][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null)
[2021-02-03T16:38:01,572][INFO ][org.apache.kafka.clients.FetchSessionHandler][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Error sending fetch request (sessionId=1455350722, epoch=2665) to node 0: {}.
org.apache.kafka.common.errors.DisconnectException: null
[2021-02-03T16:38:01,585][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null) is unavailable or invalid, will attempt rediscovery
[2021-02-03T16:38:03,556][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null)
[2021-02-03T16:38:03,560][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null) is unavailable or invalid, will attempt rediscovery
[2021-02-03T16:38:03,664][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][5f6aa7eb9c5d8e0614720d7b335f8771464d38e2be323dcdb318ca41dfd3e53a] [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator tej-Vostro-3590:9092 (id: 2147483647 rack: null)

【讨论】：