kafka_exporter 不适用于计算实例答案

【问题标题】：kafka_exporter does not work on compute instancekafka_exporter 不适用于计算实例
【发布时间】：2020-03-26 14:35:13
【问题描述】：

我尝试在运行 Kafka 的计算实例 (CentOS 7) 上按照 github 上的步骤设置 kafka_exporter。

当我运行以下命令时，我得到“错误初始化 Kafka 客户端”：

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --kafka.version="2.11.0" --tls.enabled  --log.enable-sarama

非常感谢在运行 Kafka 的计算实例上运行 kafka_exporter 的任何建议。

在此先感谢您，

控制台输出：

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --kafka.version="2.11.0" --tls.enabled  --log.enable-sarama

INFO[0000] Starting kafka_exporter (version=1.2.0, branch=HEAD, revision=830660212e6c109e69dcb1cb58f5159fe3b38903)  source="kafka_exporter.go:474"
INFO[0000] Build context (go=go1.10.3, user=root@981cde178ac4, date=20180707-14:34:48)  source="kafka_exporter.go:475"
[sarama] 2020/03/27 05:47:00 Initializing new client
[sarama] 2020/03/27 05:47:00 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:00 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:00 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:00 client/metadata retrying after 250ms... (3 attempts remaining)
[sarama] 2020/03/27 05:47:00 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:00 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:00 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:00 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:00 client/metadata retrying after 250ms... (2 attempts remaining)
[sarama] 2020/03/27 05:47:01 client/metadata fetching metadata for all topics from broker kafka1:9092

[sarama] 2020/03/27 05:47:01 Failed to connect to broker kafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:01 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:01 client/metadata retrying after 250ms... (1 attempts remaining)

[sarama] 2020/03/27 05:47:01 client/metadata fetching metadata for all topics from broker kafka1:9092
[sarama] 2020/03/27 05:47:01 Failed to connect to broker kkafka1:9092: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata got error from broker while fetching metadata: x509: certificate signed by unknown authority
[sarama] 2020/03/27 05:47:01 client/metadata no available broker to send metadata request to
[sarama] 2020/03/27 05:47:01 client/brokers resurrecting 1 dead seed brokers
[sarama] 2020/03/27 05:47:01 Closing Client
ERRO[0000] Error Init Kafka Client                       source="kafka_exporter.go:210"
panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

goroutine 1 [running]:
main.NewExporter(0xc42017a6f0, 0x1, 0x1, 0x100, 0x8f22f8, 0x0, 0x8f22f8, 0x0, 0x1, 0x8f22f8, ...)
  /go/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:211 +0x847
main.main()
  /go/src/github.com/danielqsj/kafka_exporter/kafka_exporter.go:481 +0x215b

我可以 ping 到 kafka1 实例

【问题讨论】：

Ping 很好，但是你能telnet 吗？ “你的集群是否可达？”似乎暗示你不能。
为什么要在另一个节点上运行导出器，然后是 kafka 节点？
感谢您的意见@mazaneicha 和@rickyA！我能够访问我的 Kafka 实例，实际上我正在我的 Kafka 计算实例本身上运行 kafka_exporter
--kafka.server=localhost:9092 工作吗？
@RickyA 结果是一样的

标签： go apache-kafka jmx-exporter

【解决方案1】：

在将 Java 密钥库和信任库 (JKS) 证书转换为 PEM 并将它们用作 kafka_exporter 的输入（标志）后解决了该问题。（如https://medium.com/processone/using-tls-authentication-for-your-go-kafka-client-3c5841f2a625中提到的）

/opt/kafkaexporter/kafka_exporter --kafka.server=kafka1:9092 --tls.enabled --tls.ca-file="/home/kafka_user/serverpub_truststore_go.cer.pem"  --tls.key-file="/home/kafka_user/serverpub_keystore_go.cer.key.pem"  --tls.cert-file="/home/kafka_user/serverpub_keystore_go.cer.pem" --kafka.version="2.11.0" --log.enable-sarama

输出：

INFO[0000] Starting kafka_exporter (version=1.2.0, branch=HEAD, revision=8306602b58f5159fe3b38903)  source="kafka_exporter.go:474"
INFO[0000] Build context (go=go1.10.3, user=root@981ac4, date=20180707-14:34:48)  source="kafka_exporter.go:475"
[sarama] 2020/03/28 01:20:38 Initializing new client
[sarama] 2020/03/28 01:20:38 client/metadata fetching metadata for all topics from broker kafka2:9092
[sarama] 2020/03/28 01:20:38 Connected to broker at kafka2:9092 (unregistered)
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #2 at kafka2:9092
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #1 at kafka1:9092
[sarama] 2020/03/28 01:20:38 client/brokers registered new broker #3 at kafka3:9092
[sarama] 2020/03/28 01:20:38 Successfully initialized new client
INFO[0000] Done Init Clients                             source="kafka_exporter.go:213"
INFO[0000] Listening on :9308                            source="kafka_exporter.go:499"

【讨论】：