【问题标题】:Kafka failed to start using confluent commandKafka 无法使用 confluent 命令启动
【发布时间】:2019-01-23 08:48:03
【问题描述】:

我在尝试通过以下方式启动 ZooKeeper、Kafka 和模式注册表时遇到了一些困难:

$confluent start schema-registry

我正在运行一个由 3 个节点组成的集群,由于某种原因,当我运行上述命令时,Kafka 无法启动:

#root: /confluent/confluent-5.0.0/bin/confluent start schema-registry
This CLI is intended for development only, not for production
https://docs.confluent.io/current/cli/index.html

Using CONFLUENT_CURRENT: /tmp/confluent.h266AuQQ
Starting zookeeper
zookeeper is [UP]
Starting kafka
/Kafka failed to start
kafka is [DOWN]
Cannot start Schema Registry, Kafka Server is not running. Check your deployment

zookeeper.properties:

root: /confluent/confluent-5.0.0# cat ./etc/kafka/zookeeper.properties 

tickTime=2000
dataDir=/infrastructure/zookeeper/data/
clientPort=2181
initLimit=5
syncLimit=2
server.1=Hostname.DOMAIN.com:2888:3888
server.2=Hostname.DOMAIN.com:2888:3888
server.3=Hostname.DOMAIN.com:2888:3888

卡夫卡日志:

root: /confluent/confluent-5.0.0# ./bin/confluent log kafka

[2018-08-16 14:09:02,830] INFO Socket connection established to localhost/127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2018-08-16 14:09:02,830] INFO Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2018-08-16 14:09:03,422] INFO [ZooKeeperClient] Closing. (kafka.zookeeper.ZooKeeperClient)
[2018-08-16 14:09:04,201] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2018-08-16 14:09:04,201] INFO Socket connection established to localhost/127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2018-08-16 14:09:04,304] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2018-08-16 14:09:04,306] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2018-08-16 14:09:04,307] INFO [ZooKeeperClient] Closed. (kafka.zookeeper.ZooKeeperClient)
[2018-08-16 14:09:04,310] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
        at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply$mcV$sp(ZooKeeperClient.scala:230)
        at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:226)
        at kafka.zookeeper.ZooKeeperClient$$anonfun$kafka$zookeeper$ZooKeeperClient$$waitUntilConnected$1.apply(ZooKeeperClient.scala:226)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:251)
        at kafka.zookeeper.ZooKeeperClient.kafka$zookeeper$ZooKeeperClient$$waitUntilConnected(ZooKeeperClient.scala:226)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:95)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1580)
        at kafka.server.KafkaServer.kafka$server$KafkaServer$$createZkClient$1(KafkaServer.scala:348)
        at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:372)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:202)
        at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:117)
        at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
[2018-08-16 14:09:04,312] INFO shutting down (kafka.server.KafkaServer)
[2018-08-16 14:09:04,313] WARN  (kafka.utils.CoreUtils$)
java.lang.NullPointerException
        at kafka.server.KafkaServer$$anonfun$shutdown$5.apply$mcV$sp(KafkaServer.scala:579)
        at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:86)
        at kafka.server.KafkaServer.shutdown(KafkaServer.scala:579)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:329)
        at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:117)
        at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
[2018-08-16 14:09:04,315] INFO shut down completed (kafka.server.KafkaServer)
[2018-08-16 14:09:04,316] INFO shutting down (kafka.server.KafkaServer)

不知道这行告诉我什么:

ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING

日志中的 Zookeeper 似乎也有问题:

java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:558)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957)
[2018-08-16 14:32:26,540] INFO Resolved hostname: HOSTNAME1.DNS.com to address: HOSTNAME1.DNS.com/10.***.***.*** (org.apache.zookeeper.server.quorum.QuorumPeer)
[2018-08-16 14:32:26,540] WARN Cannot open channel to 3 at election address HOSTNAME1.DNS.com/10.***.***.***:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:558)
        at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
        at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
        at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957)
[2018-08-16 14:32:26,541] INFO Resolved hostname: HOSTNAME2.DNS.com to address: HOSTNAME2.DNS.com/10.***.***.*** (org.apache.zookeeper.server.quorum.QuorumPeer)
[2018-08-16 14:32:26,541] INFO Notification time out: 60000 (org.apache.zookeeper.server.quorum.FastLeaderElection)

知道这里有什么问题吗?

感谢您的帮助。

【问题讨论】:

  • 架构注册表无法启动,因为 Kafka 代理已关闭。因此,您需要找出 Kafka 代理没有启动和运行的原因。如果你也提供 server.properties 文件会很有帮助。
  • 感谢您的回复,如果我错了,请纠正我,但是“confluent start”命令应该会启动所有 kafka 代理,这就是他们现在全部关闭的原因。这个问题是命令无法启动它们,我不知道为什么。这就是发布这些日志的原因。
  • 如果您在 server.properties 文件中提供正确的属性和配置,那么 confluent start 将启动所有 kafka 代理
  • 为了让事情更清楚:模式注册表要求 Kafka 代理启动并运行,而 kafka 代理反过来要求 Zookeeper 实例启动并运行。首先单独运行每个服务;例如confluent start zookeeper 并等待几分钟。阅读日志并确保一切正常,然后对每个代理执行相同的操作。您是否尝试过仅使用一个 Kafka 代理运行环境?
  • 上次检查时,confluent命令只能运行单个broker+Zookeeper(本地,不连接远程服务器)

标签: apache-kafka apache-zookeeper confluent-platform


【解决方案1】:

您在启动 zookeper 时遇到问题。

zookeeper 属性中不需要以下行,因为 kafka 属性文件包含连接所有节点的 zokkeeper 地址。

server.1 主机名.DOMAIN.com:2888:3888

【讨论】:

    【解决方案2】:

    我也有类似的问题。当我开始融合时得到:

    Starting zookeeper
    zookeeper is [UP]
    Starting kafka
    Kafka failed to start
    kafka is [DOWN]
    Cannot start Schema Registry, Kafka Server is not running. Check your deployment
    

    问题是我本地跑在Kafka之前,9092端口还在监听一个进程。所以首先我检查端口是否在监听:

    sudo lsof -i :9092 
    

    输出:

    命令 PID 用户 FD 类型设备大小/关闭节点名称 java
    17150 savi01 97u IPv6 134739 0t0 TCP *:9092 (LISTEN) java
    17150 savi01 113u IPv6 134741 0t0 TCP localhost:42368->savi01:9092 (CLOSE_WAIT)

    然后我终止进程:

    sudo kill -9 17150
    

    现在我再次运行 confluent,一切正常。希望对未来的顾问有所帮助

    【讨论】:

      猜你喜欢
      • 2018-04-12
      • 1970-01-01
      • 1970-01-01
      • 2021-09-22
      • 1970-01-01
      • 2019-06-23
      • 2018-08-02
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多