【发布时间】:2019-07-23 18:48:24
【问题描述】:
我正在尝试部署一个具有 3 个 zookeeper 和 3 个 kafka 节点的 docker kafka 集群。 kafka 节点不断打印以下错误:
[main-SendThread(zookeeper-1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper-1/10.0.0.5:2181. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(zookeeper-1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to zookeeper-1/10.0.0.5:2181, initiating session
[main-SendThread(zookeeper-1:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
当 zookeeper 节点持续打印时:
WARN Cannot open channel to 1 at election address
zookeeper-1/10.0.0.5:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:558)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957)
[2019-03-01 17:24:30,930] INFO Resolved hostname: zookeeper-1 to address: zookeeper-1/10.0.0.5 (org.apache.zookeeper.server.quorum.QuorumPeer)
[2019-03-01 17:24:30,931] WARN Cannot open channel to 2 at election address zookeeper-2/10.0.0.7:3888 (org.apache.zookeeper.server.quorum.QuorumCnxManager)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:558)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectAll(QuorumCnxManager.java:610)
at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:838)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:957)
看起来zookeeper节点无法相互通信,但我检查了几次docker-compose yaml,我找不到奇怪的。有人可以帮帮我吗?
docker-compose.yaml:
version: '3.3'
networks:
kafka_example:
driver: overlay
services:
zookeeper-1:
image: confluentinc/cp-zookeeper:5.0.1
volumes:
- '/volumedockerkafka/zookeeper-1/zookeeper_data:/var/lib/zookeeper:rw' # mount point
- '/volumedockerkafka/zookeeper-1/zookeeper_etc:/etc/zookeeper:rw' # mount point
environment:
ZOOKEEPER_SERVER_ID: 1
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_SERVERS: zookeeper-1:2888:3888;zookeeper-2:2888:3888;zookeeper-3:2888:3888
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
networks:
- kafka_example
zookeeper-2:
image: confluentinc/cp-zookeeper:5.0.1
volumes:
- '/volumedockerkafka/zookeeper-2/zookeeper_data:/var/lib/zookeeper:rw' # mount point
- '/volumedockerkafka/zookeeper-2/zookeeper_etc:/etc/zookeeper:rw' # mount point
environment:
ZOOKEEPER_SERVER_ID: 2
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_SERVERS: zookeeper-1:2888:3888;zookeeper-2:2888:3888;zookeeper-3:2888:3888
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
networks:
- kafka_example
zookeeper-3:
image: confluentinc/cp-zookeeper:5.0.1
volumes:
- '/volumedockerkafka/zookeeper-3/zookeeper_data:/var/lib/zookeeper:rw' # mount point
- '/volumedockerkafka/zookeeper-3/zookeeper_etc:/etc/zookeeper:rw' # mount point
environment:
ZOOKEEPER_SERVER_ID: 3
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_SERVERS: zookeeper-1:2888:3888;zookeeper-2:2888:3888;zookeeper-3:2888:3888
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_INIT_LIMIT: 5
ZOOKEEPER_SYNC_LIMIT: 2
networks:
- kafka_example
message_queue_kafka-1:
image: confluentinc/cp-kafka:5.0.1
volumes:
- '/volumedockerkafka/kafka-1/kafka_data:/var/lib/kafka/data:rw' # mount point
- '/volumedockerkafka/kafka-1/kafka_etc:/etc/kafka:rw'
environment:
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181'
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://message_queue_kafka-1:9092
KAFKA_BROKER_ID: 1
KAFKA_SESSION_TIMEOUT_MS: 60000
KAFKA_HEARTBEAT_INTERVAL_MS: 20000
KAFKA_REBALANCE_TIMEOUT_MS: 300000
depends_on:
- zookeeper-1
- zookeeper-2
- zookeeper-3
networks:
- kafka_example
message_queue_kafka-2:
image: confluentinc/cp-kafka:5.0.1
volumes:
- '/volumedockerkafka/kafka-2/kafka_data:/var/lib/kafka/data:rw' # mount point
- '/volumedockerkafka/kafka-2/kafka_etc:/etc/kafka:rw'
environment:
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181'
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://message_queue_kafka-2:9092
KAFKA_BROKER_ID: 2
KAFKA_SESSION_TIMEOUT_MS: 60000
KAFKA_HEARTBEAT_INTERVAL_MS: 20000
KAFKA_REBALANCE_TIMEOUT_MS: 300000
depends_on:
- zookeeper-1
- zookeeper-2
- zookeeper-3
networks:
- kafka_example
message_queue_kafka-3:
image: confluentinc/cp-kafka:5.0.1
volumes:
- '/volumedockerkafka/kafka-3/kafka_data:/var/lib/kafka/data:rw' # mount point
- '/volumedockerkafka/kafka-3/kafka_etc:/etc/kafka:rw'
environment:
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181'
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://message_queue_kafka-3:9092
KAFKA_BROKER_ID: 3
KAFKA_SESSION_TIMEOUT_MS: 60000
KAFKA_HEARTBEAT_INTERVAL_MS: 20000
KAFKA_REBALANCE_TIMEOUT_MS: 300000
depends_on:
- zookeeper-1
- zookeeper-2
- zookeeper-3
networks:
- kafka_example
【问题讨论】:
标签: docker apache-kafka apache-zookeeper confluent-platform