从外部访问 kafka docker

Question

我已经设置了一个 docker-compose.yaml

version: '3'

services:
  zookeeper:
    image: zookeeper:latest
    restart: unless-stopped
    ports:
        - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:
    image: ligato/kafka-arm64
    ports:
      - "9093:9093"
    expose:
      - "9093"
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS: INTERNAL://kafka:9092,OUTSIDE://localhost:9093
      KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:9092,OUTSIDE://localhost:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INTERNAL
      KAFKA_BROKER_ID: 1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      - zookeeper

由于我部署在JETSON NX上，所以选择镜像“ligato/kafka-arm64”，容器信息为

CONTAINER ID   IMAGE                COMMAND                  CREATED             STATUS         PORTS                                                  NAMES
c8d890135148   ligato/kafka-arm64   "supervisord -n"         5 seconds ago       Up 2 seconds   2181/tcp, 9092/tcp, 0.0.0.0:9093->9093/tcp             kafka_docker_kafka_1
edba89568398   zookeeper:latest     "/docker-entrypoint.…"   About an hour ago   Up 5 seconds   2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp   kafka_docker_zookeeper_1

然后在容器内成功创建主题，并运行生产者

docker exec -it c8d890135148 /bin/bash

./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic01

./bin/kafka-console-producer.sh --broker-list kafka:9092 --topic topic01

并尝试输入一些字母，但在容器外什么都没有……只是卡住了任何东西。

./bin/kafka-console-consumer.sh --bootstrap-server localhost:9093 --topic topic01 --from-beginning

有人在这个问题上有经验吗？请与我分享一些cmets！

在上面之后，我尝试在容器外使用最新的kafka版本，kafka_2.13,2.8,0，并使用相同的命令，但得到

[2021-05-30 03:52:48,240] WARN [Consumer clientId=consumer-console-consumer-6086-1, groupId=console-consumer-6086] Bootstrap broker 127.0.0.1:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-05-30 03:52:48,347] WARN [Consumer clientId=consumer-console-consumer-6086-1, groupId=console-consumer-6086] Bootstrap broker 127.0.0.1:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-05-30 03:52:48,453] WARN [Consumer clientId=consumer-console-consumer-6086-1, groupId=console-consumer-6086] Bootstrap broker 127.0.0.1:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-05-30 03:52:48,558] WARN [Consumer clientId=consumer-console-consumer-6086-1, groupId=console-consumer-6086] Bootstrap broker 127.0.0.1:9093 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

消息不断重复。

对于这两个问题有任何 cmet 吗？提前感谢！

Answer 1

我不确定容器是否支持这些属性...而且它自 2019 年初以来一直没有更新，因此如果您想要最新的 Kafka 版本，请查看 Confluent 或 Bitnami Kafka 图像。

在任何情况下，侦听器都应该在外部绑定以允许外部流量

KAFKA_LISTENERS: INTERNAL://0.0.0.0:9092,OUTSIDE://0.0.0.0:9093

另外：它本身也包含 Zookeeper，所以你不应该有一个单独的 Zookeeper 容器，你应该避免挂载 Docker 套接字