Kafka Connect - 无法查找分区元数据

Question

在关闭了自动创建主题的 Kafka 集群上运行 Kafka Connect。据我所见，在另一个正在运行的集群上，我们创建了连接配置、连接偏移和连接状态等主题。因此，这些主题在具有相同复制因子和分区的新集群上重新创建。

虽然我有从连接实例到代理的 netcat 并且那里没有连接问题，但我们在启动服务时遇到错误

Could not look up partition metadata for offset backing store topic in allotted period. This could indicate a connectivity issue, unavailable topic partitions, or if this is your first use of the topic it may have taken too long to create

我可以从 connect utils 类看到它在下面抛出的位置。但是已经检查过，这些主题现在已经创建了所以有点丢失了

        producer = createProducer();
        consumer = createConsumer();

        List<TopicPartition> partitions = new ArrayList<>();

        // We expect that the topics will have been created either manually by the user or automatically by the herder
        List<PartitionInfo> partitionInfos = null;
        long started = time.milliseconds();
        while (partitionInfos == null && time.milliseconds() - started < CREATE_TOPIC_TIMEOUT_MS) {
            partitionInfos = consumer.partitionsFor(topic);
            Utils.sleep(Math.min(time.milliseconds() - started, 1000));
        }
        if (partitionInfos == null)
            throw new ConnectException("Could not look up partition metadata for offset backing store topic in" +
                    " allotted period. This could indicate a connectivity issue, unavailable topic partitions, or if" +
                    " this is your first use of the topic it may have taken too long to create

Answer 1

主题是手动创建的，

Kafka Connect 将使用 AdminClient 来初始化自己的主题。没有理由手动进行。

如果这样做，您需要确保设置正确，这些设置都记录在 Confluent 和 Apache Kafka 文档中，用于开始使用 Connect