Kafka Schema Registry 错误：无法将 Noop 记录写入 kafka 存储

Question

我正在尝试启动 kafka 模式注册表，但出现以下错误：无法将 Noop 记录写入 kafka 存储。堆栈跟踪如下。我检查了与 zookeeper、kafka 经纪人的连接——一切都很好。我可以向卡夫卡发布消息。我试图删除 _schema 主题，甚至重新安装 kafka，但这个问题仍然存在。昨天一切正常，但是今天，重新启动我的流浪盒后，出现了这个问题。我能做些什么吗？谢谢

[2015-11-19 19:12:25,904] INFO SchemaRegistryConfig values: 
master.eligibility = true
port = 8081
kafkastore.timeout.ms = 500
kafkastore.init.timeout.ms = 60000
debug = false
kafkastore.zk.session.timeout.ms = 30000
request.logger.name = io.confluent.rest-utils.requests
metrics.sample.window.ms = 30000
schema.registry.zk.namespace = schema_registry
kafkastore.topic = _schemas
avro.compatibility.level = none
shutdown.graceful.ms = 1000
response.mediatype.preferred = [application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json]
metrics.jmx.prefix = kafka.schema.registry
host.name = 12bac2a9529f
metric.reporters = []
kafkastore.commit.interval.ms = -1
kafkastore.connection.url = master.mesos:2181
metrics.num.samples = 2
response.mediatype.default = application/vnd.schemaregistry.v1+json
kafkastore.topic.replication.factor = 3
(io.confluent.kafka.schemaregistry.rest.SchemaRegistryConfig:135)

[2015-11-19 19:12:25,904] INFO SchemaRegistryConfig values: 
master.eligibility = true
port = 8081
kafkastore.timeout.ms = 500
kafkastore.init.timeout.ms = 60000
debug = false
kafkastore.zk.session.timeout.ms = 30000
request.logger.name = io.confluent.rest-utils.requests
metrics.sample.window.ms = 30000
schema.registry.zk.namespace = schema_registry
kafkastore.topic = _schemas
avro.compatibility.level = none
shutdown.graceful.ms = 1000
response.mediatype.preferred = [application/vnd.schemaregistry.v1+json, application/vnd.schemaregistry+json, application/json]
metrics.jmx.prefix = kafka.schema.registry
host.name = 12bac2a9529f
metric.reporters = []
kafkastore.commit.interval.ms = -1
kafkastore.connection.url = master.mesos:2181
metrics.num.samples = 2
response.mediatype.default = application/vnd.schemaregistry.v1+json
kafkastore.topic.replication.factor = 3
(io.confluent.kafka.schemaregistry.rest.SchemaRegistryConfig:135)
[2015-11-19 19:12:26,535] INFO Initialized the consumer offset to -1        (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:87)
[2015-11-19 19:12:27,167] WARN Creating the schema topic _schemas using a replication factor of 1, which is less than the desired one of 3. If this is a production environment, it's crucial to add more brokers and increase the replication factor of the topic.   (io.confluent.kafka.schemaregistry.storage.KafkaStore:172)
[2015-11-19 19:12:27,262] INFO [kafka-store-reader-thread-_schemas], Starting  (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:68)
[2015-11-19 19:13:27,350] ERROR Error starting the schema registry (io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication:57)
io.confluent.kafka.schemaregistry.exceptions.SchemaRegistryInitializationException: Error initializing kafka store while initializing schema registry
at   io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:164)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:55)
at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.setupResources(SchemaRegistryRestApplication.java:37)
at io.confluent.rest.Application.createServer(Application.java:104)
at io.confluent.kafka.schemaregistry.rest.Main.main(Main.java:42)
Caused by: io.confluent.kafka.schemaregistry.storage.exceptions.StoreInitializationException: io.confluent.kafka.schemaregistry.storage.exceptions.StoreException: Failed to write Noop record to kafka store.
at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:151)
at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:162)
... 4 more
Caused by: io.confluent.kafka.schemaregistry.storage.exceptions.StoreException: Failed to write Noop record to kafka store.
at io.confluent.kafka.schemaregistry.storage.KafkaStore.getLatestOffset(KafkaStore.java:363)
at io.confluent.kafka.schemaregistry.storage.KafkaStore.waitUntilKafkaReaderReachesLastOffset(KafkaStore.java:220)
at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:149)
... 5 more

Answer 1

错误消息具有误导性，正如其他开发人员在其他帖子中所建议的那样，我建议如下。

1) 确保 Zookeeper 正在运行。 （检查日志文件以及进程是否处于活动状态）

2) 确保你的 kafka 集群中的各个节点可以相互通信（telnet 到主机和端口）

3) 如果 1 和 2 都很好，那么 我不建议 创建另一个主题（如某些人在其他帖子中推荐的 _schema2）并使用以下内容更新 schemaregistry 配置文件 kafkastore.topic新主题。
反而 3.1）停止进程（zookeeper、kafka server） 3.2）清理zookeeper数据目录下的数据 3.3）重启zookeeper，kafka服务器，最后重启schemaregistry服务（应该可以了！）

P.S：如果您确实尝试创建另一个主题，那么当您尝试使用来自 kafka 主题的数据时可能会卡住。（发生在我身上，我花了几个小时才弄清楚这一点！！！）。

Answer 2

我遇到了同样的错误。问题是我希望 Kafka 在 ZooKeeper 中使用 kafka 命名空间，所以我将它设置在 schema-registry.properties

kafkastore.connection.url=localhost:2181/kafka

但是在 Kafka server.properties 我根本没有设置它。配置包含

zookeeper.connect=localhost:2181

所以我只是将 ZooKeeper 命名空间添加到此属性并重新启动 Kafka

zookeeper.connect=localhost:2181/kafka

可能您的问题是您的架构注册表需要“/”命名空间，但在您的 Kafka 配置中您定义了其他内容。你能发布Kafka配置吗？

或者，您可以使用 zkCli.sh 找出 ZooKeeper Kafka 存储主题信息的位置。

/bin/zkCli.sh localhost:2181
Welcome to ZooKeeper!
ls /kafka
[cluster, controller, controller_epoch, brokers, admin, isr_change_notification, consumers, latest_producer_id_block, config]

Answer 3

我在 schema-registry.properties 中进行了以下对我有用的更改：

#kafkastore.connection.url=localhost:2181
kafkastore.bootstrap.servers=PLAINTEXT://localhost:9092
kafkastore.topic=<topic name>

我还针对启动服务器的另一个问题执行了以下命令：

./bin/kafka-topics --alter --zookeeper localhost:2181 --topic <topic name> --config cleanup.policy=compact

祝你好运！

Answer 4

根据我的经验，当架构注册表的内部启动损坏时会发生这种情况。如果重新创建主题无助于重新启动架构注册表即可解决问题。