SSL 配置后 Apache Kafka 不启动

Question

我在远程 Ubuntu 服务器上安装了 Apache Kafka (v. 2.13-3.0.0)。 我按照本教程来保护我的集群：

https://medium.com/egen/securing-kafka-cluster-using-sasl-acl-and-ssl-dec15b439f9d

但是当我尝试使用 jaas conf 文件使用以下命令启动 Kafka 时：

export KAFKA_OPTS=-Djava.security.auth.login.config=<kafka-binary- 
dir>/config/kafka_server_jaas.conf
./bin/kafka-server-start.sh ./config/server.properties

我收到错误：

[2021-11-12 10:30:47,864] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2021-11-12 10:30:48,089] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2021-11-12 10:30:48,099] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.ClassNotFoundException: kafka.security.auth.SimpleAclAuthorizer
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:417)
        at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
        at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
        at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
        at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
        at kafka.Kafka$.buildServer(Kafka.scala:67)
        at kafka.Kafka$.main(Kafka.scala:87)
        at kafka.Kafka.main(Kafka.scala)

这些是 server.properties 文件中的 SSL 配置：

########### SECURITY using SCRAM-SHA-512 and SSL 
listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
advertised.listeners=PLAINTEXT://localhost:9092,SASL_PLAINTEXT://localhost:9093,SASL_SSL://localhost:9094
security.inter.broker.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=
ssl.client.auth=required
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-512
sasl.enabled.mechanisms=SCRAM-SHA-512

# Broker security settings
ssl.truststore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/truststore/kafka.truststore.jks
ssl.truststore.password=giuseppe
ssl.keystore.location=/home/kafka/Downloads/kafka_2.13-3.0.0/config/keystore/kafka.keystore.jks
ssl.keystore.password=giuseppe
ssl.key.password=giuseppe

# ACLs
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:admin

#zookeeper SASL
zookeeper.set.acl=false
########### SECURITY using SCRAM-SHA-512 and SSL 

如果我尝试评论 2 行 ACL，我会收到错误：

[2021-11-12 11:05:29,301] INFO [ThrottledChannelReaper- 
ControllerMutation]: Starting 
(kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2021-11-12 11:05:29,331] ERROR [KafkaServer id=0] Fatal error 
during KafkaServer startup. Prepare to shutdown 
(kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Failed to acquire lock on 
file .lock in /tmp/kafka-logs. A Kafka instance in another process 
or thread is using this directory.
at kafka.log.LogManager.$anonfun$lockLogDirs$1(LogManager.scala:241)
at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:117)
at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:104)
    at scala.collection.mutable.ArraySeq.flatMap(ArraySeq.scala:37)
    at kafka.log.LogManager.lockLogDirs(LogManager.scala:236)
    at kafka.log.LogManager.<init>(LogManager.scala:112)
    at kafka.log.LogManager$.apply(LogManager.scala:1283)
    at kafka.server.KafkaServer.startup(KafkaServer.scala:254)
    at kafka.Kafka$.main(Kafka.scala:109)
    at kafka.Kafka.main(Kafka.scala)

原因是什么？可能是错误的配置？ 谢谢。

更新： 更改行：

# ACLs authorizer.class.name=org.apache.kafka.server.authorizer.Authorizer
there is this error: org.apache.kafka.common.KafkaException: Could not find 
a public no-argument constructor for 
org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)

我收到这个新错误：

[2021-11-12 16:51:57,613] ERROR Exiting Kafka due to fatal exception 
(kafka.Kafka$)
org.apache.kafka.common.KafkaException: Could not find a public no-argument 
constructor for org.apache.kafka.server.authorizer.Authorizer at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:392)
    at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:406)
    at kafka.security.authorizer.AuthorizerUtils$.createAuthorizer(AuthorizerUtils.scala:31)
    at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1583)
    at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1394)
    at kafka.Kafka$.buildServer(Kafka.scala:67)
    at kafka.Kafka$.main(Kafka.scala:87)
    at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.NoSuchMethodException: 
org.apache.kafka.server.authorizer.Authorizer.<init>()
    at java.base/java.lang.Class.getConstructor0(Class.java:3508)
    at java.base/java.lang.Class.getDeclaredConstructor(Class.java:2711)
    at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:390)
    ... 7 more

Answer 1

Kafka 3.0 移除了 SimpleAclAuthorizer

拉取请求 - https://github.com/apache/kafka/commit/976e78e405d57943b989ac487b7f49119b0f4af4#diff-e0ccf1b5c964d2c303b6a69a8b8b67df5a6bfbae8aa514f580d353c4c6bf8e36

该博客似乎使用的是 2.2.0 版本。

Answer 2

看来，如果你改变了

kafka.security.auth.SimpleAclAuthorizer

到

kafka.security.authorizer.AclAuthorizer

它应该可以工作；它对我有用。