【发布时间】：2019-12-02 23:29:54
【问题描述】：
我在尝试运行 kafka elasticsearch sink conect 时遇到以下错误，我已经验证，我可以在 plugin.path 位置看到所需的 jar。
罐子列表：
/home/confluentkafka/confluent-5.3.1/share/java/kafka-connect-elasticsearch/
audience-annotations-0.5.0.jar
gson-2.8.5.jar
httpcore-nio-4.4.6.jar
jsr305-3.0.2.jar
spotbugs-annotations-3.1.8.jar
commons-codec-1.9.jar
httpasyncclient-4.1.3.jar
jest-6.3.1.jar
kafka-connect-elasticsearch-5.3.1.jar
zkclient-0.10.jar
commons-logging-1.2.jar
httpclient-4.5.3.jar
jest-common-6.3.1.jar
netty-3.10.6.Final.jar
zookeeper-3.4.14.jar
common-utils-5.3.1.jar
httpcore-4.4.6.jar
jline-0.9.94.jar
slf4j-api-1.7.26.jar
错误信息：
[2019-12-02 02:02:05,555] ERROR WorkerSinkTask{id=elasticsearch-sink-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
java.lang.NoClassDefFoundError: org/apache/http/nio/conn/SchemeIOSessionStrategy
at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start(ElasticsearchSinkTask.java:122)
at io.confluent.connect.elasticsearch.ElasticsearchSinkTask.start(ElasticsearchSinkTask.java:51)
at org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:300)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:189)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.http.nio.conn.SchemeIOSessionStrategy
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 11 more
[2019-12-02 02:02:05,557] ERROR WorkerSinkTask{id=elasticsearch-sink-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
[2019-12-02 02:02:05,557] INFO Stopping ElasticsearchSinkTask. (io.confluent.connect.elasticsearch.ElasticsearchSinkTask:185)
[2019-12-02 02:03:26,374] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)
worker.proerties
bootstrap.servers=00.0.0.000:6667 #fake IP
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/home/confluentkafka/confluent-5.3.1/share/java/kafka-connect-elasticsearch
Connect.properties
name=elasticsearch-sink
connector.class=io.confluent.connect.elasticsearch.ElasticsearchSinkConnector
tasks.max=1
topics=test
topic.index.map=test:twitter-index
connection.url=http://00.0.0.000:5601
type.name=test
key.ignore=true
schema.ignore=true
【问题讨论】：
-
您的 Elasticsearch 主机真的在
http://00.0.0.000:5601吗？ (a) 看起来像一个无效的 IP 地址，并且 (b) 通常是 Kibana 端口，而 Elasticsearch 通常在9200 -
我这里已经用零替换了原来的IP，端口号是正确的。我可以使用这个 URL 访问 Kibana - 00.0.0.000:5601
-
对，但是 Kibana ！= Elasticsearch。如果您尝试将用于 Elasticsearch 的数据发送到 Kibana，它将失败。
-
我会检查这个并在更新后尝试，谢谢...
标签： elasticsearch apache-kafka apache-kafka-connect