kafka connect avro 找不到适合 jdbc 的驱动程序：mysql://127.0.0.1:3306/connect_test

Question

我正在关注 Confluent Kafka Connectdocker教程。

我启动了 kafka-connect docker 镜像，并在 docker 日志中检查它是否启动正常。

docker run -d \
  --name=kafka-connect-avro \
  --net=host \
  -e CONNECT_BOOTSTRAP_SERVERS=localhost:29092 \
  -e CONNECT_REST_PORT=28083 \
  -e CONNECT_GROUP_ID="quickstart-avro" \
  -e CONNECT_CONFIG_STORAGE_TOPIC="quickstart-avro-config" \
  -e CONNECT_OFFSET_STORAGE_TOPIC="quickstart-avro-offsets" \
  -e CONNECT_STATUS_STORAGE_TOPIC="quickstart-avro-status" \
  -e CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_STATUS_STORAGE_REPLICATION_FACTOR=1 \
  -e CONNECT_KEY_CONVERTER="io.confluent.connect.avro.AvroConverter" \
  -e CONNECT_VALUE_CONVERTER="io.confluent.connect.avro.AvroConverter" \
  -e CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL="http://localhost:8081" \
  -e CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL="http://localhost:8081" \
  -e CONNECT_INTERNAL_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_INTERNAL_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_REST_ADVERTISED_HOST_NAME="localhost" \
  -e CONNECT_LOG4J_ROOT_LOGLEVEL=DEBUG \
  -e CONNECT_PLUGIN_PATH=/usr/share/java,/etc/kafka-connect/jars \
  -v /tmp/quickstart/file:/tmp/quickstart \
  -v /tmp/quickstart/jars:/etc/kafka-connect/jars \
  confluentinc/cp-kafka-connect:latest

当我尝试发布到数据库时

curl -X POST   -H "Content-Type: application/json"   --data '{ "name": "quickstart-jdbc-source", "config": { "connector.class": "io.confluent.connect.jdbc.JdbcSourceConnector", "tasks.max": 1, "connection.url": "jdbc:mysql://127.0.0.1:3306/connect_test?user=root&password=confluent", "mode": "incrementing", "incrementing.column.name": "id", "timestamp.column.name": "modified", "topic.prefix": "quickstart-jdbc-", "poll.interval.ms": 1000 } }'   http://$CONNECT_HOST:28083/connectors

我明白了

"error_code":400,"message":"Connector configuration is invalid and contains the following 2 error(s):\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://127.0.0.1:3306/connect_test?user=root&password=confluent for configuration Couldn't open connection to jdbc:mysql://127.0.0.1:3306/connect_test?user=root&password=confluent\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://127.0.0.1:3306/connect_test?user=root&password=confluent for configuration Couldn't open connection to jdbc:mysql://127.0.0.1:3306/connect_test?user=root&password=confluent\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"}

mysql jdbc jar 有问题吗？

更详细地说，从下面的日志看来，CONNECT_PLUGIN_PATH 什么都不做。

[2019-08-28 17:19:27,113] INFO Added alias 'BasicAuthSecurityRestExtension' to plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
    plugin.path = [/usr/share/java, /etc/kafka-connect/jars]
[2019-08-28 17:19:27,231] WARN The configuration 'plugin.path' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)

从下方挂载成功

docker inspect kafka-connect-avro

{
       "Type": "bind",
       "Source": "/tmp/quickstart/jars",
       "Destination": "/etc/kafka-connect/jars",
       "Mode": "",
       "RW": true,
       "Propagation": "rprivate"
},

Answer 1

我通过将 jar 复制到 docker 映像中来完成这项工作

docker cp /tmp/quickstart/jars/mysql-connector-java-8.0.17.jar kafka-connect-avro:/usr/share/java/kafka