Fluentd 无法连接到 Elasticsearch - [警告]：#0 [elasticsearch] 无法连接 Elasticsearch 或获取版本。假设 Elasticsearch 5

Question

我正在尝试在裸机 Kubernetes 上安装 Fluentd 并将消息转发到 ElasticSearch。 我想安装的 Fluentd 是“stable/fluentd --version 2.4.3”舵图。 我的 ElasticSearch 有自定义 tls 证书并且可以正常工作。

#### Steps to replicate
bash-5.0# curl -u "elastic:636ByujI053Pl3Zu5E0ZsnG6" -k "https://escluster-es-http.elastic-system.svc.cluster.local:9200" | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   546  100   546    0     0   7280      0 --:--:-- --:--:-- --:--:--  7280
{
  "name": "escluster-es-default-0",
  "cluster_name": "escluster",
  "cluster_uuid": "0XnwElYuQICbi-QELvNjsw",
  "version": {
    "number": "7.8.0",
    "build_flavor": "default",
    "build_type": "docker",
    "build_hash": "757314695644ea9a1dc2fecd26d1a43856725e65",
    "build_date": "2020-06-14T19:35:50.234439Z",
    "build_snapshot": false,
    "lucene_version": "8.5.1",
    "minimum_wire_compatibility_version": "6.8.0",
    "minimum_index_compatibility_version": "6.0.0-beta1"
  },
  "tagline": "You Know, for Search"
}
bash-5.0#

我的流利配置是；

...
output:
  host: escluster-es-http.elastic-system.svc.cluster.local
  port: 9200
  scheme: https
  sslVersion: TLSv1_2
  buffer_chunk_limit: 2M
  buffer_queue_limit: 8

env: {}

extraEnvVars:
  - name: ELASTIC_PASSWORD
    valueFrom:
      secretKeyRef:
        name: escluster-es-elastic-user
        key: elastic
extraVolumes:
  - name: es-certs
    secret:
      defaultMode: 420
      secretName: es-tls
extraVolumeMounts:
  - name: es-certs
    mountPath: /certs
    readOnly: true
...

Fluentd 已安装，但无法连接到 ElasticSearch。

pod 日志：

devadmin@vdi-mk2-ubn:~/fluentd-stable$ kubectl logs fluentd-784b48c5c9-78cf4 -n elastic-system
2020-07-03 16:33:47 +0000 [info]: parsing config file is succeeded path="/etc/fluent/fluent.conf"
2020-07-03 16:33:47 +0000 [warn]: [elasticsearch] Could not connect Elasticsearch or obtain version. Assuming Elasticsearch 5.
2020-07-03 16:33:47 +0000 [info]: using configuration file: <ROOT>
  <match fluent.**>
    @type null
  </match>
  <source>
    @type forward
    port 24224
    bind "0.0.0.0"
  </source>
  <match fluentd.**>
    @type null
  </match>
  <source>
    @type http
    port 9880
    bind "0.0.0.0"
  </source>
  <source>
    @type monitor_agent
    bind "0.0.0.0"
    port 24220
    tag "fluentd.monitor.metrics"
  </source>
  <match **>
    @id elasticsearch
    @type elasticsearch
    @log_level "info"
    include_tag_key true
    host "escluster-es-http.elastic-system.svc.cluster.local"
    port 9200
    scheme http
    ssl_version TLSv1_2
    logstash_format true
    validate_client_version false
    <buffer>
      @type "file"
      path "/var/log/fluentd-buffers/kubernetes.system.buffer"
      flush_mode interval
      retry_type exponential_backoff
      flush_thread_count 2
      flush_interval 5s
      retry_forever
      retry_max_interval 30
      chunk_limit_size 2M
      queue_limit_length 8
      overflow_action block
    </buffer>
  </match>
  <system>
    root_dir "/tmp/fluentd-buffers/"
  </system>
</ROOT>
2020-07-03 16:33:47 +0000 [info]: starting fluentd-1.3.3 pid=1 ruby="2.3.3"
2020-07-03 16:33:47 +0000 [info]: spawn command to main:  cmdline=["/usr/bin/ruby2.3", "-Eascii-8bit:ascii-8bit", "/usr/local/bin/fluentd", "--under-supervisor"]
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-concat' version '2.3.0'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.11'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '3.0.2'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '2.1.6'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-prometheus' version '1.3.0'
2020-07-03 16:33:48 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.1'
2020-07-03 16:33:48 +0000 [info]: gem 'fluentd' version '1.3.3'
2020-07-03 16:33:48 +0000 [info]: adding match pattern="fluent.**" type="null"
2020-07-03 16:33:48 +0000 [info]: adding match pattern="fluentd.**" type="null"
2020-07-03 16:33:48 +0000 [info]: adding match pattern="**" type="elasticsearch"
2020-07-03 16:33:48 +0000 [warn]: #0 [elasticsearch] Could not connect Elasticsearch or obtain version. Assuming Elasticsearch 5.
2020-07-03 16:33:48 +0000 [info]: adding source type="forward"
2020-07-03 16:33:48 +0000 [info]: adding source type="http"
2020-07-03 16:33:48 +0000 [info]: adding source type="monitor_agent"
2020-07-03 16:33:48 +0000 [info]: #0 starting fluentd worker pid=10 ppid=1 worker=0
2020-07-03 16:33:48 +0000 [info]: #0 listening port port=24224 bind="0.0.0.0"
2020-07-03 16:33:48 +0000 [info]: #0 fluentd worker is now running worker=0

你能告诉我缺少什么吗？

感谢和问候

Answer 1

您的 Elasticsearch 集群似乎已启用 ssl。 Fluentd 还需要通过 ssl 连接 elastic