【发布时间】：2021-08-27 09:30:22
【问题描述】：
我配置了一个 AWS ElasticSearch 域。我在 EC2 中安装了 LogStash，它用于将数据推送到 ES 域中，我可以在 Kibana Discover 中查看它们。从昨天开始，Discover 无法正常加载。当我单击主屏幕中的发现时，它会加载一段时间然后停止，但屏幕全是空白。
在 EC2 服务器内部，当我检查 LogStash 状态 (service logstash status) 时，我看到多个错误和警告。
logstash[8997]: [INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
logstash[8997]: [INFO ][org.logstash.beats.Server][main][e4620d298b3dbd13b2a68a1f1fcd717a9e12c8414d40373b93cefd5adeacb1c9] Starting server on port: 5044
logstash[8997]: [WARN ][logstash.outputs.amazonelasticsearch][main][43b973399a83b46ed67e936feeb816720e9ea3d1d511c25b9079ddc2091ccef7] Marking url as dead. Last error: [LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::ClientProtocolException] _esdomain_.es.amazonaws.com:443 failed to respond {:url=>https://_esdomain_.es.amazonaws.com:443/, :error_message=>"Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::ClientProtocolException] _esdomain_.es.amazonaws.com:443 failed to respond", :error_class=>"LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError"}
logstash[8997]: [ERROR][logstash.outputs.amazonelasticsearch][main][43b973399a83b46ed67e936feeb816720e9ea3d1d511c25b9079ddc2091ccef7] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::ClientProtocolException] _esdomain_.es.amazonaws.com:443 failed to respond", :class=>"LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>4}
logstash[8997]: [INFO ][logstash.outputs.amazonelasticsearch][main] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://_esdomain_.es.amazonaws.com:443/, :path=>"/"}
logstash[8997]: [WARN ][logstash.outputs.amazonelasticsearch][main] Restored connection to ES instance {:url=>"https://_esdomain_.es.amazonaws.com:443/"}
logstash[8997]: [WARN ][logstash.outputs.amazonelasticsearch][main][43b973399a83b46ed67e936feeb816720e9ea3d1d511c25b9079ddc2091ccef7] Marking url as dead. Last error: [LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::SocketException] Broken pipe (Write failed) {:url=>https://_esdomain_.es.amazonaws.com:443/, :error_message=>"Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::SocketException] Broken pipe (Write failed)", :error_class=>"LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError"}
logstash[8997]: [ERROR][logstash.outputs.amazonelasticsearch][main][43b973399a83b46ed67e936feeb816720e9ea3d1d511c25b9079ddc2091ccef7] Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down! {:error_message=>"Elasticsearch Unreachable: [https://_esdomain_.es.amazonaws.com:443/][Manticore::SocketException] Broken pipe (Write failed)", :class=>"LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::HostUnreachableError", :will_retry_in_seconds=>16}
logstash[8997]: [INFO ][logstash.outputs.amazonelasticsearch][main] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>https://_esdomain_.es.amazonaws.com:443/, :path=>"/"}
logstash[8997]: [WARN ][logstash.outputs.amazonelasticsearch][main] Restored connection to ES instance {:url=>"https://_esdomain_.es.amazonaws.com:443/"}
我不知道它们是否与 Kibana Discover 的无响应有关。
【问题讨论】：
-
你检查过 elasticsearch 日志吗？由于 Kibana 和 Logstash 都无法正常工作，您似乎在使用 elasticsearch 时可能会遇到一些问题。检查您的 elasticsearch 日志。
-
@leandrojmp 我正在使用 AWS ElasticSearch 服务。所以我无法连接到任何 ElasticSearch 服务器并检查其日志。 AWS ES 仪表板上有一个名为“日志”的选项卡，尽管我必须手动激活它们。
-
好吧，根据您的 logstash 日志和无响应的 Kibana，您似乎遇到了 elasticsearch 问题，因此您需要一种方法来查看您的 elasticsearch 服务发生了什么。我不知道这个日志选项卡是否有帮助，从未使用过 AWS Elasticsearch，但根据documentation，您需要启用它。
标签： elasticsearch logstash kibana elastic-stack aws-elasticsearch