【问题标题】:Elasticsearch RAM issue on production server生产服务器上的 Elasticsearch RAM 问题
【发布时间】:2017-01-31 23:40:15
【问题描述】:

我在生产服务器上使用带有 searchkick 和 MongoDB 的 elasticsearch 2.3.4。我们在 2GB RAM aws 上运行我们的整个 rails 应用程序。 Elasticsearch 正在消耗 RAM,导致服务器宕机。请问..我该如何解决这个问题?需要帮助

这是我的 elasticsearch.yml 文件

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
 cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
# 
 node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
# bootstrap.mlockall: true
#
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
# bootstrap.mlockall: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
# network.host: ********
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
# network.host: *******
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
# 
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
# 
# discovery.zen.minimum_master_nodes: 3
# 
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------

#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
# 
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:

#
# gateway.recover_after_nodes: 3
# 
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
# 
# Require explicit names when deleting indices:
# 
# action.destructive_requires_name: true

【问题讨论】:

  • Elasticsearch 在某处建议使用 8 GB 固定内存对其进行配置。如果你没有空间的东西或大量的数据,那么工作量就会减少,但所有东西的 2 GB 似乎有点低。您的索引中有多少文档?
  • 目前我们有大约 12000 个文档。未来它会变得巨大!
  • “巨大”是非常主观的。如果你有几百万个文件(很容易得到父子)。将内存配置为 4+ GB 左右,看看它的表现如何。 “巨大”的 2 GB 是不够的。
  • 是的,就像你说的以百万计。它在 6GB 机器上运行良好。这是我遇到问题的 2GB 服务器。
  • 在你的 yml 中阅读他们在记忆下所说的内容。只要您只有 12000 个文档,您就可以尝试设置环境变量并给它 512MB。但不要指望它有很好的性能。

标签: ruby-on-rails amazon-web-services elasticsearch


【解决方案1】:

2 GB RAM 不足以运行所有这些。垂直分离多个实例中的组件或将实例扩展为具有更多 RAM 的更大实例类型。

【讨论】:

  • 是的,这正是我的意思。这将有助于释放一些 RAM,并在未来为您提供更多的灵活性和可扩展性。
猜你喜欢
  • 1970-01-01
  • 2011-04-18
  • 2013-09-21
  • 1970-01-01
  • 2013-05-17
  • 2011-09-14
  • 1970-01-01
  • 1970-01-01
  • 2012-06-25
相关资源
最近更新 更多