Ruby on Rails - ElasticSearch 结果窗口太大

Question

我正在使用 elasticsearch-rails gem。

当用户在我的分页中单击更高页码的链接时，我收到以下错误。这是错误：

Elasticsearch::Transport::Transport::Errors::InternalServerError ([500] {"error":{"root_cause":[{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [48700]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"products","node":"fNcaDjwzRRGu2fq0KjTWUQ","reason":{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [48700]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}}]},"status":500}):

我在哪里设置 index.max_result_window 或者这甚至是解决此问题的正确方法？

Answer 1

您真的需要让用户对 48000 个条目进行深度分页吗？这里的问题在于 elasticsearch 内部是如何工作的。如果您查询第 N 个结果，elasticsearch 必须同时获取前 N-1 个结果才能丢弃它们。这就是为什么随着用户分页越深，此类查询的开销就越大。

滚动 API 可能是一个很好的选择，如果您只有很少的用户并且他们都表现良好（因为他们不会同时打开无数滚动上下文并使它们永远保持活动状态）但可能不是。

如果可以选择，请将您的分页限制为这 10k 个结果（甚至更少），并鼓励您的用户在他们的查询中更加具体。如果没有，请准备好扩展集群中的硬件（内存！）并进行长时间运行的查询。