【问题标题】:Redis closing connections to clients while saving changed keys in backgroundRedis 关闭与客户端的连接,同时在后台保存更改的密钥
【发布时间】:2014-06-26 13:34:20
【问题描述】:

我在 EC2 上有一台 Redis 服务器,并连接了 2 台应用服务器。所有小型/中型。流量不高; 300 秒内只更改了 10 个键。我开始注意到应用服务器与 Redis 机器的连接错误:

 Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool

起初我认为这是我的池配置或我用来与 Redis 交互的 java 客户端的问题,但是当我注意到两个应用程序服务器总是会同时生成这些异常时,我很快就揭穿了这个理论他们总是成群结队地进来。然后我查看了 redis.log 并在出现错误时注意到以下输出:

[13939] 08 May 22:31:05.051 * 10 changes in 300 seconds. Saving... [13939] 08 May 22:31:05.342 * Background saving started by pid 13945 [13939] 08 May 22:31:09.357 - DB 0: 606477 keys (0 volatile) in 1048576 slots HT. [13939] 08 May 22:31:09.357 - 3 clients connected (0 slaves), 764180208 bytes in use [13939] 08 May 22:31:14.542 - DB 0: 606477 keys (0 volatile) in 1048576 slots HT. [13939] 08 May 22:31:25.947 - 3 clients connected (0 slaves), 764180208 bytes in use [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Accepted 10.123.29.90:56301 [13939] 08 May 22:31:25.947 - Accepted 10.42.105.60:35315 [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Accepted 10.123.29.90:56302 [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Accepted 10.42.105.60:35317 [13939] 08 May 22:31:25.947 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.947 - Accepted 10.123.29.90:56306 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.42.105.60:35318 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.123.29.90:56308 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.42.105.60:35319 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.42.105.60:35320 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.123.29.90:56310 [13939] 08 May 22:31:25.948 - Error writing to client: Connection reset by peer [13939] 08 May 22:31:25.948 - Accepted 10.42.105.60:35322 [13939] 08 May 22:31:27.652 - Accepted 10.42.105.60:35327 [13939] 08 May 22:31:27.872 - Accepted 10.42.105.60:35329 [13945] 08 May 22:31:27.926 * DB saved on disk

仅当 Redis 在后台保存新数据时才会发生错误。我正在使用 Redis 2.6。任何帮助表示赞赏。

编辑:下面使用 spring-data 配置 Redis 连接池

<bean id="redisPoolConfig" class="redis.clients.jedis.JedisPoolConfig" lazy-init="false"
      p:maxTotal="500"
      p:maxIdle="20"
      p:testOnBorrow="true"
      p:testOnCreate="true"
      p:testOnReturn="true"
      p:maxWaitMillis="30000"
    />
<bean id="jedisConnectionFactory" class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
      p:hostName="${REDIS_HOST}"
      p:port="${REDIS_PORT}"
      p:usePool="true"
      p:poolConfig-ref="redisPoolConfig"
    />

【问题讨论】:

  • 您有两个主机。但是每台主机都通过许多端口连接到 Redis 服务器。你知道这是为什么吗?多线程?例如,“接受 10.123.29.90:56301”、“接受 10.123.29.90:56302”。
  • 我不知道为什么会这样。我正在使用带有上述池配置(刚刚发布)的标准 redis-data Java 驱动程序(Jedis)。我没有做任何我自己的连接处理,一切都是开箱即用的。我将尝试升级到最新版本的 Redis,看看是否能解决问题。

标签: redis


【解决方案1】:

我想我找到了答案。升级到 2.8.9 并重新启动后,日志提醒我我的操作系统将打开文件描述符的数量限制为 1024,运行 Redis 实例很容易超过该数量。所以可能发生的事情是后台保存过程正在打开新的文件描述符,这会阻止新连接被接受,因为每个新的客户端连接都会打开一个文件。通过 ulimit -n 将限制增加到 10000 后,一切似乎都正常运行。

【讨论】:

    猜你喜欢
    • 2018-03-16
    • 2014-04-04
    • 1970-01-01
    • 2023-03-17
    • 1970-01-01
    • 2019-11-05
    • 1970-01-01
    • 2015-07-16
    • 1970-01-01
    相关资源
    最近更新 更多