基于 Spring WebFlux/Reactive Mongo 的应用程序打开到 mongo db 的多个连接

【问题标题】：Spring WebFlux/Reactive Mongo based application opens multiple connections to mongo db基于 Spring WebFlux/Reactive Mongo 的应用程序打开到 mongo db 的多个连接
【发布时间】：2020-04-03 16:41:45
【问题描述】：

我有一个简单的 web 应用程序，带有 spring boot 2.x.x 、用于响应式 API 的 Spring WebFlux 和响应式 Mongo 存储库。 在应用程序中，我有一个与 get API 关联的@Tailablequery，以无限观察保存在 DB 中的更改。而且我有一个发布 API 来将数据推送到数据库中。 但是应用程序会随机打开多个到 mongo 的连接。有时这些连接会被重用。

日志

Opened connection [connectionId{localValue:4, serverValue:342}] to localhost:27017
Opened connection [connectionId{localValue:5, serverValue:344}] to localhost:27017

等等.. 所以我添加了一个配置类来限制每个主机的连接

  @Configuration
public class MongoConfig {

    @Value("${spring.data.mongodb.maxConnectionIdleTime}")
    private int  maxConnectionIdleTime;
    @Value("${spring.data.mongodb.connectionsPerHost}")
    private int connectionsPerHost;
    @Value("${spring.data.mongodb.minConnectionsPerHost}")
    private int minConnectionsPerHost;
    @Value("${spring.data.mongodb.socketTimeout}")
    private int socketTimeout;


    @Bean
    public MongoClientOptions mongoOptions() {
        return MongoClientOptions.builder()
                .maxConnectionIdleTime(this.maxConnectionIdleTime)
                .connectionsPerHost(this.connectionsPerHost)
                .minConnectionsPerHost(this.minConnectionsPerHost)
                .socketTimeout(this.socketTimeout)
                .build();

    }
}

现在，当我运行 Get api（无限期观察数据库）时，应用程序仍会打开多个到 DB 的连接，但由于空闲超时而关闭了其中的几个。

日志：

Opened connection [connectionId{localValue:6, serverValue:345}] to localhost:27017
Closed connection [connectionId{localValue:6, serverValue:345}] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionId{localValue:7, serverValue:348}] to localhost:27017
Closed connection [connectionId{localValue:7, serverValue:348}] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionId{localValue:8, serverValue:351}] to localhost:27017
Closed connection [connectionId{localValue:8, serverValue:351}] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionId{localValue:9, serverValue:354}] to localhost:27017

编辑

进一步观察，它正在打开一个连接，然后在 60 秒后关闭它，如果该连接没有被使用。例如，在下面的日志中，本地 id 为 5 和 6 的连接被用于通过 api 跟踪 mongo 事件，剩余的连接只是一分钟后关闭的空闲连接。 EDIT 将 mongo 配置更改为 ->

 server:
    port: 8081
spring:
    data:
        mongodb:
            database: LocalMongo
            uri: mongodb://localhost:27017/LocalMongo?maxpoolsize=5
            maxConnectionIdleTime: 15000
            connectionsPerHost: 40
            minConnectionsPerHost: 1
            socketTimeout: 60000

这样好吗？

2019-12-10 21:45:32.862[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[           main][0;39m [36mc.p.test.Application     [0;39m [2m:[0;39m Started Application in 3.489 seconds (JVM running for 4.87)
[2m2019-12-10 21:46:31.977[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionId{localValue:3, serverValue:899}] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:46:31.983[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:4, serverValue:903}] to localhost:27017
[2m2019-12-10 21:47:28.917[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[ntLoopGroup-2-2][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:5, serverValue:905}] to localhost:27017
[2m2019-12-10 21:47:31.122[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[ntLoopGroup-2-3][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:6, serverValue:906}] to localhost:27017
[2m2019-12-10 21:47:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionId{localValue:4, serverValue:903}] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:47:31.981[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:7, serverValue:907}] to localhost:27017
[2m2019-12-10 21:48:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionId{localValue:7, serverValue:907}] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:48:31.983[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:8, serverValue:910}] to localhost:27017
[2m2019-12-10 21:49:31.975[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionId{localValue:8, serverValue:910}] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:49:31.981[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionId{localValue:9, serverValue:913}] to localhost:27017
[2m2019-12-10 21:50:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionId{localValue:9, serverValue:913}] to localhost:27017 because it is past its maximum allowed idle time.
[2m

在这方面有最佳实践吗？

进一步的编辑 正如 cmets 中所建议的，我使用了uri: mongodb://localhost:27017/LocalMongo?maxpoolsize=5 将最大池大小设置为 5。mongo 确保按预期最多打开 5 个连接，并且一旦达到空闲超时，连接就会过期。 但是，每次连接终止时，都会打开一个新连接。基本上，看起来它确保打开了至少 5 个连接。有没有我在这里遗漏的概念？

日志：

Opened connection [connectionId{localValue:2, serverValue:2843}] to localhost:27017
Opened connection [connectionId{localValue:3, serverValue:2844}] to localhost:27017
: Opened connection [connectionId{localValue:4, serverValue:2846}] to localhost:27017
2019-12-13 13:00:17.316  INFO 10748 --- [ntLoopGroup-2-3] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:5, serverValue:2848}] to localhost:27017
2019-12-13 13:00:20.436  INFO 10748 --- [ntLoopGroup-2-4] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:6, serverValue:2849}] to localhost:27017
2019-12-13 13:00:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:3, serverValue:2844}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:00:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:7, serverValue:2851}] to localhost:27017
2019-12-13 13:01:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:7, serverValue:2851}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:01:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:8, serverValue:2854}] to localhost:27017
2019-12-13 13:02:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:8, serverValue:2854}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:02:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:9, serverValue:2856}] to localhost:27017
2019-12-13 13:03:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:9, serverValue:2856}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:03:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:10, serverValue:2859}] to localhost:27017
2019-12-13 13:04:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:10, serverValue:2859}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:04:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:11, serverValue:2862}] to localhost:27017
2019-12-13 13:05:39.122  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:11, serverValue:2862}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:05:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:12, serverValue:2865}] to localhost:27017
2019-12-13 13:06:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:12, serverValue:2865}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:06:39.126  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:13, serverValue:2868}] to localhost:27017
2019-12-13 13:07:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:13, serverValue:2868}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:07:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:14, serverValue:2870}] to localhost:27017
2019-12-13 13:08:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:14, serverValue:2870}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:08:39.128  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:15, serverValue:2873}] to localhost:27017
2019-12-13 13:09:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:15, serverValue:2873}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:09:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:16, serverValue:2876}] to localhost:27017
2019-12-13 13:10:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:16, serverValue:2876}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:10:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:17, serverValue:2879}] to localhost:27017
2019-12-13 13:11:39.122  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:17, serverValue:2879}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:11:39.129  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:18, serverValue:2882}] to localhost:27017
2019-12-13 13:12:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionId{localValue:18, serverValue:2882}] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:12:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:19, serverValue:2885}] to localhost:27017

如果需要更多信息，请告诉我。我将在此处编辑此空间。

【问题讨论】：

  • 它似乎正在关闭所有以前打开的连接，不是吗？
  • 不是全部...例如本地值为 1,2,3 的 connectionId 没有关闭...对于生产系统，每台主机的理想连接数是多少？

标签： mongodb spring-boot spring-webflux project-reactor


【解决方案1】：

这是由设置maxConnectionIdleTime=1引起的，您也可以在日志中看到：...because it is past its maximum allowed idle time.

【讨论】：

  • 但是每个主机的最大连接数也是1，那么它是如何打开多个连接的呢？我在第二组日志之前也更改了配置。我将在编辑部分添加它们
  • 如果您需要限制每个应用程序的连接数，请改用连接池设置。例如，只需像这样更改连接字符串：spring.data.mongodb.uri=mongodb://localhost:27017?maxIdleTimeMS=1500000&maxpoolsize=1
  • @ChiragC 以防万一它是如此简单 - 您粘贴在那里的配置将 minConnectionsPerHost 设置为 1，而不是 maxConnectionsPerHost。
  • @VladK 我使用您建议的参数将最大池大小设置为 5。现在，我在应用程序上添加了一些负载，它最终按预期打开了最多 5 个连接。一旦这些连接变得空闲，连接就会关闭，这也是预期的。但每次关闭一个连接时，它都会打开一个新连接，确保至少有 5 个连接。基本上， maxpoolsize=5 似乎更多的是要打开的最小连接数。我在这里错过了什么吗？
  • 没有这样的概念，max pool只定义了上限。可能它是由 MongoClientOptions 以某种方式引起的。我建议调试 Spring Mongo AutoConfigs，有时这很棘手......我最终没有暴露 MongoClientOptions 并在连接字符串中拥有所有设置，检查类 com.mongodb.ConnectionString 以获取属性名称和描述。
猜你喜欢
  • 2015-08-22
  • 2023-01-26
  • 2021-03-05
  • 2011-12-15
  • 2022-10-04
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2012-09-29
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode