【问题标题】:failed: Connection reset by peer失败:对等方重置连接
【发布时间】:2018-02-19 15:06:01
【问题描述】:

我一直在使用 apache cassandra 3.0.11

  • Datastax 驱动程序:3.1.3
  • Netty 版本:4.1.6.Final

我们在数据库端经常遇到以下错误:

INFO  [SharedPool-Worker-2] 2017-09-11 01:39:31,738 Message.java:615 - Unexpected exception during request; channel = [id: 0x22d0c179, L:/xx.xx.xx.xx:9042 ! R:/yy.yy.yy.yy:57774]
io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed: Connection reset by peer
        at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
INFO  [SharedPool-Worker-2] 2017-09-11 01:44:25,789 Message.java:615 - Unexpected exception during request; channel = [id: 0xace60b99, L:/xx.xx.xx.xx:9042 ! R:/yy.yy.yy.yy:54372]
io.netty.channel.unix.Errors$NativeIoException: syscall:read(...)() failed: Connection reset by peer
        at io.netty.channel.unix.FileDescriptor.readAddress(...)(Unknown Source) ~[netty-all-4.0.44.Final.jar:4.0.44.Final]

从应用程序端我们得到以下错误:

08/24/17 05:30:08.321   DEBUG [CassandraAccessor:57] - executing CQL [select * from src where host_ip= 'cc.cc.cc.cc']
08/24/17 05:30:08.319   DEBUG [Qwerty:49] - Finding route for [ Source IP : yy.yy.yy.yy , Source Port : 16107 , Messaage Type : ISONMQ
08/24/17 05:30:08.322   DEBUG [CassandraAccessor:57] - executing CQL [select * from test]
08/24/17 05:30:08.321   ERROR [RequestHandler$SpeculativeExecution$1:333] - Unexpected error while querying /xx.xx.xx.xx
com.datastax.driver.core.exceptions.ConnectionException: [/xx.xx.xx.xx:9042] Pool is CLOSING
        at com.datastax.driver.core.HostConnectionPool.borrowConnection(HostConnectionPool.java:196)
        at com.datastax.driver.core.RequestHandler$SpeculativeExecution.query(RequestHandler.java:293)
        at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:272)
        at com.datastax.driver.core.RequestHandler.startNewExecution(RequestHandler.java:115)
        at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:95)
        at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:132)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43)
        at org.springframework.cassandra.core.CqlTemplate$3.doInSession(CqlTemplate.java:286)
        at org.springframework.cassandra.core.CqlTemplate$3.doInSession(CqlTemplate.java:283)
        at org.springframework.cassandra.core.CqlTemplate.doExecute(CqlTemplate.java:276)
        at org.springframework.cassandra.core.CqlTemplate.doExecuteQueryReturnResultSet(CqlTemplate.java:283)
        at org.springframework.data.cassandra.core.CassandraTemplate.select(CassandraTemplate.java:594)
        at org.springframework.data.cassandra.core.CassandraTemplate.select(CassandraTemplate.java:376)
        at com.fs.searshc.commlink.db.cassandra.RouterData.findBySourceIp(RouterData.java:27)
        at com.fs.searshc.commlink.router.tcpip.CommLinkRouter.findRoute(CommLinkRouter.java:75)

任何形式的帮助都将不胜感激。

【问题讨论】:

    标签: netty datastax-enterprise datastax-java-driver cassandra-3.0


    【解决方案1】:

    Connection reset by peer 表示远程peer强行关闭了连接。所以你需要找出它为什么这样做。

    【讨论】:

    • 感谢诺曼的回复。它是一个集群,关闭连接的节点也是集群的一部分。有没有办法找出为什么对等节点强制关闭连接?
    • 您将检查对等体,并且在使用 tcpdump 或 wireshark 之类的东西时可能能够更好地理解问题
    • 按照 Norman 的建议,我们运行 tcpdump 并得到以下输出。
    • 我们已经运行了 tcpdump 并得到了以下输出。应用服务器:xx.xx.xx.139 Cassandra server1:xx.xx.xx.78 cassandra server2:xx.xx.xx.79 TCP dump from application server:xx.xx.xx.139 2017-09-20 05: 09:22.170273 IP xx.xx.xx.78.9042 > xx.xx.xx.139.44408:标志 [R.],seq 558647898,ack 47750391,win 6145,选项 [nop,nop,TS val 3419854254 ecr 1293665900],长度 0 2017-09-20 05:09:22.170319 IP xx.xx.xx.79.9042 > xx.xx.xx.139.42696: 标志 [R.], seq 594588671, ack 44660661, win 6146, options [nop,nop,TS val 3410573711 ecr 1293665900],长度 0
    • 谁能帮我解码 tcpdump 输出。我想知道哪个主机是这里的罪魁祸首并发送 RESET。
    猜你喜欢
    • 2013-08-25
    • 1970-01-01
    • 1970-01-01
    • 2013-01-05
    • 1970-01-01
    • 1970-01-01
    • 2014-03-09
    • 2017-02-22
    • 2012-10-13
    相关资源
    最近更新 更多