Apache DefaultHttpClient - java.net.BindException：地址已在使用中：连接

Question

我正在访问 Tomcat 8.5 Web 服务器的 Java“客户端”中运行性能测试。在大约 13,000 个请求之后，HTTP 请求失败并出现错误，

org.apache.http.impl.client.DefaultHttpClient tryConnect
INFO: Retrying connect
java.net.BindException: Address already in use: connect
    at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
    at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
    at java.net.PlainSocketImpl.connect(Unknown Source)
    at java.net.SocksSocketImpl.connect(Unknown Source)
    at java.net.Socket.connect(Unknown Source)
    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
    at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)

代码是，

    for (int i = 0; i < 15000; i++) {
        try {
            if (i % 1000 == 0) System.out.println("Iterations: " + Integer.toString(i));
            HttpGet request = new HttpGet("http://localhost:9080");
            DefaultHttpClient client = new DefaultHttpClient();
            HttpResponse response = client.execute(request, new BasicHttpContext());
            HttpEntity entity = response.getEntity();
            EntityUtils.consume(entity);
        } catch (Exception e) {
            e.printStackTrace();
            System.out.println("Iterations: " + Integer.toString(i));
            System.exit(1);
        }
    }

如果我缓存 DefaultHttpClient，则不会发生错误。 也试过了，

        request.releaseConnection();
        client.getConnectionManager().shutdown();

但不会改变错误。 该错误似乎不是由客户端引起的。如果我在 URL 中访问另一个网站，那就没问题了。似乎是由于 Windows 中的 Tomcat 耗尽了文件句柄或套接字资源之类的。 如果我在它崩溃后立即将它作为另一个进程再次运行，它将在 1 次运行中失败，而不是 13,000 次，所以问题是 Tomcat 资源不足。似乎 DefaultHttpClient 没有关闭它的连接，或者 Tomcat 在 gc 发生之前没有释放它的连接。

使用 HTTPClient 4.2.5

任何想法为什么会发生，或者如何解决？

Answer 1

我无法重现您遇到的相同错误。无论如何，当我在单线程中运行您的示例时，我收到 NoRouteToHostException。

13:37:57.917 [main] DEBUG org.apache.http.impl.conn.BasicClientConnectionManager - Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@2c7ceffa
Iterations: 16329
java.net.NoRouteToHostException: Can't assign requested address (Address not available)
        at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
        at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
        at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
        at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
        at java.base/java.net.Socket.connect(Socket.java:591)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:605)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
    at com.demo.DemoApplication.main(DemoApplication.java:25)

您打开连接的速度比关闭连接的速度要快 - 关闭套接字在释放它们可用于新连接之前处于 TIME_WAIT 状态。

仅出于测试目的，您可以设置 tcp_tw_reuse 以允许重用 TIME_WAIT 套接字。

在我的 OSX 上，我可以更改最大段生命周期以进行测试，并且错误消失了。

sudo sysctl -w net.inet.tcp.msl=1000 net.inet.tcp.msl: 15000 -> 1000

DefaultHttpClient 由 BasicClientConnectionManager 支持，它创建和管理单个连接，并且每次只为任何路由保留一个活动连接。

当连接释放回连接管理器时，它会保持活动状态以供重用并标记为可重用。

如果我缓存 DefaultHttpClient，则不会发生错误。

这正是解决方案。我相信正确的方法是使用单个 http 客户端并允许连接管理器完成其工作。

所有的连接管理在apache documentation中解释

4.2.5 已经很老了（2013 年 4 月）。如果您开始一个新项目，那么在撰写本文时更新到最新（4.5.12）是有意义的。

参考资料：

https://cwiki.apache.org/confluence/display/HTTPCOMPONENTS/FrequentlyAskedConnectionManagementQuestions

https://raby.sh/osx-where-are-my-time_wait.html

Answer 2

您似乎以错误的方式使用 HttpClient（即可能复制粘贴旧的遗留 sn-p）

• 首先一个客户端就足够了（一个线程）。
• 根据httpcomponents-client documentation 最好使用池连接管理
• 您的回复应该关闭

即

  PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
  CloseableHttpClient client = HttpClients.custom()
          .setConnectionManager(cm)
          .build();
  HttpContext context = HttpClientContext.create();
  try {
    final HttpGet request = new HttpGet("http://localhost:9080");
    for(int i = 0; i < 15000; i++) {
      CloseableHttpResponse response = httpClient.execute(request, context);
      try {
        HttpEntity entity = response.getEntity();
        EntityUtils.consume(entity);
      } finally {
        response.close();
      }
    }
  } finally {
    client.close();
  }
}

Answer 3

最佳实践是使用多线程HttpClient。下面应该工作：

package com.demo.httpclient;

import java.net.URI;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.DefaultClientConnection;
import org.apache.http.impl.conn.PoolingClientConnectionManager;
import org.apache.http.util.EntityUtils;

public class ThreadedHttpClientTest {

    private static URI rootUri = URI.create("http://localhost:8080/");
    private static int worker = 100;
    private static int count = 15000;

    private static HttpClient httpClient;

    public static void main(String[] args) throws Exception {

        long startTime = System.currentTimeMillis();
        PoolingClientConnectionManager poolingClientConnectionManager = new PoolingClientConnectionManager();
        poolingClientConnectionManager.setMaxTotal(worker);
        poolingClientConnectionManager.setDefaultMaxPerRoute(worker);

        httpClient = new DefaultHttpClient(poolingClientConnectionManager);

        List<Callable<Void>> workers = new ArrayList<Callable<Void>>();

        for (int i = 0; i < count; i++) {
            workers.add(new WorkerThread(httpClient, rootUri.toString()));
        }

        ExecutorService pool = Executors.newFixedThreadPool(worker);

        int i=0;
        for (Future<Void> future : pool.invokeAll(workers)) {
            future.get();
            System.out.println("Response " + i++);
        }

        System.out.println("Time Taken :: " + (System.currentTimeMillis() - startTime) + "ms");
        pool.shutdown();
    }

    static class WorkerThread implements Callable<Void> {

        HttpClient client;
        String url;

        public WorkerThread(HttpClient httpClient, String url) {
            this.client = httpClient;
            this.url = url;
        }

        @Override
        public Void call() throws Exception {
            HttpGet get = new HttpGet(url);
            HttpResponse response = client.execute(get, new DefaultClientConnection());
            HttpEntity entity = response.getEntity();
            EntityUtils.consume(entity);
            return null;
        }
    }
}