Jetty 的 httpClient.setResponseBufferSize() 方法有什么作用吗？

Question

我正在使用 Jetty 的 HttpClient 构建一个简单的代理服务器。我在 Java 1.8.0_45 上使用 Jetty 版本 9.3.10.v20160621。

我有一种情况，我对资源执行 GET 操作，该资源将返回大小约为 3.5M 的响应（我使用soapUI 确定了这一点）。知道 Jetty 的默认最大响应大小为 2M，我在创建 HttpClient 实例时执行以下操作。

        HttpClient client = new HttpClient();
        client.setResponseBufferSize(4194304);
        client.start();

稍后我执行一个同步 GET 请求，如下所示：

        System.out.println("response buffer size = " + client.getResponseBufferSize());
        retVal = client.GET(uri);

控制台日志有以下内容：

响应缓冲区大小 = 4194304

尽管如此，当我执行 GET()（下面的堆栈跟踪）时，我得到了 java.util.concurrent.ExecutionException。我只能得出结论，要么 (a) setResponseBufferSize() 方法中存在错误，要么 (b) setResponseBufferSize() 方法没有按照文档所说的那样执行（设置响应缓冲区大小）。有人知道这是怎么回事吗？

java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Buffering capacity exceeded
    at org.eclipse.jetty.client.util.FutureResponseListener.getResult(FutureResponseListener.java:118)
    at org.eclipse.jetty.client.util.FutureResponseListener.get(FutureResponseListener.java:101)
    at org.eclipse.jetty.client.HttpRequest.send(HttpRequest.java:652)
    at org.eclipse.jetty.client.HttpClient.GET(HttpClient.java:343)
    at oracle.paas.tools.sifter.proxy.ProxySession.get(ProxySession.java:106)
    at oracle.paas.tools.sifter.proxy.Endpoint.doGet(Endpoint.java:75)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:837)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:524)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Buffering capacity exceeded
    at org.eclipse.jetty.client.util.BufferingResponseListener.onContent(BufferingResponseListener.java:114)
    at org.eclipse.jetty.client.api.Response$Listener$Adapter.onContent(Response.java:248)
    at org.eclipse.jetty.client.ResponseNotifier.notifyContent(ResponseNotifier.java:124)
    at org.eclipse.jetty.client.ResponseNotifier.access$100(ResponseNotifier.java:35)
    at org.eclipse.jetty.client.ResponseNotifier$ContentCallback.process(ResponseNotifier.java:272)
    at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
    at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
    at org.eclipse.jetty.client.ResponseNotifier.notifyContent(ResponseNotifier.java:117)
    at org.eclipse.jetty.client.HttpReceiver.responseContent(HttpReceiver.java:326)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.content(HttpReceiverOverHTTP.java:256)
    at org.eclipse.jetty.http.HttpParser.parseContent(HttpParser.java:1584)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1332)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:158)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:119)
    at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
    at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
    at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:115)
    ... 9 more

Answer 1

HttpClient.setResponseBufferSize(int) 用于配置用于从网络读取单个缓冲区的内部缓冲区大小。

它与您的响应正文内容大小无关。

您正在处理的情况是简单的HttpClient.GET(uri) 调用会将响应缓冲到FutureResponseListener 中，该FutureResponseListener 限制为2MB 的内存使用量。

这个限制只是 java Future 概念的副作用。

这是一种简写/方便的方法，适用于小型响应。

您有一个更大的响应，因此您可以使用 HttpClient 的异步功能从 HttpClient 管理的响应内容缓冲区/流中读取数据，并根据您的需要处理数据。

例子：

    // In Initialization Code, start the client
    HttpClient client = new HttpClient();
    client.start();

    // In code much later on, use the client
    // Don't constantly start/stop the HttpClient
    // Treat the HttpClient as a browser, and each newRequest() as
    //   a tab on that browser.
    InputStreamResponseListener listener = new InputStreamResponseListener();
    // Send asynchronously with the InputStreamResponseListener
    client.newRequest(uri).send(listener);

    // Call to the listener's get() blocks until the headers arrived
    Response response = listener.get(5, TimeUnit.SECONDS);

    // Now check the response information that arrived to decide whether to read the content
    if (response.getStatus() == 200)
    {
        byte[] buffer = new byte[256];
        try (InputStream input = listener.getInputStream())
        {
            while (true)
            {
                int read = input.read(buffer);
                if (read < 0)
                    break;
                // Do something with the bytes just read
            }
        }
    }
    else
    {
        response.abort(new Exception());
    }

查看 HttpClient Usage.java 了解更多示例

Answer 2

我找到了另一种解决问题的方法。这个解决方案不像 Joakim 的解决方案那样通用。如果响应大于 4 MiB（或您指定的任何内容），它将获得相同的异常。另一方面，它的代码更少。

Request request = client.newRequest(uri);
FutureResponseListener listener = new FutureResponseListener(request, 4 * 1024 * 1024);
request.send(listener);
retVal = listener.get();