Apache Http Client 4.5.11 生产环境配置最佳实践

Question

下面是我的ApacheHttpClient Spring bean，我想知道我的配置是否适合生产，还是我需要调整任何配置？如果您在生产中使用最新的 Apache Http 客户端，有人可以分享一些信息吗

@Service
public class ApacheHttpClient implements IHttpClient {
    private static final Logger              LOGGER                                             = Logger
            .getInstance(ApacheHttpClient.class);
    private static final int                 DEFAULT_MAX_TOTAL_CONNECTIONS                      = 400;
    private static final int                 DEFAULT_IDLE_CONNECTION_EVICTION_FREQUENCY_SECONDS = 300;
    private static final int                 DEFAULT_MAX_CONNECTIONS_PER_ROUTE                  = DEFAULT_MAX_TOTAL_CONNECTIONS;

    private static final int                 DEFAULT_CONNECTION_TIMEOUT_MILLISECONDS            = (60 * 1000);
    private static final int                 DEFAULT_READ_TIMEOUT_MILLISECONDS                  = (4 * 60 * 1000);
    private static final int                 DEFAULT_WAIT_TIMEOUT_MILLISECONDS                  = (60 * 1000);
    private static final int                 DEFAULT_VALIDATE_AFTER_INACTIVITY_MILLISECONDS     = (5 * 60 * 1000);

    private static final int                 DEFAULT_KEEP_ALIVE_MILLISECONDS                    = (5 * 60 * 1000);
    private static final int                 DEFAULT_REQUEST_RETRY                              = 2;

    @Autowired
    private SSLContextHelper                 customSSLContext;

    private int                              keepAlive                                          = DEFAULT_KEEP_ALIVE_MILLISECONDS;
    private int                              maxTotalConnections                                = DEFAULT_MAX_TOTAL_CONNECTIONS;

    private int                              maxConnectionsPerRoute                             = DEFAULT_MAX_CONNECTIONS_PER_ROUTE;
    private int                              connectTimeout                                     = DEFAULT_CONNECTION_TIMEOUT_MILLISECONDS;
    private int                              readTimeout                                        = DEFAULT_READ_TIMEOUT_MILLISECONDS;

    private int                              waitTimeout                                        = DEFAULT_WAIT_TIMEOUT_MILLISECONDS;
    private int                              requestRetry                                       = DEFAULT_REQUEST_RETRY;

    private CloseableHttpClient              httpClient;

    private ConnectionKeepAliveStrategy      keepAliveStrategy                                  = (response,
            context) -> {
                                                                                                    HeaderElementIterator it = new BasicHeaderElementIterator(
                                                                                                            response.headerIterator(
                                                                                                                    HTTP.CONN_KEEP_ALIVE));
                                                                                                    while (it
                                                                                                            .hasNext()) {
                                                                                                        HeaderElement he = it
                                                                                                                .nextElement();
                                                                                                        String param = he
                                                                                                                .getName();
                                                                                                        String value = he
                                                                                                                .getValue();
                                                                                                        if (value != null
                                                                                                                && param.equalsIgnoreCase(
                                                                                                                        "timeout")) {
                                                                                                            try {
                                                                                                                return Long
                                                                                                                        .parseLong(
                                                                                                                                value)
                                                                                                                        * 1000;
                                                                                                            } catch (NumberFormatException ignore) {}
                                                                                                        }
                                                                                                    }
                                                                                                    return keepAlive;
                                                                                                };


    @PostConstruct
    public void initializeApacheHttpClient() {

        // config timeout
        RequestConfig config = RequestConfig.custom()
                .setConnectTimeout(connectTimeout)
                .setConnectionRequestTimeout(waitTimeout)
                .setSocketTimeout(readTimeout).build();
        Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory> create()
                .register("http", PlainConnectionSocketFactory.INSTANCE)
                .register("https", new SSLConnectionSocketFactory(customSSLContext.getSSLContext())).build();

        PoolingHttpClientConnectionManager connManager = new PoolingHttpClientConnectionManager(socketFactoryRegistry);

        connManager.setMaxTotal(maxTotalConnections);

        // Increase default max connection per route
        connManager.setDefaultMaxPerRoute(maxConnectionsPerRoute);

        // Defines period of inactivity in milliseconds after which persistent connections must be re-validated prior to
        // being reused
        connManager.setValidateAfterInactivity(DEFAULT_VALIDATE_AFTER_INACTIVITY_MILLISECONDS);

        httpClient = HttpClients.custom().setKeepAliveStrategy(keepAliveStrategy).setConnectionManager(connManager)
                .setConnectionManagerShared(true).setSSLContext(customSSLContext.getSSLContext())
                .setDefaultRequestConfig(config)
                .setRetryHandler(new DefaultHttpRequestRetryHandler(requestRetry, true))
                .build();

        // detect idle and expired connections and close them
        IdleConnectionEvictor staleMonitor = new IdleConnectionEvictor(connManager, DEFAULT_IDLE_CONNECTION_EVICTION_FREQUENCY_SECONDS);
        staleMonitor.start();

        LOGGER.log(Level.INFO, "Initialize ApacheHttpClient is successful");
    }
}   

Answer 1

我不确定是否可以对这个问题给出明确客观的答案。没有适用于所有应用程序的单一设置集。您的生产环境和应用程序将是独一无二的。

要评估的一些项目是：

• 设置与您的用户或上游调用者可以容忍的响应时间一致的超时
• 设置与支持预期最大流量所需的连接数一致的连接限制
• 为您的应用设置适当的重试 - 您的应用会容忍重试，还是会因为重放请求而引发问题行为？

您还需要查看监控并观察您的日志和实时指标，了解连接、内存使用、响应时间、超时、错误等。

来自 Apache 的默认值可能是很好的默认值。 Apache 基金会和为其提供协助的工程师经验丰富，并且对这些默认设置深思熟虑。

Answer 2

（非官方）Apache HttpClient 5.0 迁移指南中有一节介绍了Apache HttpClient 4.5.x 版本的推荐配置参数和良好做法。

https://ok2c.github.io/httpclient-migration-guide/preparation.html

两个最重要的建议：

• 始终重复使用 CloseableHttpClient 实例。创建它们的成本很高，但它们也是完全线程安全的，因此多个线程可以使用同一个 CloseableHttpClient 实例同时执行多个请求，从而充分利用持久连接重用和连接池。

• 直接从内容流中使用响应内容并将其转换为更高级别的对象，而无需将其转换为中间字符串或字节数组。

Answer 3

原来requestRetry 的值大于0 在我们的生产环境中导致SocketException，所以我们设置了requestRetry = 0，但我们没有得到任何SocketException 异常