【问题标题】:Connecting https site responseHeader delays连接 https 站点 responseHeader 延迟
【发布时间】:2021-01-31 12:59:21
【问题描述】:

我制作了一个依赖于最新消息的应用,其中一部分需要带来特定国家/地区的新闻,以便在网站上发布一些最新的新闻文章。我选择了 world.einnews.com,因为他们有非常好的特定国家/地区的布局,他们还提供 RSS,但是您必须在他们生成 rss 链接的网站上订阅每个国家/地区。 我走捷径,直接去他们的网站,阅读我需要的东西。几周前还好,现在我开始看到延迟收到该网站的回复,就在我决定上线之前。

下面是我的 okhttp 事件监听器日志,你可以看到 responseHeader 启动时总是出现延迟

0.000 callStart
0.001 connectionAcquired
0.001 requestHeadersStart
0.002 requestHeadersEnd
0.002 responseHeadersStart
86.383 responseHeadersEnd
86.799 responseBodyStart
87.817 responseBodyEnd
87.977 connectionReleased
87.998 callEnd

现在这就是我的情况,所以当尝试 SSL 握手时似乎发生了一些事情我尝试 curl 并得到了这个输出:

 curl -G https://world.einnews.com/country/burkinafaso -v -e www.google.com -k
* About to connect() to world.einnews.com port 443 (#0)
* Trying 173.203.107.183...
* Connected to world.einnews.com (173.203.107.183) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
* Server certificate:
*   subject: CN=*.einnews.com
*   start date: Feb 17 00:00:00 2020 GMT
*   expire date: Mar 18 12:00:00 2021 GMT
*   common name: *.einnews.com
*   issuer: CN=Thawte RSA CA 2018,OU=www.digicert.com,O=DigiCert Inc,C=US
> GET /country/burkinafaso HTTP/1.1
> User-Agent: curl/7.29.0
> Host: world.einnews.com
> Accept: */*
> Referer: www.google.com
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Fri, 16 Oct 2020 19:09:54 GMT
< Content-Type: text/html
< Content-Length: 209078
< Connection: keep-alive
< Keep-Alive: timeout=10
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Status: 200 OK
< Content-Security-Policy-Report-Only: default-src https: data: 'unsafe-eval' 'unsafe-inline'; 
 report-uri /csp_reports
< Expires: Thu, 15 Oct 2020 19:09:52 GMT
< Cache-Control: no-cache
< Set-Cookie: ein_visit=1602875392%3AjrJa9zNl7kWh1O4JiDEY894a2IJ- 
 eRnsMhow4fJAeHyPzerWcCrYBn7xgrN82ktxe90__N-BL73l4y4w_UJVj-ycYbubg- 
 Ew9wAin5pFxdycFXl9lwDEqb4KVYtlq_lEhCvW2JDWoIsV9SxNXVsWOsfrrbZSDL8rtyGjAmVnlPV1TI- 
 XovsUf722O06qIPqJcgtupdjxgHzptJ3t0sCtcBw39Vd2EkxTxVzxhNK0o- 
< Set-Cookie: ein_ags=gLLGYrJqzY4BCiOT%3A1602875392%3A1602875392; domain=einnews.com; path=/; 
expires=Sat, 17-Oct-2020 13:09:52 GMT
< Set-Cookie: 
prev_nav_1=qZvmCmbfK7v3yxTY+1602875392+worldnews+%2Fcountry%2Fburkinafaso+Burkina+Faso+Newswire; 
domain=einnews.com; path=/; expires=Sat, 17-Oct-2020 19:09:52 GMT
< X-Frame-Options: SAMEORIGIN
< Etag: "02c6c59d2763f21d92d95d686866a473"
< Set-Cookie: uid=AgAAAALwiV/RLE+xAYMFRw==; expires=Thu, 31-Dec-37 23:55:55 GMT; domain=.einnews.com; 
path=/
< 

即使 curl 也需要很长时间,我真的束手无策。我在拨打电话之前尝试信任所有 ssl,我还尝试使用不同的用户代理和其他参数,但我无法弄清楚为什么网站响应缓慢。

我只需要点击一次并将结果缓存一天,这样我就可以避免网络故障,最初还可以,但现在我有很长的延迟,但有时会更快。如果我通过浏览器正常访问该站点,站点速度很快,只有无头浏览是一种痛苦。

有什么提示和技巧可以加快这个速度吗?

【问题讨论】:

  • 您在浏览器中的测量结果如何?您甚至可以在浏览器请求中用眼睛检测到 86 毫秒吗?听起来该站点只需要 86 毫秒即可生成响应。在大多数 HTTP Web 框架中,您需要先生成正文,然后才能设置响应(200,响应标头 + 正文)。
  • 那不是毫秒那秒,请检查okhttp listener event
  • 没有足够的信息来继续这个问题。没有重现该问题的示例代码。当您从单个服务器对它进行大量并发调用时,它可能只是一个过载的服务器。

标签: performance ssl web-crawler okhttp


【解决方案1】:

我没有看到那种延迟。您可以启用更多调试来查看 HTTP/2 帧,甚至 OkHttp 中的线程池活动。这是一个测试客户端,它向我显示了相同的请求。

可能有趣的 java.util 记录器是

好的http3.internal.http2.Http2 okhttp3.internal.concurrent.TaskRunner

$  okurl --debug https://world.einnews.com/country/burkinafaso
13:50:48.596    url https://world.einnews.com/country/burkinafaso
13:50:48.598    Request Request{method=GET, url=https://world.einnews.com/country/burkinafaso, headers=[User-Agent:okurl/dev OkHttp/4.10.0-RC1], tags={class com.baulsupp.okurl.credentials.Token=TokenSet(name=default)}}
13:50:48.634    Dns (world.einnews.com): world.einnews.com/173.203.107.183, world.einnews.com/173.203.107.239
13:50:49.126    Q10002 scheduled after   5 s : OkHttp world.einnews.com ping
13:50:49.127    >> CONNECTION 505249202a20485454502f322e300d0a0d0a534d0d0a0d0a
13:50:49.127    >> 0x00000000     6 SETTINGS
13:50:49.127    >> 0x00000000     4 WINDOW_UPDATE
13:50:49.127    Q10005 scheduled after   0 Ás: OkHttp world.einnews.com
13:50:49.128    Q10005 starting              : OkHttp world.einnews.com
13:50:49.128    Q10001 scheduled after   0 Ás: OkHttp ConnectionPool
13:50:49.128    --> GET https://world.einnews.com/country/burkinafaso h2
13:50:49.128    User-Agent: okurl/dev OkHttp/4.10.0-RC1
13:50:49.128    Accept-Encoding: br,gzip
13:50:49.128    Host: world.einnews.com
13:50:49.128    Connection: Keep-Alive
13:50:49.128    --> END GET
13:50:49.128    Q10001 starting              : OkHttp ConnectionPool
13:50:49.128    << 0x00000000    18 SETTINGS
13:50:49.128    Q10001 run again after 300 s : OkHttp ConnectionPool
13:50:49.128    Q10001 finished run in 534 Ás: OkHttp ConnectionPool
13:50:49.128    Q10002 scheduled after   0 Ás: OkHttp world.einnews.com applyAndAckSettings
13:50:49.129    << 0x00000000     4 WINDOW_UPDATE
13:50:49.129    Q10002 starting              : OkHttp world.einnews.com applyAndAckSettings
13:50:49.129    Q10004 scheduled after   0 Ás: OkHttp world.einnews.com onSettings
13:50:49.129    >> 0x00000000     0 SETTINGS      ACK
13:50:49.129    Matching interceptor: null
13:50:49.129    Q10004 starting              : OkHttp world.einnews.com onSettings
13:50:49.129    Q10002 finished run in 453 Ás: OkHttp world.einnews.com applyAndAckSettings
13:50:49.129    Q10004 finished run in 262 Ás: OkHttp world.einnews.com onSettings
13:50:49.129    >> 0x00000003    65 HEADERS       END_STREAM|END_HEADERS
13:50:49.379    << 0x00000000     0 SETTINGS      ACK
13:50:52.142    << 0x00000003   955 HEADERS       END_HEADERS
13:50:52.142    << 0x00000003  8192 DATA
13:50:52.142    <-- 200 https://world.einnews.com/country/burkinafaso (3014ms)
13:50:52.142    << 0x00000003  8192 DATA
13:50:52.142    server: nginx
13:50:52.142    date: Sun, 18 Oct 2020 12:50:52 GMT
13:50:52.143    content-type: text/html
13:50:52.143    vary: Accept-Encoding
13:50:52.143    vary: Accept-Encoding
13:50:52.143    status: 200 OK
13:50:52.143    content-security-policy-report-only: default-src https: data: 'unsafe-eval' 'unsafe-inline'; report-uri /csp_reports
13:50:52.143    expires: Sat, 17 Oct 2020 12:50:51 GMT
13:50:52.143    cache-control: no-cache
13:50:52.143    set-cookie: ein_ags=k4mN99M2gOHvl8kp%3A1603025451%3A1603025451; domain=einnews.com; path=/; expires=Mon, 19-Oct-2020 06:50:51 GMT
13:50:52.143    set-cookie: ein_visit=1603025451%3AngzpmQnb9_71Sm9SVQL5aW9oh0fr9UUdqU9fGmUNQR0TedOpAZyAzRLiqmE_sF4XkU7esQ4PAojbnsynVCVtoOwLkcoXNnBjq95q7zmbCi9mJbNhdS6x_phE2tcjtrS1FVZI1O1cyc1_zlZy8twjRvAAUm1BSp4MHpbzlzEu6ROySFESHRTlimi560THEByAnCVrWoeTgV_QloTZZgV29Azjr-qEeBpGiF35_2TihnBNTb6Ptddt_X3nYmknHbv27XKKB8UGVs3iDwFL6_oX7NnO-smkTeMdvdlnGMLesQOCc9d6uoiwmM613UPHi79gwW4HJ2_NCxneE244536FZw%3D%3D; domain=einnews.com; path=/; expires=Fri, 17-Oct-2025 12:50:51 GMT
13:50:52.143    set-cookie: prev_nav_1=yXmftb5d13-tVPZa+1603025451+worldnews+%2Fcountry%2Fburkinafaso+Burkina+Faso+Newswire; domain=einnews.com; path=/; expires=Mon, 19-Oct-2020 12:50:51 GMT
13:50:52.144    x-frame-options: SAMEORIGIN
13:50:52.144    etag: W/"b6c4cf414dc0fc83bf1b1cb2ba803dd5"
13:50:52.144    set-cookie: uid=BgAAACw6jF9vKk55AQJrLQ==; expires=Thu, 31-Dec-37 23:55:55 GMT; domain=.einnews.com; path=/
13:50:52.144    content-encoding: gzip
13:50:52.144    <-- END HTTP
13:50:52.144    OkHttp Platform: Jdk9Platform
13:50:52.144    Protocol: h2
13:50:52.144    TLS Version: TLS_1_2
13:50:52.144    Cipher: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
13:50:52.145    Peer Principal: CN=*.einnews.com
13:50:52.145    Local Principal: none
13:50:52.145    JVM: GraalVM 20.2.0 Java 11
<!DOCTYPE html>
<html lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml">

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2023-03-03
    相关资源
    最近更新 更多