通过 Python 和请求调用 URL 时的 SSL 超时

【问题标题】：SSL timeout when calling an URL via Python and requests通过 Python 和请求调用 URL 时的 SSL 超时
【发布时间】：2016-06-09 16:37:21
【问题描述】：

我遇到了以下情况：当从 Python 和 requests 调用 HTTPS 链接时，我的 Ubuntu 笔记本电脑以及我在 VirtualBox 和另一台笔记本电脑中尝试过的 Linux 上出现超时。但是，相同的 Python 代码在 Windows 7、Mac OS X 和最新的 FreeBSD 上返回有效响应。这个问题大约在两周前出现，我怀疑这可能是因为服务器上的某些东西发生了变化，这只影响了 Linux 网络堆栈。欢迎任何想法为什么它可能会发生以及如何解决它。

以下是详细信息。

  • 我用过的 Python 版本：最新，2.7.11
  • requests 版本：2.10
  • 我尝试过的操作系统：Ubuntu 16.04、Ubuntu 15.10、Fedora 23、CentOS 6、Windows 7、FreeBSD、Mac OS X。

注意：以下所有重要数据均因保密协议而被跳过。

实际失败的 Python 代码：

import requests
import logging
import httplib as http_client
import ssl


http_client.HTTPConnection.debuglevel = 1

logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

url = 'https://subdomain.example1.com/products/12345'
cert = ('example.pem', 'example.key')
response = requests.get(url=url,
                        params={'locale': 'en_US'},
                        headers={'Content-Type': 'application/x-www-form-urlencoded', 'Accept': 'application/json'},
                        timeout=10.0,
                        cert=cert,
                        verify=False,
                        cookies=None)
print response

以上代码在 Linux 上运行的结果：

$ python req.py 
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): subdomain.example1.com
Traceback (most recent call last):
  File "req.py", line 23, in <module>
    cookies=None)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 449, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='subdomain.example1.com', port=443): Read timed out. (read timeout=10.0)

Linux 上相同代码的结果，超时设置为 300.0：

$ python req.py 
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): subdomain.example1.com
Traceback (most recent call last):
  File "req.py", line 23, in <module>
    cookies=None)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 447, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_read_bytes', 'sslv3 alert unexpected message')],)",)

来自 FreeBSD 的正确 Python 代码运行结果：

$ python req.py  
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): subdomain.example1.com
/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
send: 'GET /products/12345?locale=en_US HTTP/1.1\r\nHost: subdomain.example1.com\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: application/json\r\nUser-Agent: python-requests/2.10.0\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: Apache-Coyote/1.1
header: Cache-Control: private
header: Expires: Wed, 31 Dec 1969 17:00:00 MST
header: Content-Type: application/json;charset=UTF-8
header: Transfer-Encoding: chunked
header: Date: Thu, 09 Jun 2016 16:13:37 GMT
DEBUG:requests.packages.urllib3.connectionpool:"GET /products/12345?locale=en_US HTTP/1.1" 200 None
<Response [200]>

来自openssl的尝试：

$ openssl s_client -connect subdomain.example1.com:443 -cert example.pem -key example.key
CONNECTED(00000003)
depth=2 CN = Root.Example2.com
verify return:1
depth=1 DC = com, DC = example3, CN = ca-example3
verify return:1
depth=0 ST = Colorado, L = Boulder, O = Example4.com, OU = WebOps, CN = *.example1.com
verify return:1
139622025893528:error:140943F2:SSL routines:ssl3_read_bytes:sslv3 alert unexpected message:s3_pkt.c:1472:SSL alert number 10
139622025893528:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
Certificate chain
 0 s:/ST=Colorado/L=Boulder/O=Example4.com/OU=WebOps/CN=*.example1.com
   i:/DC=com/DC=example3/CN=ca-example3
---
Server certificate
-----BEGIN CERTIFICATE-----
........ SKIPPED .........                                                                                                                                                                                                              
-----END CERTIFICATE-----                                                                                                                                                                                                                                                      
subject=/ST=Colorado/L=Boulder/O=Example4.com/OU=WebOps/CN=*.example1.com                                                                                                                                                                                                          
issuer=/DC=com/DC=example3/CN=ca-example3                                                                                                                                                                                                                            
---                                                                                                                                                                                                                                                                            
Acceptable client certificate CA names                                                                                                                                                                                                                                         
/O=Example4.com/OU=WebOps/CN=cl_ABCDEFGHI                                                                                                                                                                                                                                      
/O=Example4.com/OU=WebOps/CN=cl_QWERTYUIO                                                                                                                                                                                                                                      
/CN=Root.Example2.com
/DC=com/DC=example3/CN=ca-example3
/O=Example4.com/OU=WebOps/CN=cl_ASDFGHJKL
/O=Example4.com/OU=WebOps/CN=cl_ZXCVBNabc
/O=Example4.com/OU=WebOps/CN=cl_poiuytrew
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:ECDSA+SHA224:RSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1:RSA+MD5
Shared Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:ECDSA+SHA224:RSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 2573 bytes and written 4570 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: 57591435C56B9FDFXXXBB896F8140E7B72EE07A2ACDB0446F0AB04C5D2928FD1
    Session-ID-ctx: 
    Master-Key: F830C1F0330391B5XXX3E58F07C23D7A8C2020F824C0C4389D798CCA16A6AD5D556881671001A7DD6641AC4739BE2E28
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1465455669
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

主机似乎有效，我可以 ping 它，来自任何操作系统的openssl 调用，包括 Linux 框显示它是活动的并响应请求。

提前致谢。

【问题讨论】：

  • 如果我理解正确，openssl s_client 适用于所有机器，但 python/ssl 不能？您确定您也在 FreeBSD 上使用 Python 2.7.11，因为有关 InsecureRequest... 的警告表明与在 Linux 上运行的版本有所不同。如果您将 SNI 与 -servername 参数一起使用，s_client 是否仍然有效？
  • openssl s_client 在所有机器上都能正常工作，是的。而 Python/requests 仅适用于非 Linux。
  • 是的，它是 FreeBSD 上的 Python 2.7.11，刚刚检查过。
  • 试过-servername，看起来它并没有改变任何东西。 openssl 的输出和之前一样。
  • 很奇怪。然后我建议查看网络层（即数据包捕获），看看它挂在哪里。理想情况下，捕获将在客户端和服务器端完成，以防某些网络设备导致问题。也可以尝试降低 MTU、关闭 ECN 或类似的功能。另见squidproxy.wordpress.com/2007/06/05/…

标签： python ssl https openssl python-requests


【解决方案1】：

问题似乎是由于跨多个版本使用 nat 网络时 virtualbox 中的错误引起的。根据this issue，该错误已在版本 5.1.10 中修复。但是我在 Windows 主机上的 Virtual Box 5.1.12 中遇到了同样的问题。

【讨论】：

  • 很高兴听到他们已修复它。同时我们已经切换到 Docker，所以又一个问题消失了。感谢提及！
猜你喜欢
  • 2016-11-19
  • 1970-01-01
  • 1970-01-01
  • 2018-04-25
  • 2022-01-04
  • 2018-03-25
  • 2011-02-26
  • 1970-01-01
  • 2016-07-10
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode