Python3.6.8 urllib.request.urlopen 错误“连接由对等方重置”

Question

我正在尝试使用 urllib.request.urlopen 从 maps.googleapis.com 获取地理编码信息。

我遇到了一个奇怪的问题，即 urllib.request.urlopen 在启用代理服务器的情况下工作，但在没有代理的情况下无法工作。

我在 ssl hadnshake 期间收到连接重置错误，

Traceback (most recent call last):
  File "/usr/lib64/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/lib64/python3.6/http/client.py", line 1415, in connect
    server_hostname=server_hostname)
  File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib64/python3.6/ssl.py", line 773, in __init__
    self.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake
    self._sslobj.do_handshake()
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/usr/lib64/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/usr/lib64/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib64/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 104] Connection reset by peer>

我遇到了 requests 模块的类似问题，但是当我如下更新它时，我能够使用 requests.get 方法获得响应，

pip install requests[security]

我与我们的网络安全团队核实，maps.googleapis.com 的所有 IP 都已列入白名单。

我检查了 cURL 命令，它的工作也很好。

有人知道 urllib.request.urlopen 有什么问题吗？

我使用的是python 3.6.8版本。

Answer 1

1) 自动机器人检测机制很可能会断开您的连接。您应该提供一个 User-Agent 标头来伪造浏览器访问。

headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36'})

1. 在安装请求库时，它会跳过一些 SSL/Https 连接所需的可选安全包（“pyOpenSSL”、“ndg-httpsclient”和“pyasn1”）。您可以通过运行此命令来修复它

   pip install "requests[security]"
   

或

pip install pyopenssl ndg-httpsclient pyasn1

如果它不起作用

1. 您可以（有时）通过策略性地放置 time.sleep(0.01) 来纠正此问题。

参考这个https://stackoverflow.com/a/383816/13126651