使用来自python的请求或urllib时出现“OSError：[Errno 0]错误”

Question

我想使用GTEx API。我最近（几周前）确实使用了requests，它运行良好，但由于我不知道的原因，它不再运行了。

例如，我想使用 API 中可用的 48 种不同组织检索信息。
以下是获取错误的最小可重现代码：

# Import libraries
import json
import urllib
import requests

# Set the paremeters  
gene = 'NOS1AP'
snp = 'rs10918593'
datasetId = 'gtex_v7'
tissues = ['Adipose_Subcutaneous', 'Adipose_Visceral_Omentum', 'Adrenal_Gland', 'Artery_Aorta', 'Artery_Coronary', 'Artery_Tibial', 'Brain_Amygdala', 'Brain_Anterior_cingulate_cortex_BA24', 'Brain_Caudate_basal_ganglia', 'Brain_Cerebellar_Hemisphere', 'Brain_Cerebellum', 'Brain_Cortex', 'Brain_Frontal_Cortex_BA9', 'Brain_Hippocampus', 'Brain_Hypothalamus', 'Brain_Nucleus_accumbens_basal_ganglia', 'Brain_Putamen_basal_ganglia', 'Brain_Spinal_cord_cervical_c-1', 'Brain_Substantia_nigra', 'Breast_Mammary_Tissue', 'Cells_EBV-transformed_lymphocytes', 'Cells_Transformed_fibroblasts', 'Colon_Sigmoid', 'Colon_Transverse', 'Esophagus_Gastroesophageal_Junction', 'Esophagus_Mucosa', 'Esophagus_Muscularis', 'Heart_Atrial_Appendage', 'Heart_Left_Ventricle', 'Liver', 'Lung', 'Minor_Salivary_Gland', 'Muscle_Skeletal', 'Nerve_Tibial', 'Ovary', 'Pancreas', 'Pituitary', 'Prostate', 'Skin_Not_Sun_Exposed_Suprapubic', 'Skin_Sun_Exposed_Lower_leg', 'Small_Intestine_Terminal_Ileum', 'Spleen', 'Stomach', 'Testis', 'Thyroid', 'Uterus', 'Vagina', 'Whole_Blood']

当我运行以下代码时，我收到一些初始组织的错误：

# using requests
for tissue in tissues:
    print(f'Getting eQTL for tissue {tissue}...')
    server = 'https://gtexportal.org/rest/v1/'
    ext = f'association/dyneqtl?gencodeId={gene}&variantId={snp}&tissueSiteDetailId={tissue}&datasetId={datasetId}'
    r = requests.get(server+ext, headers={"Accept" : "application/json"})
    if not r.ok:
        print(f'--- Request for SNP "{snp}" and "{gene}" returned an error! ---')
        continue
    decoded = r.json()
    r.close()

以上代码出错：

Getting eQTL for tissue Adipose_Subcutaneous...
Getting eQTL for tissue Adipose_Visceral_Omentum...
Getting eQTL for tissue Adrenal_Gland...
Getting eQTL for tissue Artery_Aorta...
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    669             # Make the request on the httplib connection object.
--> 670             httplib_response = self._make_request(
    671                 conn,

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    977         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 978             conn.connect()
    979 

~/miniconda3/lib/python3.8/site-packages/urllib3/connection.py in connect(self)
    361 
--> 362         self.sock = ssl_wrap_socket(
    363             sock=conn,

~/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data)
    385         if HAS_SNI and server_hostname is not None:
--> 386             return context.wrap_socket(sock, server_hostname=server_hostname)
    387 

~/miniconda3/lib/python3.8/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    499         # ctx._wrap_socket()
--> 500         return self.sslsocket_class._create(
    501             sock=sock,

~/miniconda3/lib/python3.8/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040                     self.do_handshake()
   1041             except (OSError, ValueError):

~/miniconda3/lib/python3.8/ssl.py in do_handshake(self, block)
   1308                 self.settimeout(None)
-> 1309             self._sslobj.do_handshake()
   1310         finally:

OSError: [Errno 0] Error

During handling of the above exception, another exception occurred:

ProtocolError                             Traceback (most recent call last)
~/miniconda3/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    438             if not chunked:
--> 439                 resp = conn.urlopen(
    440                     method=request.method,

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    725 
--> 726             retries = retries.increment(
    727                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]

~/miniconda3/lib/python3.8/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    409             if read is False or not self._is_method_retryable(method):
--> 410                 raise six.reraise(type(error), error, _stacktrace)
    411             elif read is not None:

~/miniconda3/lib/python3.8/site-packages/urllib3/packages/six.py in reraise(tp, value, tb)
    733             if value.__traceback__ is not tb:
--> 734                 raise value.with_traceback(tb)
    735             raise value

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    669             # Make the request on the httplib connection object.
--> 670             httplib_response = self._make_request(
    671                 conn,

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    380         try:
--> 381             self._validate_conn(conn)
    382         except (SocketTimeout, BaseSSLError) as e:

~/miniconda3/lib/python3.8/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    977         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 978             conn.connect()
    979 

~/miniconda3/lib/python3.8/site-packages/urllib3/connection.py in connect(self)
    361 
--> 362         self.sock = ssl_wrap_socket(
    363             sock=conn,

~/miniconda3/lib/python3.8/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password, ca_cert_data)
    385         if HAS_SNI and server_hostname is not None:
--> 386             return context.wrap_socket(sock, server_hostname=server_hostname)
    387 

~/miniconda3/lib/python3.8/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    499         # ctx._wrap_socket()
--> 500         return self.sslsocket_class._create(
    501             sock=sock,

~/miniconda3/lib/python3.8/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040                     self.do_handshake()
   1041             except (OSError, ValueError):

~/miniconda3/lib/python3.8/ssl.py in do_handshake(self, block)
   1308                 self.settimeout(None)
-> 1309             self._sslobj.do_handshake()
   1310         finally:

ProtocolError: ('Connection aborted.', OSError(0, 'Error'))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-6-7dc3e6722683> in <module>
      4     server = 'https://gtexportal.org/rest/v1/'
      5     ext = f'association/dyneqtl?gencodeId={gene}&variantId={snp}&tissueSiteDetailId={tissue}&datasetId={datasetId}'
----> 6     r = requests.get(server+ext, headers={"Accept" : "application/json"})
      7     if not r.ok:
      8         print(f'--- Request for SNP "{snp}" and "{gene}" returned an error! ---')

~/miniconda3/lib/python3.8/site-packages/requests/api.py in get(url, params, **kwargs)
     74 
     75     kwargs.setdefault('allow_redirects', True)
---> 76     return request('get', url, params=params, **kwargs)
     77 
     78 

~/miniconda3/lib/python3.8/site-packages/requests/api.py in request(method, url, **kwargs)
     59     # cases, and look like a memory leak in others.
     60     with sessions.Session() as session:
---> 61         return session.request(method=method, url=url, **kwargs)
     62 
     63 

~/miniconda3/lib/python3.8/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    528         }
    529         send_kwargs.update(settings)
--> 530         resp = self.send(prep, **send_kwargs)
    531 
    532         return resp

~/miniconda3/lib/python3.8/site-packages/requests/sessions.py in send(self, request, **kwargs)
    641 
    642         # Send the request
--> 643         r = adapter.send(request, **kwargs)
    644 
    645         # Total elapsed time of the request (approximately)

~/miniconda3/lib/python3.8/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    496 
    497         except (ProtocolError, socket.error) as err:
--> 498             raise ConnectionError(err, request=request)
    499 
    500         except MaxRetryError as e:

ConnectionError: ('Connection aborted.', OSError(0, 'Error'))

有趣的是，当我重新运行脚本时，它会在不同的地方出现错误。例如，上面的错误是使用组织“Artery_Aorta”发生的，但是当我再次运行时，它卡在了组织“Brain_Cerebellum”中。因此，它似乎不是特定于 URL 的。

我也尝试过使用 urllib：

# using urllib
for tissue in tissues:
    print(f'Getting eQTL for tissue {tissue}...')
    server = 'https://gtexportal.org/rest/v1/'
    ext = f'association/dyneqtl?gencodeId={gene}&variantId={snp}&tissueSiteDetailId={tissue}&datasetId={datasetId}'
    r = urllib.request.urlopen(server+ext)
    if r.code != 200:
        print(f'--- Request for SNP "{snp}" and "{gene}" returned an error! ---')
        continue
    decoded = json.load(r)
    r.close()

然后我得到：

Getting eQTL for tissue Adipose_Subcutaneous...
Getting eQTL for tissue Adipose_Visceral_Omentum...
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
~/miniconda3/lib/python3.8/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1318             try:
-> 1319                 h.request(req.get_method(), req.selector, req.data, headers,
   1320                           encode_chunked=req.has_header('Transfer-encoding'))

~/miniconda3/lib/python3.8/http/client.py in request(self, method, url, body, headers, encode_chunked)
   1229         """Send a complete request to the server."""
-> 1230         self._send_request(method, url, body, headers, encode_chunked)
   1231 

~/miniconda3/lib/python3.8/http/client.py in _send_request(self, method, url, body, headers, encode_chunked)
   1275             body = _encode(body, 'body')
-> 1276         self.endheaders(body, encode_chunked=encode_chunked)
   1277 

~/miniconda3/lib/python3.8/http/client.py in endheaders(self, message_body, encode_chunked)
   1224             raise CannotSendHeader()
-> 1225         self._send_output(message_body, encode_chunked=encode_chunked)
   1226 

~/miniconda3/lib/python3.8/http/client.py in _send_output(self, message_body, encode_chunked)
   1003         del self._buffer[:]
-> 1004         self.send(msg)
   1005 

~/miniconda3/lib/python3.8/http/client.py in send(self, data)
    943             if self.auto_open:
--> 944                 self.connect()
    945             else:

~/miniconda3/lib/python3.8/http/client.py in connect(self)
   1398 
-> 1399             self.sock = self._context.wrap_socket(self.sock,
   1400                                                   server_hostname=server_hostname)

~/miniconda3/lib/python3.8/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    499         # ctx._wrap_socket()
--> 500         return self.sslsocket_class._create(
    501             sock=sock,

~/miniconda3/lib/python3.8/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
   1039                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
-> 1040                     self.do_handshake()
   1041             except (OSError, ValueError):

~/miniconda3/lib/python3.8/ssl.py in do_handshake(self, block)
   1308                 self.settimeout(None)
-> 1309             self._sslobj.do_handshake()
   1310         finally:

OSError: [Errno 0] Error

During handling of the above exception, another exception occurred:

URLError                                  Traceback (most recent call last)
<ipython-input-8-d7e4dc39c80e> in <module>
      4     server = 'https://gtexportal.org/rest/v1/'
      5     ext = f'association/dyneqtl?gencodeId={gene}&variantId={snp}&tissueSiteDetailId={tissue}&datasetId={datasetId}'
----> 6     r = urllib.request.urlopen(server+ext)
      7     if r.code != 200:
      8         print(f'--- Request for SNP "{snp}" and "{gene}" returned an error! ---')

~/miniconda3/lib/python3.8/urllib/request.py in urlopen(url, data, timeout, cafile, capath, cadefault, context)
    220     else:
    221         opener = _opener
--> 222     return opener.open(url, data, timeout)
    223 
    224 def install_opener(opener):

~/miniconda3/lib/python3.8/urllib/request.py in open(self, fullurl, data, timeout)
    523 
    524         sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
--> 525         response = self._open(req, data)
    526 
    527         # post-process response

~/miniconda3/lib/python3.8/urllib/request.py in _open(self, req, data)
    540 
    541         protocol = req.type
--> 542         result = self._call_chain(self.handle_open, protocol, protocol +
    543                                   '_open', req)
    544         if result:

~/miniconda3/lib/python3.8/urllib/request.py in _call_chain(self, chain, kind, meth_name, *args)
    500         for handler in handlers:
    501             func = getattr(handler, meth_name)
--> 502             result = func(*args)
    503             if result is not None:
    504                 return result

~/miniconda3/lib/python3.8/urllib/request.py in https_open(self, req)
   1360 
   1361         def https_open(self, req):
-> 1362             return self.do_open(http.client.HTTPSConnection, req,
   1363                 context=self._context, check_hostname=self._check_hostname)
   1364 

~/miniconda3/lib/python3.8/urllib/request.py in do_open(self, http_class, req, **http_conn_args)
   1320                           encode_chunked=req.has_header('Transfer-encoding'))
   1321             except OSError as err: # timeout error
-> 1322                 raise URLError(err)
   1323             r = h.getresponse()
   1324         except:

URLError: <urlopen error [Errno 0] Error>

同样的非 URL 特性也适用于此...

有趣的是，当使用同一台机器和同一互联网时，我可以使用 R 成功运行此（实际上，我的完整代码具有更多基因和 SNP）：

library(glue)
library(httr)
library(jsonlite)

gene = "NOS1AP" 
snp = "rs10918593"
datasetId='gtex_v7'
tissues <- c('Adipose_Subcutaneous', 'Adipose_Visceral_Omentum', 'Adrenal_Gland', 'Artery_Aorta', 'Artery_Coronary', 'Artery_Tibial', 'Brain_Amygdala', 'Brain_Anterior_cingulate_cortex_BA24', 'Brain_Caudate_basal_ganglia', 'Brain_Cerebellar_Hemisphere', 'Brain_Cerebellum', 'Brain_Cortex', 'Brain_Frontal_Cortex_BA9', 'Brain_Hippocampus', 'Brain_Hypothalamus', 'Brain_Nucleus_accumbens_basal_ganglia', 'Brain_Putamen_basal_ganglia', 'Brain_Spinal_cord_cervical_c-1', 'Brain_Substantia_nigra', 'Breast_Mammary_Tissue', 'Cells_EBV-transformed_lymphocytes', 'Cells_Transformed_fibroblasts', 'Colon_Sigmoid', 'Colon_Transverse', 'Esophagus_Gastroesophageal_Junction', 'Esophagus_Mucosa', 'Esophagus_Muscularis', 'Heart_Atrial_Appendage', 'Heart_Left_Ventricle', 'Liver', 'Lung', 'Minor_Salivary_Gland', 'Muscle_Skeletal', 'Nerve_Tibial', 'Ovary', 'Pancreas', 'Pituitary', 'Prostate', 'Skin_Not_Sun_Exposed_Suprapubic', 'Skin_Sun_Exposed_Lower_leg', 'Small_Intestine_Terminal_Ileum', 'Spleen', 'Stomach', 'Testis', 'Thyroid', 'Uterus', 'Vagina', 'Whole_Blood')

for(tissue in tissues) {
  print(glue('Getting eQTL for tissue {tissue}...'))
  server = 'https://gtexportal.org/rest/v1/'
  ext = glue('association/dyneqtl?gencodeId={gene}&variantId={snp}&tissueSiteDetailId={tissue}&datasetId={datasetId}')
  r <- GET(paste0(server, ext), content_type("application/json"))
  if(r$status_code!=200){
    print(glue('--- Request for SNP "{snp}" and "{gene}" returned an error! ---'))
    next
  }
  decoded <- fromJSON(toJSON(content(r)))
}

虽然它可以在 R 中运行，但我已经用 Python 组织了一个脚本，我真的很想正常使用 requests 和 urllib...

还有，我使用的版本：

python==3.8.2
requests==2.24.0

有什么帮助吗？

Answer 1

我最近遇到了这个问题，在我们开始一些不浪费时间的调试之前，我们需要解决一些问题。

首先，requests 2.24 需要urllib3 < 1.26，而在urllib3 1.25 中，wrap_socket 调用不是来自SSLContext；在进行SSL 握手时，这似乎会导致问题；在urllib3 1.26 中，团队已将其更改为context.wrap_socket，这似乎解决了问题。

其次，请记住在升级urllib3 后，也要将requests 更新到最新版本。

然后升级或安装最新的httplib2，它默默地给我带来了问题，直到我发现有人在 SO 上喊出来。

毕竟，导致Errno 0 error 的主要问题之一是不稳定的 TCP 连接；这在大多数情况下是我们无法控制的。