HTTPError: 429 Client Error: Too Many Requests on IBM Cloudant with python

Question

IBM Cloudant 已连接到 Jupyter 笔记本。我可以使用不是很大的数据库。但是当我想从有很多数据的数据库中获取数据时，我得到了这个错误：

HTTPError: 429 Client Error: Too Many Requests too_many_requests 您已经超出了当前查询类每秒 5 个请求的限制。请稍后再试

这个问题有什么办法吗？ 任何想法都会很有帮助。

这是我的代码：

from cloudant.client import Cloudant
from cloudant.error import CloudantException
from cloudant.result import Result, ResultByKey
import cloudant
import pandas as pd
import json

serviceUsername = "x"
servicePassword = "y"
serviceURL = "z"
client = Cloudant(serviceUsername, servicePassword, url=serviceURL)
client.connect()

database1 = client['comfort']
userID= input("what is your user ID?")

for item in database1:
  if userID == 'user_id':
    print(item)

这就是我得到的

what is your user ID?ss


> HTTPError                                 Traceback (most recent call
> last)
>     <ipython-input-7-98e9b2d333f5> in <module>()
>           4 userID= input("what is your user ID?")
>           5 
>     ----> 6 for item in database1:
>           7     if userID == 'user_id':
>           8         print(item)
>     
>     ~\Anaconda3\lib\site-packages\cloudant\database.py in __iter__(self, remote)
>         634                                                   # next_startkey
>         635                     include_docs=True,
>     --> 636                     startkey=next_startkey
>         637                 ).get('rows', [])
>         638 
>     
>     ~\Anaconda3\lib\site-packages\cloudant\database.py in all_docs(self, **kwargs)
>         391                         '/'.join([self.database_url, '_all_docs']),
>         392                         self.client.encoder,
>     --> 393                         **kwargs)
>         394         return resp.json()
>         395 
>     
>     ~\Anaconda3\lib\site-packages\cloudant\_common_util.py in get_docs(r_session, url, encoder, headers, **params)
>         263     else:
>         264         resp = r_session.get(url, headers=headers, params=f_params)
>     --> 265     resp.raise_for_status()
>         266     return resp
>         267 
>     
>     ~\Anaconda3\lib\site-packages\requests\models.py in raise_for_status(self)
>         933 
>         934         if http_error_msg:
>     --> 935             raise HTTPError(http_error_msg, response=self)
>         936 
>         937     def close(self):
>     
>     HTTPError: 429 Client Error: Too Many Requests too_many_requests You've exceeded your current limit of 5 requests per second for query
> class. Please try later. for url:
> https://2b5a4b.......

Answer 1

尝试这样查询：

database1 = client['comfort']
userID= input("what is your user ID?")

selector = {'user_id': {'$eq': userID}}
docs = database1.get_query_result(selector)
for doc in docs:
    do stuff...

直接从the docs得到这个

Answer 2

因此，对此的实际确定答案是在连接中添加一个所谓的 Replay429Adapter。

这将负责捕获 HTTP429 错误并重试和调整连接速度，以减轻 Cloudant Lite 计划的每秒 5 次查询限制对 IBM Cloud 的直接影响。
这实际上已记录在案，一旦您知道要查找的内容：
https://python-cloudant.readthedocs.io/en/latest/getting_started.html?highlight=Replay429Adapter 您将需要添加： from cloudant.adapters import Replay429Adapter 然后，添加 ,adapter=Replay429Adapter(retries=10, initialBackoff=0.01)) 到您的 Cloudant 客户端连接器创建

另见https://github.com/cloudant/python-cloudant/issues/249

Answer 3

添加 Replay429Adapter 是解决此错误的正确方法，也是真正的弹性客户端。

我的客户端连接如下所示（使用 IAM 身份验证）：

from cloudant.client import Cloudant
from cloudant.adapters import Replay429Adapter
db_client = Cloudant.iam(cloudant_username, cloudant_apikey, connect=True, adapter=Replay429Adapter(retries=10, initialBackoff=0.01))