google.api_core.exceptions.ServiceUnavailable：超过 503 截止日期答案

【问题标题】：google.api_core.exceptions.ServiceUnavailable: 503 Deadline Exceededgoogle.api_core.exceptions.ServiceUnavailable：超过 503 截止日期
【发布时间】：2020-06-05 02:55:58
【问题描述】：

google.api_core.exceptions.ServiceUnavailable：超过 503 截止日期
使用 python 3.7 ，google-cloud-pubsub ==1.1.0 发布主题数据。在我的本地机器上，它工作得非常好，能够发布关于该主题的数据，还能够通过订阅者从该主题中提取数据。但不明白当我在服务器上部署代码时它不起作用并且它因 INLINE ERROR 而失败但是当我在服务器上显式调用发布者方法时它在服务器框上也发布正常。失败的代码发布时在下面一行：

future = publisher.publish(topic_path, data=data)

**ERROR:2020-02-20 14:24:42,714 ERROR Failed to publish 1 messages.**
Trackback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 826, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAVAILABLE
        details = "Deadline Exceeded"
        debug_error_string = "{"created":"@1582208682.711481693","description":"Deadline Exceeded","file":"src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":14}"
The above exception was the direct cause of the following exception: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/google/api_core/retry.py", line 184, in retry_target
    return target()
  File "/usr/local/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.ServiceUnavailable: 503 Deadline Exceeded

上述异常是以下异常的直接原因：

Traceback（最近一次调用最后一次）： _commit 中的文件“/usr/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/publisher/_batch/thread.py”，第 219 行响应 = self._client.api.publish(self._topic, self._messages) 发布中的文件“/usr/local/lib/python3.7/site-packages/google/cloud/pubsub_v1/gapic/publisher_client.py”，第 498 行请求，重试=重试，超时=超时，元数据=元数据调用中的文件“/usr/local/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py”，第 143 行返回 Wrapped_func(*args, **kwargs) retry_wrapped_func 中的文件“/usr/local/lib/python3.7/site-packages/google/api_core/retry.py”，第 286 行 on_error=on_error, retry_target 中的文件“/usr/local/lib/python3.7/site-packages/google/api_core/retry.py”，第 206 行 last_exc, 文件“”，第 3 行，在 raise_from google.api_core.exceptions.RetryError: 调用 functools.partial(.error_remapped_callable at 0x7f67d064e950>

时超过了 60.0s 的最后期限

【问题讨论】：

这看起来像是权限或网络问题。你把它部署到哪里了？您部署的服务器是否具有正确的网络或 IAM 设置？
我在下面发布了一个可能的解决方案，您能否再发布一点您的代码，以便我可以准确地看到会发生什么？

标签： publish-subscribe google-cloud-pubsub google-publisher-tag

【解决方案1】：

您应该尝试将数据分块成合理大小的块 (max_messages)，并且不要忘记添加 done 回调。

# Loop over json containing records/rows
for idx, row in enumerate(rows_json):
    publish_json(row, idx, rowmax=len(rows_json), topic_name)

# Publish messages asynchronous 
def publish_json(msg, rowcount, rowmax, topic_project_id, topic_name):
    batch_settings = pubsub_v1.types.BatchSettings(max_messages=100)
    publisher = pubsub_v1.PublisherClient(batch_settings)
    topic_path = publisher.topic_path(topic_project_id, topic_name)
    future = publisher.publish(
        topic_path, bytes(json.dumps(msg).encode('utf-8')))
    future.add_done_callback(
        lambda x: logging.info(
            'Published msg with ID {} ({}/{} rows).'.format(
                future.result(), rowcount, rowmax))
    )

【讨论】：