google cloud gsutil cp ResumableUploadAbortException: 408 Upload Client Broken Connection 错误

Question

我正在使用 gsutil cp 将一个大文件（大约 40g）从本地驱动程序移动到 gcs 存储桶。它以前运行良好，不知何故现在我不断收到“ResumableUploadAbortException: 408 Upload Client Broken Connection”。我尝试使用 gsutil cp 来移动一个小文件，它工作正常。

我在命令中加了一个-D：gsutil -D cp，这是我得到的跟踪：

DEBUG: Exception stack trace:iB]   80.0 MiB/s                                   
    Traceback (most recent call last):
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1592, in _PerformResumableUpload
        additional_headers=addl_headers)
      File "/home/test/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 1013, in StreamMedia
        additional_headers=additional_headers, use_chunks=False)
      File "/home/test/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/transfer.py", line 968, in __StreamMedia
        raise exceptions.HttpError.FromResponse(response)
    apitools.base.py.exceptions.HttpError: HttpError accessing <https://storage.googleapis.com/resumable/upload/storage/v1/b/charleszhan-dss-playground-bucket/o?alt=json&fields=etag%2Cgeneration%2Cmd5Hash%2Csize%2CcustomerEncryption%2Ccrc32c&uploadType=resumable&upload_id=AAANsUlrnIDRqoTNIeFM87J5TpvSOiKlxr-kfWMkYG0sO1qocki09C773h-z1V_JPAJ_jFGMyk-3fi7z35uf8-_ZsWBE8svx7w>: response: <{'x-guploader-uploadid': 'AAANsUlrnIDRqoTNIeFM87J5TpvSOiKlxr-kfWMkYG0sO1qocki09C773h-z1V_JPAJ_jFGMyk-3fi7z35uf8-_ZsWBE8svx7w', 'content-type': 'application/json; charset=UTF-8', 'date': 'Fri, 07 Aug 2020 15:28:34 GMT', 'vary': 'Origin, X-Origin', 'cache-control': 'no-cache, no-store, max-age=0, must-revalidate', 'expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'pragma': 'no-cache', 'content-length': '251', 'server': 'UploadServer', 'alt-svc': 'h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'status': '408'}>, content <{
      "error": {
        "code": 408,
        "message": "Upload Client Broken Connection",
        "errors": [
          {
            "message": "Upload Client Broken Connection",
            "domain": "global",
            "reason": "uploadBrokenConnection"
          }
        ]
      }
    }
    >
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions
        user_project=user_project)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
        return_code = command_inst.RunCommand()
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 1195, in RunCommand
        seek_ahead_iterator=seek_ahead_iterator)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1515, in Apply
        arg_checker, should_return_results, fail_on_error)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1586, in _SequentialApply
        worker_thread.PerformTask(task, self)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2306, in PerformTask
        results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 780, in _CopyFuncWrapper
        preserve_posix=cls.preserve_posix_attrs)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 995, in CopyFunc
        preserve_posix=preserve_posix)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3928, in PerformCopy
        gzip_encoded=gzip_encoded)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2220, in _UploadFileToObject
        parallel_composite_upload, logger)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2036, in _DelegateUploadFileToObject
        elapsed_time, uploaded_object = upload_delegate()
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2209, in CallResumableUpload
        gzip_encoded=gzip_encoded_file)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1837, in _UploadFileToObjectResumable
        gzip_encoded=gzip_encoded)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 421, in UploadObjectResumable
        gzip_encoded=gzip_encoded)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1734, in UploadObjectResumable
        gzip_encoded=gzip_encoded)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1517, in _UploadObject
        additional_headers, progress_callback, gzip_encoded)
      File "/home/test/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1655, in _PerformResumableUpload
        raise resumable_ex
    gslib.cloud_api.ResumableUploadAbortException: ResumableUploadAbortException: 408 Upload Client Broken Connection

Answer 1

gsutil 会在您使用 cp 命令上传大于 8 MiB 的对象时自动执行可恢复上传。您不需要指定任何特殊的命令行选项来实现这一点。如果您的上传被中断，您可以通过运行与开始上传相同的 cp 命令重新开始上传。在上传成功完成之前，它在目标对象中不可见，并且不会替换上传要覆盖的任何现有对象。