【问题标题】:gsutil times out on every commandgsutil 在每个命令上都会超时
【发布时间】:2019-06-19 18:19:20
【问题描述】:

我在项目中设置了一个存储桶来存储上传的文件。试图确保我可以使用将文件上传到失败的存储桶的测试节点程序写入存储桶。然后我尝试使用 gsutil 将文件复制到存储桶并重试直到超时。经过进一步调查,使用 gsutil 的每个命令都会导致超时错误。互联网上似乎很少有关于此错误的信息。

我已经使用gcloud init 删除并重新生成了 .boto 文件,此命令有效。我还两次干净地重新安装了sdk,但仍然存在问题。项目的 ACL 和存储桶包含我。下面是来自gsutil cp test.txt gs://mybucket的node程序和超时错误

const {Storage} = require('@google-cloud/storage');

// Creates a client
const storage = new Storage();

/**
 * TODO(developer): Uncomment the following lines before running the sample.
 */
const bucketName = 'mybucket';
const filename = 'test.txt';

// Uploads a local file to the bucket
storage.bucket(bucketName).upload(filename, {
  // Support for HTTP requests made with `Accept-Encoding: gzip`
  gzip: true,
  metadata: {
    // Enable long-lived HTTP caching headers
    // Use only if the contents of the file will never change
    // (If the contents will change, use cacheControl: 'no-cache')
    cacheControl: 'public, max-age=31536000',
  },
});

console.log(`${filename} uploaded to ${bucketName}.`);

节点程序出错

(node:18471) UnhandledPromiseRejectionWarning: Error: Anonymous caller does not have storage.objects.create access to medchainvault/test.txt.
    at Util.parseHttpRespBody (/home/russell/go/src/github.com/medchain/business-network/storage-server/node_modules/@google-cloud/common/build/src/util.js:172:38)
    at Util.handleResp (/home/russell/go/src/github.com/medchain/business-network/storage-server/node_modules/@google-cloud/common/build/src/util.js:116:117)
    at retryRequest (/home/russell/go/src/github.com/medchain/business-network/storage-server/node_modules/@google-cloud/common/build/src/util.js:403:22)
    at onResponse (/home/russell/go/src/github.com/medchain/business-network/storage-server/node_modules/@google-cloud/common/node_modules/retry-request/index.js:200:7)
    at /home/russell/go/src/github.com/medchain/business-network/storage-server/node_modules/teeny-request/build/src/index.js:158:17
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:189:7)
(node:18471) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 3)
(node:18471) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

来自 gsutil 的错误:


Traceback (most recent call last):    0.0 B/s                                   
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil", line 22, in <module>
    gsutil.RunMain()
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gsutil.py", line 117, in RunMain
    sys.exit(gslib.__main__.main())
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 407, in main
    perf_trace_token=perf_trace_token, user_project=user_project)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 590, in _RunNamedCommandAndHandleExceptions
    user_project=user_project)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 372, in RunNamedCommand
    return_code = command_inst.RunCommand()
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 1184, in RunCommand
    seek_ahead_iterator=seek_ahead_iterator)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1420, in Apply
    arg_checker, should_return_results, fail_on_error)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1491, in _SequentialApply
    worker_thread.PerformTask(task, self)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/command.py", line 2157, in PerformTask
    results = task.func(cls, task.args, thread_state=self.thread_gsutil_api)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 795, in _CopyFuncWrapper
    preserve_posix=cls.preserve_posix_attrs)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 992, in CopyFunc
    gzip_exts=self.gzip_exts, preserve_posix=preserve_posix))
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 3520, in PerformCopy
    allow_splitting=allow_splitting, gzip_encoded=gzip_encoded)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2026, in _UploadFileToObject
    parallel_composite_upload, logger)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1877, in _DelegateUploadFileToObject
    elapsed_time, uploaded_object = upload_delegate()
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 2009, in CallNonResumableUpload
    gzip_encoded=gzip_encoded_file)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1588, in _UploadFileToObjectNonResumable
    fields=UPLOAD_RETURN_FIELDS, gzip_encoded=gzip_encoded)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 292, in UploadObject
    gzip_encoded=gzip_encoded)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1385, in UploadObject
    gzip_encoded=gzip_encoded)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 1229, in _UploadObject
    global_params=global_params)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/third_party/storage_apitools/storage_v1_client.py", line 1197, in Insert
    download=download)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/base_api.py", line 729, in _RunMethod
    http, http_request, **opts)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 346, in MakeRequest
    check_response_func=check_response_func)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/apitools/apitools/base/py/http_wrapper.py", line 396, in _MakeRequestNoRetry
    redirections=redirections, connection_type=connection_type)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_media.py", line 276, in NewRequest
    connection_type=override_connection_type)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/transport.py", line 159, in new_request
    credentials._refresh(orig_request_method)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/client.py", line 761, in _refresh
    self._do_refresh_request(http)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/google-reauth-python/google_reauth/reauth_creds.py", line 120, in _do_refresh_request
    headers=headers))
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/google-reauth-python/google_reauth/reauth.py", line 269, in refresh_access_token
    headers=headers)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/google-reauth-python/google_reauth/_reauth_client.py", line 151, in refresh_grant
    headers=headers)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/google-reauth-python/google_reauth/reauth_creds.py", line 107, in http_request
    body=body, headers=headers)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/oauth2client/oauth2client/transport.py", line 282, in request
    connection_type=connection_type)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/httplib2/python2/httplib2/__init__.py", line 1694, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/httplib2/python2/httplib2/__init__.py", line 1434, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/gcs_json_media.py", line 529, in _conn_request
    conn.connect()
  File "/usr/lib/google-cloud-sdk/platform/gsutil/third_party/httplib2/python2/httplib2/__init__.py", line 1100, in connect
    sock.connect(sockaddr)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.timeout: timed out


【问题讨论】:

    标签: javascript command-line google-cloud-platform gsutil


    【解决方案1】:

    我假设这是一个作为服务器运行的 node.js 项目,而不是捆绑在浏览器中运行的代码。

    您的问题在这一行中定义:

    Error: Anonymous caller does not have storage.objects.create access to medchainvault/test.txt.
    

    此错误表示您的代码未使用凭据。

    通常,您的这行代码将使用 ADC 选择您的一个凭据:

    const storage = new Storage();
    

    但是,没有找到。

    你有几个选择:

    1. 运行 gcloud auth login 以设置凭据。
    2. 创建服务帐户,下载 Json 文件并在创建客户端时使用这些凭据(推荐)。

    使用类似这样的代码:

    const storage = new Storage({
      projectId: 'project-id',
      keyFilename: '/path/to/keyfile.json'
    });
    

    【讨论】:

    • 感谢您的良好响应 - 我已成功运行 gcloud auth login 多次,但仍然从节点程序和 gsutils 中收到错误。但是,我刚刚运行gcloud auth login 并收到有关 ADC 的警告 ``` 警告:gcloud auth login 不再写入应用程序默认凭据。如果需要使用 ADC,请参见:gcloud auth application-default --help
    • 推荐的方法是使用服务帐户。跳过使用用户凭据并从一开始就正确执行。
    猜你喜欢
    • 1970-01-01
    • 2018-12-15
    • 1970-01-01
    • 1970-01-01
    • 2021-03-22
    • 2020-11-03
    • 2021-01-25
    • 2021-08-24
    • 2013-12-15
    相关资源
    最近更新 更多