从 S3 转移到 GCS 时的 PERMISSION_DENIED

Question

我是 GCS 新手，希望能很好地解释我的问题。我有 S3 用于将我的 Aperture/照片从我的 Macbook 备份到云端。我想将该数据迁移到 GCS 以便于使用、更好的界面、gsutil 等。

我在 GCS 网站上使用了传输数据实用程序，并输入了我的 S3 凭据（访问密钥和秘密密钥）。我已经启用了互操作性，并在 GCS 中生成了一个密钥，尽管我认为没有必要。

在 S3 中，我在 IAM (GoogleTransfer) 中创建了一个用户，并为该用户创建了一个内联策略，其中选择了所有选项，但以 Delete* 开头的选项除外。这些权限是：

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "<my ID>",
        "Effect": "Allow",
        "Action": [
            "s3:AbortMultipartUpload",
            "s3:CreateBucket",
            "s3:GetAccelerateConfiguration",
            "s3:GetBucketAcl",
            "s3:GetBucketCORS",
            "s3:GetBucketLocation",
            "s3:GetBucketLogging",
            "s3:GetBucketNotification",
            "s3:GetBucketPolicy",
            "s3:GetBucketRequestPayment",
            "s3:GetBucketTagging",
            "s3:GetBucketVersioning",
            "s3:GetBucketWebsite",
            "s3:GetLifecycleConfiguration",
            "s3:GetObject",
            "s3:GetObjectAcl",
            "s3:GetObjectTorrent",
            "s3:GetObjectVersion",
            "s3:GetObjectVersionAcl",
            "s3:GetObjectVersionTorrent",
            "s3:GetReplicationConfiguration",
            "s3:ListAllMyBuckets",
            "s3:ListBucket",
            "s3:ListBucketMultipartUploads",
            "s3:ListBucketVersions",
            "s3:ListMultipartUploadParts",
            "s3:PutAccelerateConfiguration",
            "s3:PutBucketAcl",
            "s3:PutBucketCORS",
            "s3:PutBucketLogging",
            "s3:PutBucketNotification",
            "s3:PutBucketPolicy",
            "s3:PutBucketRequestPayment",
            "s3:PutBucketTagging",
            "s3:PutBucketVersioning",
            "s3:PutBucketWebsite",
            "s3:PutLifecycleConfiguration",
            "s3:PutReplicationConfiguration",
            "s3:PutObject",
            "s3:PutObjectAcl",
            "s3:PutObjectVersionAcl",
            "s3:ReplicateDelete",
            "s3:ReplicateObject",
            "s3:RestoreObject"
        ],
        "Resource": [
            "arn:aws:s3:::*"
        ]
    }
]

}

我在 GCS 中启动传输，它开始复制文件。它获取文件夹结构，但随后出现错误：

PERMISSION_DENIED: (showing 5 of 104342 failures)
Object: Aperture Library.aplibrary/Database/Albums/L2z6kBLyQPKRGvv8Jp%yig.apalbum
Object: Aperture Library.aplibrary/Database/Albums/LxYeHbbiSgaQzhjBgpbJBg.apalbum
Object: Aperture Library.aplibrary/Database/Albums/TT+4B27jQx+Us1lc3XJszQ.apalbum

PERMISSION_DENIED: (showing 5 of 104342 failures)
Object:
Object:
Object:
Object:
Object:

我已经多次尝试使用 S3 中设置为管理员的组权限，使用内联策略并将两者结合起来。此外，即使源不同，我也已选中该框以覆盖 GCS 文件。它获取文件夹结构和大量文件，然后点击这些对象并停止。

我的下一次尝试是使用 gsutil 来找出问题发生的确切位置，但我感觉会得到相同的结果。

有什么建议吗？

谢谢！

==============

更新：

正如预期的那样，gsutil 发出了错误，尽管这次我可以看到它没有立即工作，而不是等待 GCS 传输函数结束。我的输出：

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>MyReqID</RequestId><HostId>MyHostID</HostId></Error>

Copying s3://benmctee-aperture/Aperture Library.aplibrary/Database/Albums/CtW8oCcESyOINcweX1imtQ.apalbum [Content-Type=application/octet-stream]...
Copying s3://benmctee-aperture/Aperture Library.aplibrary/Database/Albums/DFVs%e5uSwqZ16gNqgycPQ.apalbum [Content-Type=application/octet-stream]...
Copying s3://benmctee-aperture/Aperture Library.aplibrary/Database/Albums/DLYiL8uITfuo6zLqdgYr1w.apalbum [Content-Type=application/octet-stream]...
Copying s3://benmctee-aperture/Aperture Library.aplibrary/Database/Albums/D7UOGRrXS3muB2ilQ80Fmw.apalbum [Content-Type=application/octet-stream]...
Exception in thread Thread-85:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/daisy_chain_wrapper.py", line 197, in PerformDownload
    decryption_tuple=self.decryption_tuple)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 252, in GetObjectMedia
    decryption_tuple=decryption_tuple)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 493, in GetObjectMedia
    generation=generation)
  File "/usr/lib/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 1430, in _TranslateExceptionAndRaise
    raise translated_exception
AccessDeniedException: AccessDeniedException: 403 InvalidObjectState
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidObjectState</Code><Message>The operation is not valid for the object's storage class</Message><RequestId>MyReqID</RequestId><HostId>MyHostID</HostId></Error>

Answer 1

这些对象是否存储在 Amazon Glacier 中？如果您尝试从 Amazon Glacier 读取对象但尚未“恢复”对象，“InvalidObjectState”是 S3 会出现的错误。

如果是这种情况，请注意，从 Amazon Glacier 恢复对象可能需要数小时，并且取决于有多少和许多其他因素，可能会非常昂贵。