Aws Batch docker s3.download_fileobj() 错误答案

【问题标题】：Aws Batch docker s3.download_fileobj() errorAws Batch docker s3.download_fileobj() 错误
【发布时间】：2018-12-17 11:31:12
【问题描述】：

您好，我正在通过 aws Batch 启动我的 docker 容器。我的 aws 批处理一直失败。我目前正在尝试下载 file_object 并将其重新上传到不同的 s3 存储桶。每次我收到 OSERROR 第一次是：

OSError: [Errno 30] Read-only file system

这是我的下载功能：

 def download(self):
        s3 = boto3.client('s3')
        file_name = self.flow_cells[10:]
        try:
            with open(file_name, 'wb') as data:
                s3.download_fileobj(
                    self.source_s3_bucket,
                    self.source_key,
                    data
                )
            return True
        except botocore.exceptions.ClientError as error:
            print(error.response['Error']['Code'])

错误发生在 s3.download)fileobj 调用中

点击数据时会被标记。

我第二次运行这个来检查我得到的错误

OSError: [Errno 5] Input/output error

以下是我的容器定义。

container_properties = <<CONTAINER_PROPERTIES
  {
      "command": [
      "--object_key", "Ref::object_key",
      "--glacier_s3_bucket", "Ref::glacier_s3_bucket",
      "--output_s3_bucket", "Ref::output_s3_bucket",
      "--default_s3_bucket", "Ref::default_s3_bucket"  
      ],
      "environment": [],
      "image": "temp_image_name",
      "jobRoleArn": "${aws_iam_role.task-role.arn}",
      "memory": 1024,
      "mountPoints": [],
      "privileged": true,
      "readonlyRootFilesystem": false,
      "ulimits": [],
      "vcpus": 1,
      "volumes": [],
    "jobDefinitionName": "docker-flowcell-restore-${var.environment}"
  }

这是程序的完整日志：

File "src/main.py", line 101, in download
17:10:55
data
17:10:55
File "/usr/local/lib/python3.5/dist-packages/boto3/s3/inject.py", line 678, in download_fileobj
17:10:55
return future.result()
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/futures.py", line 73, in result
17:10:55
return self._coordinator.result()
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/futures.py", line 233, in result
17:10:55
raise self._exception
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/tasks.py", line 126, in __call__
17:10:55
return self._execute_main(kwargs)
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/tasks.py", line 150, in _execute_main
17:10:55
return_value = self._main(**kwargs)
17:10:55
File "/usr/local/lib/python3.5/dist-packages/s3transfer/download.py", line 583, in _main
17:10:55
fileobj.write(data)
17:10:55
OSError: [Errno 5] Input/output error

【问题讨论】：

标签： docker amazon-s3 terraform aws-batch

【解决方案1】：

解决这个问题的方法是

os.chdir('/tmp')

在 docker 容器将运行的代码内部。

【讨论】：