如何使用 boto3 连接 S3 存储桶并访问 csv 文件？

Question

我已经使用 boto3 库创建了与 S3 的连接。但我仍然无法访问那里的文件。你能帮我看看我在做什么错吗？

Bucket : itx-acm-pas-dev-incoming-sourcefiles
File Name : Maestro_Data_loop_assignment_content.csv
file Location : itx-acm-cde-prd-incoming-sourcefiles/janssen_learn/TgtFiles/Maestro_Data_loop_assignment_content/

我也试过一种代码。

import boto3
import csv
import StringIO
import pandas as pd
from boto.s3.connection import S3Connection

s3 = boto3.client('s3',
         aws_access_key_id='yyyyyyyy',
         aws_secret_access_key='xxxxxxxxxxx')

AWS_KEY = 'yyyyyyyyyy'
AWS_SECRET = 'xxxxxxxxxx'
aws_connection = S3Connection(AWS_KEY, AWS_SECRET)
bucket = aws_connection.get_bucket('itx-acm-pas-dev-incoming-sourcefiles')

file_name = "itx-acm-cde-prd-incoming-sourcefiles/janssen_learn/TgtFiles/Maestro_Data_loop_assignment_content/Maestro_Data_loop_assignment_content.csv"

content = bucket.get_key(file_name).get_contents_as_string()
df = pd.read_csv(StringIO.StringIO(content))

print(df.head(3))

Answer 1

请参考本期：Read a csv file from aws s3 using boto and pandas

为了读取文件名，你也可以使用这个：

s3 = boto3.resource('s3')
bucket = s3.Bucket('test-bucket')
# Iterates through all the objects, doing the pagination for you. Each obj
# is an ObjectSummary, so it doesn't contain the body. You'll need to call
# get to get the whole body.
for obj in bucket.objects.all():
    key = obj.key
    body = obj.get()['Body'].read()

你也可以这样做：

import os
import boto3
import pandas as pd
import sys

if sys.version_info[0] < 3: 
    from StringIO import StringIO # Python 2.x
else:
    from io import StringIO # Python 3.x

# get your credentials from environment variables
aws_id = os.environ['AWS_ID']
aws_secret = os.environ['AWS_SECRET']

client = boto3.client('s3', aws_access_key_id=aws_id,
        aws_secret_access_key=aws_secret)

bucket_name = 'my_bucket'

object_key = 'my_file.csv'
csv_obj = client.get_object(Bucket=bucket_name, Key=object_key)
body = csv_obj['Body']
csv_string = body.read().decode('utf-8')

df = pd.read_csv(StringIO(csv_string))