boto3.client.list_objects_v2 不显示最近的文件，Python 3.6

Question

我正在尝试列出最近从 AWS S3 存储桶上传的文件

但下面的代码不显示顶级文件

S3_Access_Key, S3_Secret_Key, S3_Region_Name, S3_Bucket, S3_Key = Bucket_Credentials(Bucket) # My Function to get Credentials

S3_Client = boto3.client('s3', aws_access_key_id=S3_Access_Key, aws_secret_access_key=S3_Secret_Key, region_name=S3_Region_Name)

Response = S3_Client.list_objects_v2(Bucket=S3_Bucket, Prefix=S3_Key)
Response = pd.DataFrame(Response['Contents'])
S3_Contents = Response[['Key', 'LastModified', 'Size']]

S3_Contents['Files'] = [s.split('/')[-1] for s in S3_Contents['Key']]

S3_Contents = S3_Contents.sort_values(['LastModified'], ascending=[False])
print(S3_Contents[['Files', 'LastModified', 'Size']].head(10))

桶顶文件：

仅显示以下项目：

0  25f1a8d1-3f35-4c87-9b1a-64687ab48d4d.gz 2019-10-25 13:38:13+00:00   57071951
1  598dee47-1d57-4807-88a4-f29e27a82505.gz 2019-10-24 15:41:36+00:00   57713744
2  9c41617a-c7cd-4beb-98bb-7e49812dcac9.gz 2019-10-24 15:41:13+00:00   64267610
3          COMSTATCSITC2019_24Oct19.csv.gz 2019-10-24 11:19:40+00:00  136841622

Answer 1

AWS CLI

这是一个AWS Command-Line Interface (CLI) 命令，用于列出 Amazon S3 存储桶中最近修改的 10 个对象：

aws s3api list-objects --bucket my-bucket --prefix foo/ --query 'reverse(sort_by(Contents, &LastModified))[0:10].[Key]' --output text

boto3

此脚本以 LastModified 的相反顺序对对象进行排序，然后打印前 10 个。

import boto3

s3_client = boto3.client('s3')

response = s3_client.list_objects_v2(Bucket='my-bucket', Prefix='foo/')

sorted_contents = sorted(response['Contents'], key=lambda object: object['LastModified'], reverse=True)

for object in sorted_contents[:10]:
    print(object['Key'].split('/')[-1], object['LastModified'], object['Size'])

Answer 2

以下代码有效：

S3_Access_Key, S3_Secret_Key, S3_Region_Name, S3_Bucket, S3_Key = Bucket_Credentials(Bucket)
S3_Resource = boto3.resource('s3', aws_access_key_id=S3_Access_Key, aws_secret_access_key=S3_Secret_Key, region_name=S3_Region_Name)
S3_Resource_Bucket = S3_Resource.Bucket(S3_Bucket)
S3_RS_Bucket_Objects = S3_Resource_Bucket.objects.filter(Prefix=S3_Key)

All_Files = []
for Object in S3_RS_Bucket_Objects:
   All_Files.append(Object.key)

Files_Uploaded = [s.split('\\')[-1] for s in Files_to_Upload]   
All_Files = [s.split('/')[-1] for s in All_Files]
Not_Uploaded = [Item for Item in Files_Uploaded if Item not in All_Files]
if len(Not_Uploaded) == 0:
    print('>> Success')
else:
    print('>> Failed: Some files not uploaded, Please cross check:\n', ', '.join(Not_Uploaded))