来自 IOS 客户端的 AWS S3 POST 文件使用 AFNetworking答案

【问题标题】：AWS S3 POST File from IOS client Using AFNetworking来自 IOS 客户端的 AWS S3 POST 文件使用 AFNetworking
【发布时间】：2021-12-20 15:16:18
【问题描述】：

我希望通过这篇文章，不仅可以得到答案，还可以帮助许多在使用 AWS 安全 POST 策略基础设施方面遇到困难的人。代码在说明后显示。

我们使用运行 Django 作为服务器的 EC2 Ubuntu 20.2 实例。使用 boto3 (s3_client.generate_presigned_post)，我们使用 rest_framework 创建了一个 RetrieveAPIView。此通用视图接受 3 个 GET 参数 - file_name、meta_uuid 和 mime_type - 并生成 POST 签名和策略。此视图以 JSON 的形式返回“字段”字典、url 等。 Boto3 会忽略为“x-amz-date”输入的值并生成自己的值。

使用 Postman，我们可以成功地从 Ubuntu 服务器检索完整的策略。然后，在复制/粘贴返回字段的值后，我们使用 Postman 使用正文中的表单数据将文件发布到 AWS。文件上传正常，我们得到 HTTP 204 No Content 和 Header 中的其他字段。如果我们在发布到 AWS S3 期间对其中一个字段进行小幅更改，服务器会返回 HTTP 403 Forbidden 和代码“SignatureDoesNotMatch”，这正是人们所期望的。

但是，当尝试从 Xcode 模拟器或使用 AFNetworking 的 [AFHTTPSessionManager uploadTaskWithRequest: fromFile: progress: completionHandler:] 的实际设备运行时，我们从 AWS S3 - HTTP 400 Bad Request 和代码“IncompleteBody”中返回xml。 AWS 文档表明问题在于 S3 测量的内容长度与标头中发送的 Content-Length 不一致。

使用 iOS FileManger 测量的文件大小为 111251 。 AFNetworking outbound Header 是 Content-Length = 113071。当计算编码策略、签名、其他字段、边界值等的长度时，这是有道理的。

任何关于寻找什么的提示将不胜感激。 AWS 在计算 Content-Length 时是否只查看文件大小？这是否意味着策略正常，还是 AWS S3 在查看编码策略和签名之前首先进行一些基本的标头检查？

IOS 使用 AFNetworking 3.0 (CocoaPod)，Xcode 13.1。

谢谢，

埃里克注意：策略编码被捏造，以免暴露我们的 MY_AWS_ASSIGNED_S3_KEY_ID

代码如下，希望它能帮助许多其他人处理同样的问题：

服务器端 Django：

    def retrieve(self, request, *args, **kwargs):
        s3_client = boto3.client('s3',  aws_access_key_id=os.getenv('AWS_ACCESS_KEY_ID'),
                                        aws_secret_access_key=os.getenv('AWS_SECRET_ACCESS_KEY'),
                                        region_name=AWS_REGION_NAME,
                                        config=config.Config(signature_version='s3v4'))
        t = datetime.datetime.utcnow()
        amz_date = t.strftime('%Y%m%d')  # Date w/o time, used in credential scope
        amz_date_z = amz_date + 'T000000Z'
        amz_cred = os.getenv('AWS_ACCESS_KEY_ID') + '/' + amz_date + '/' + AWS_REGION_NAME + '/' + 's3/aws4_request'
        cont_type = request.GET.get('mime_type', None)
        meta_uuid = request.GET.get('meta_uuid', None)
        fields_dict = {
            'acl': 'authenticated-read',
            'bucket': AWS_STORAGE_BUCKET_NAME,
            'x-amz-algorithm': 'AWS4-HMAC-SHA256',
            'x-amz-meta-uuid': meta_uuid,
            'x-amz-credential': amz_cred,
            'x-amz-date': amz_date_z,
            'content-type': cont_type
        }
        cond_list = [
            {'content-type': cont_type},
            {'bucket': AWS_STORAGE_BUCKET_NAME},
            {'x-amz-meta-uuid': meta_uuid},
            {'acl': 'authenticated-read'},
        ]
        ret_dict = s3_client.generate_presigned_post(
            Bucket=AWS_STORAGE_BUCKET_NAME,
            Key=PRIVATE_MEDIA_LOCATION + request.GET.get('file_name', None),
            Fields=fields_dict,
            Conditions=cond_list,
            ExpiresIn=3600
        )
        print("ret_dict =")
        print(ret_dict)
        return Response({'status': 'success', 'data': ret_dict}, status=status.HTTP_202_ACCEPTED)

从服务器返回 JSON 给 Postman：

{
    "status": "success",
    "data": {
        "url": "https://aa-dev-media.s3.amazonaws.com/",
        "fields": {
            "acl": "authenticated-read",
            "bucket": "aa-dev-media",
            "x-amz-algorithm": "AWS4-HMAC-SHA256",
            "x-amz-meta-uuid": "some_random_string",
            "x-amz-credential": "**MY_AWS_ASSIGNED_S3_KEY_ID**/20211107/us-west-2/s3/aws4_request",
            "x-amz-date": "20211107T092556Z",
            "content-type": "image/jpeg",
            "key": "media/krTkQskg.jpg",
            "policy": "eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMDdUMTA6MjU6NTZaIiwgImNvbmRpdGlvbnMiOiBbeyJjb250ZW50LXR5cGUiOiAiaW1hZ2UvanBlZyJ9LCB7ImJ1Y2tldCI6ICJhYS1kZXYtbWVkaWEifSwgeyJ4LWFtei1tZXRhLXV1aWQiOiAic29tZV9yYW5kb21fc3RyaW5nIn0sIHsiYWNsIjogImF1dGhlbnRpY2F0ZWQtcmVhZCJ9LCB7ImJ1Y2tldCI6ICJhYS1kZXYtbWVkaWTTTSwgeyJrZXkiOiAibWVkaWEva3JUa1Fza2cuanBnIn0sIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwgeyJ4LWFtei1jcmVkZW50aWFsIjogIkFLSUE2UVZNTFlWNVRSUk9YQjVZLzIwMjExMTA3L3VzLXdlc3QtMi9zMy9hd3M0X3JlcXVlc3QifSwgeyJ4LWFtei1kYXRlIjogIjIwMjExMTA3VDA5MjU1NloifV19",
            "x-amz-signature": "8fcec036a12ae2ab9212133c5c7ce275e91961ab9486936a47ce0e6bb869e6ce"
        }
    }
}

App 中的 Objective C 代码：

-   (void)uploadCurrentMediaFileToS3ForAPI:(APIType)api usingParameters:(NSDictionary *)pDict
{
    NSLog(@"%s fileURL = %@",__FUNCTION__,currentMFO.file_url);
    NSLog(@"%s fName = %@",__FUNCTION__,currentMFO.file_name);
    NSLog(@"%s mType = %@",__FUNCTION__,currentMFO.mime_mfo.mime_type);
    NSLog(@"%s httpPath = %@",__FUNCTION__,httpPath);
    NSLog(@"%s pDict = %@",__FUNCTION__,pDict);
    NSLog(@"%s currentMFO.file_size_bytes = %lld",__FUNCTION__,currentMFO.file_size_bytes);

    NSError *reqError = nil;
    AFHTTPRequestSerializer *reqSerial  = [AFHTTPRequestSerializer serializer];
    NSMutableURLRequest *req = [reqSerial multipartFormRequestWithMethod:kHTTPPOST URLString:httpPath parameters:nil constructingBodyWithBlock:^(id<AFMultipartFormData>  _Nonnull formData)
    {
        
        NSArray *paramKeys  =   [pDict allKeys];
        for (NSString *key in paramKeys)
        {
            NSString *paramValue =  [pDict objectForKey:key];
            NSData *valData     =   [paramValue dataUsingEncoding:NSUTF8StringEncoding];
            NSLog(@"%s key = %@, value = %@ ",__FUNCTION__,key, paramValue);
            [formData   appendPartWithFormData:valData name:key];
        }
        
        BOOL fileFormSuccess    =   [formData appendPartWithFileURL:self->currentMFO.file_url name:@"file" fileName:self->currentMFO.file_name mimeType:self->currentMFO.mime_mfo.mime_type error:nil];
        NSLog(@"%s fileFormSuccess = %i",__FUNCTION__,fileFormSuccess);
    } error:&reqError];
    if (reqError) [self incrementErrorCountForAPI:api statusCode:100 locDesc:reqError.localizedDescription andReason:reqError.localizedFailureReason];
    [req addValue:@"*/*" forHTTPHeaderField:@"Accept"];
    [req addValue:@"500" forHTTPHeaderField:@"Keep-Alive"];
    NSLog(@"%s after req.allHTTPHeaderFields = %@",__FUNCTION__,req.allHTTPHeaderFields);
    NSLog(@"%s req.URL = %@",__FUNCTION__,req.URL);

    AFHTTPSessionManager *manager   =   [[AFHTTPSessionManager alloc] initWithSessionConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration]];
    manager.responseSerializer      =   [AFHTTPResponseSerializer   serializer];
    NSURLSessionUploadTask *uploadTask  =   [manager uploadTaskWithRequest:req fromFile:currentMFO.file_url progress:nil completionHandler:^(NSURLResponse * _Nonnull response, id  _Nullable responseObject, NSError * _Nullable error)
    {
        NSHTTPURLResponse *httpResp =   (NSHTTPURLResponse *)response;
        NSLog(@"%s httpResp.allHeaderFields - %@",__FUNCTION__,httpResp.allHeaderFields);
        NSLog(@"%s httpResp.MIMEType - %@",__FUNCTION__,httpResp.MIMEType);
        NSLog(@"%s httpResp.statusCode - %ld",__FUNCTION__,(long)httpResp.statusCode);
        if (responseObject)
        {
            NSString *respString    =   [[NSString alloc] initWithData:responseObject encoding:NSUTF8StringEncoding];
            NSLog(@"%s respString - %@",__FUNCTION__,respString);
        }
    }];
    [uploadTask resume];
}

这是来自 Xcode 调试器的日志：

fileURL = file:///var/mobile/Containers/Data/Application/8FF62939-0E50-4582-B242-01472D1C75D3/Library/UserMedia/horiz_windlass_photo.png
fName = horiz_windlass_photo.png
mType = image/png
httpPath = https://aa-dev-media.s3.amazonaws.com/
pDict = {
    acl = "authenticated-read";
    bucket = "aa-dev-media";
    "content-type" = "image/png";
    key = "media/horiz_windlass_photo.png";
    policy = "eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMDdUMTE6MjM6MDNaIiwgImNvbmRpdGlvbnMiOiBbeyJjb250ZW50LXR5cGUiOiAiaW1hZ2UvcG5nIn0sIHsiYnVja2V0IjogImFhLWRldi1tZWRpYSJ9LCB7IngtYW16LW1ldGEtdXVpZCI6ICJob3Jpel93aW5kbGFzc19waG90by5wbmcifSwgeyJhY2wiOiAiYXV0aGVudGljYXRlZC1yZWFkIn0sIHsiYnVja2V0IjogImFhLWRldi1tZWRpYSJ9LCB7ImtleSI6ICJtZWRpYS9ob3Jpel93aW5kbGFzc19waG90by5wbmcifSwgeyJ4LXXXXi1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LCB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQTZRVk1MWVY1VFJST1hCNVkvMjAyMTExMDcvdXMtd2VzdC0yL3MzL2F3czRfcmVxdWVzdCJ9LCB7IngtYW16LWRhdGUiOiAiMjAyMTExMDdUMTAyMzAzWiJ9XX0=";
    "x-amz-algorithm" = "AWS4-HMAC-SHA256";
    "x-amz-credential" = "**MY_AWS_ASSIGNED_S3_KEY_ID**/20211107/us-west-2/s3/aws4_request";
    "x-amz-date" = 20211107T102303Z;
    "x-amz-meta-uuid" = "horiz_windlass_photo.png";
    "x-amz-signature" = 41677f9254e324553a429bbcc3dcf5bd0f5df4af1c25fcaf51ce088a2a5dd032;
}
currentMFO.file_size_bytes = 111251
key = bucket, value = aa-dev-media
key = content-type, value = image/png
key = policy, value = eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMDdUMTE6MjM6MDNaIiwgImNvbmRpdGlvbnMiOiBbeyJjb250ZW50LXR5cGUiOiAiaW1hZ2UvcG5nIn0sIHsiYnVja2V0IjogImFhLWRldi1tZWRpYSJ9LCB7IngtYW16LW1ldGEtdXVpZCI6ICJob3Jpel93aW5kbGFzc19waG90by5wbmcifSwgeyJhY2wiOiAiYXV0aGVudGljYXRlZC1yZWFkIn0sIHsiYnVja2V0IjogImFhLWRldi1tZWRpYSJ9LCB7ImtleSI6ICJtZWRpYS9ob3Jpel93aW5kbGFzc19waG90by5wbmcifSwgeyJ4LXXXXi1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LCB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQTZRVk1MWVY1VFJST1hCNVkvMjAyMTExMDcvdXMtd2VzdC0yL3MzL2F3czRfcmVxdWVzdCJ9LCB7IngtYW16LWRhdGUiOiAiMjAyMTExMDdUMTAyMzAzWiJ9XX0=
key = x-amz-signature, value = 41677f9254e324553a429bbcc3dcf5bd0f5df4af1c25fcaf51ce088a2a5dd032
key = x-amz-algorithm, value = AWS4-HMAC-SHA256
key = acl, value = authenticated-read
key = x-amz-meta-uuid, value = horiz_windlass_photo.png
key = key, value = media/horiz_windlass_photo.png
key = x-amz-credential, value = **MY_AWS_ASSIGNED_S3_KEY_ID**/20211107/us-west-2/s3/aws4_request
key = x-amz-date, value = 20211107T102303Z
after formData = <AFStreamingMultipartFormData: 0x282c6e0d0>
fileFormSuccess = 1
req.allHTTPHeaderFields = {
    User-Agent = "AnchorAway/1.0 (iPhone; iOS 15.0.2; Scale/3.00)”,
    Accept-Language = "en-US;q=1”,
    Content-Type =  "multipart/form-data; boundary=Boundary+5D68022C08D21CC8”,
    Content-Length = 113071,
    Accept =  “*/*”,
    Keep-Alive = "500"
]
req.HTTPBody = (null)
eq.URL = https://aa-dev-media.s3.amazonaws.com/}
httpResp.allHeaderFields - {
    Server = AmazonS3
    Content-Type = "application/xml"
    Transfer-Encoding = "Identity"
    x-amz-request-id = "2BXQ22BB603CA92E"
    Date = "Sun, 07 Nov 2021 10:23:03 GMT"
    x-amz-id-2 = "tmvZxT96gJvL4SwxTdlHhkh3ZrGj1vmiG4JO1MQPyJc3bYlXMpofLxfImLuhBZEddSAS7nUdAzc="
    Connection = close
}
httpResp.MIMEType - application/xml
httpResp.statusCode - 400
respString - <?xml version="1.0" encoding="UTF-8"?><Error><Code>IncompleteBody</Code><Message>The request body terminated unexpectedly</Message> <RequestId>2BXQ22BB603CA92E<RequestId><HostId>tmvZxT96gJvL4SwxTdlHhkh3ZrGj1vmiG4JO1MQPyJc3bYlXMpofLxfImLuhBZEddSAS7nUdAzc=</HostId></Error>

【问题讨论】：

我用实际文件大小（以字节为单位）替换了 AFNetworking 生成的 content-length 标头。这并没有解决问题。
尝试将“content-length”参数添加到长度等于实际文件的策略字典中。同样，Postman 可以正常工作，但不能在 iOS 应用中使用 AFNetworking。

标签： amazon-s3 django-rest-framework postman boto3 afnetworking-3

【解决方案1】：

我找到了解决方案并愿意分享。如果您喜欢或使用它，请投“是”，这样也许我会获得足够的知名度，以便下次有人发布建议时……该方法在 Swift 中也应该可以正常工作……

将 AFNetworking 的 uploadTaskWithRequest:fromFile: 与其 multipartFormRequestWithMethod:URLString:parameters:constructingBodyWithBlock:error: ------ 结合使用是个坏主意。

AFNetworking 是 Apple 的 uploadTaskWithRequest:fromFile: 的子类。 Apple 文档状态 “此请求对象中的正文流和正文数据将被忽略。” 因此，在构建 Body 期间形成的任何内容随后都会被 uploadTask 忽略。 AWS 从未见过多种形式的正文或文件。

AWS 的 S3 响应错误代码不能按字面意思理解

这并不是说它们没有用，因为它们有用。但是，对于真正的问题，它们通常是“红鲱鱼”，这需要大量的蛮力挖掘。让我省心的是使用 WireShark 检查请求和响应中的数据包 --- 非常有帮助 - https://www.wireshark.org 我强烈推荐使用这个工具。

手动将自己的表单数据体构造为字符串，转换为 NSData，从图像文件中添加数据，附加表单数据后缀并保存到临时文件

您可能会发现这段代码很有用：

NSString * boundaryString       =   [NSString stringWithFormat:@"------XY%@",SOME_RANDOM_STRING];
NSMutableString *bodyString =   [NSMutableString stringWithFormat:@"--%@\r\n",boundaryString];
for (NSString *key in paramKeys)
{
    [bodyString appendFormat:@"Content-Disposition: form-data; name=\"%@\"\r\n\r\n",key];
    [bodyString appendFormat:@"%@",[paramDict objectForKey:key]];
    [bodyString appendFormat:@"\r\n--%@\r\n", boundaryString];
}
[bodyString appendFormat:@"Content-Disposition: form-data; name=\"%@\"; filename=\"%@\"\r\n",@"file",currentMFO.file_name];
[bodyString appendFormat:@"Content-Type: %@\r\n\r\n",currentMFO.mime_mfo.mime_type];
NSMutableData *postData =   [NSMutableData dataWithData:[bodyString dataUsingEncoding:NSUTF8StringEncoding]];
NSData *fileData        =   [NSData dataWithContentsOfFile:IMAGE_FILE_URL];
[postData appendData:fileData];
NSString *suffixString  =   [NSString stringWithFormat:@"\r\n--%@--\r\n",boundartString];
NSData *suffixData      =   [NSData dataWithData:[suffixString dataUsingEncoding:NSUTF8StringEncoding]];
[postData appendData:suffixData];
[[NSFileManager defaultManager] createFileAtPath:PATH_FOR_TEMP_FILE contents:postData attributes:nil];

最后，将临时文件 url 传递给 AFNetworking 的 uploadTaskWithRequest:fromFile: 应该可以正常工作。

【讨论】：