上传带有 MD5 哈希的流会导致“您指定的 Content-MD5 无效”

Question

我正在使用Amazon S3 进行实施。我使用Amazon C# SDK，并尝试使用 putObject 方法上传创建的 ZIP 文件。

当我上传文件时，我收到以下错误：

{Amazon.S3.AmazonS3Exception: The Content-MD5 you specified was invalid

我生成了一个正常工作的内存流，我可以将它上传到 Amazon S3 而不会出错。但是，当我提供以下行时，它会给我带来问题：

request.MD5Digest = md5;

我是否以正确的方式证明了 MD5？我的 MD5 生成是否正确？还是有其他问题？

对我的上传代码的要求：

Once the Zip file has been created and you have calculated an MD5 sum value of that file, you should 
transfer the file to the AWS S3 bucket identified in the S3Access XML. 
Transfer the file using the AmazonS3, PutObjectRequest and TransferManagerclasses. 
Ensure the following meta data attributes are included via adding an ObjectMetaDataclass instance to the
PutObjectRequest:
• MD5Sum (via setContentMD5)
• Mime ContentType (setContentType)

我的上传代码

client.PutObject() 报错：

public void UploadFile(string bucketName, Stream uploadFileStream, string remoteFileName, string md5)
        {
            using (client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKeyID, secretAccessKeyID, config))
            {
                try
                {
                    StringBuilder stringResp = new StringBuilder();

                    PutObjectRequest request = new PutObjectRequest();
                   // request.MD5Digest = md5;
                    request.BucketName = bucketName;
                    request.InputStream = uploadFileStream;
                    request.Key = remoteFileName;
                    request.MD5Digest = md5;

                    using (S3Response response = client.PutObject(request))
                    {
                        WebHeaderCollection headers = response.Headers;
                        foreach (string key in headers.Keys)
                        {
                            stringResp.AppendLine(string.Format("Key: {0}, value: {1}", key,headers.Get(key).ToString()));
                            //log headers ("Response Header: {0}, Value: {1}", key, headers.Get(key));
                        }
                    }
                }
                catch (AmazonS3Exception amazonS3Exception)
                {
                    if (amazonS3Exception.ErrorCode != null && (amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId") || amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
                    {
                        //log exception - ("Please check the provided AWS Credentials.");
                    }
                    else
                    {
                        //log exception -("An error occurred with the message '{0}' when writing an object", amazonS3Exception.Message);
                    }
                }
            }
        }

整体方法

我的流程方法（看整体流程）。很抱歉代码的当前状态，它是伪代码而不是生产代码：

 public void Process(List<Order> order)
    {
        var zipName = UserName + "-" + DateTime.Now.ToString("yy-MM-dd-hhmmss") + ".zip";
        var zipPath = HttpContext.Current.Server.MapPath("~/Content/zip-fulfillment/" + zipName);

        CreateZip(order, zipPath);


        var s3 = GetS3Access();

        var amazonService = new AmazonS3Service(s3.keyid, s3.secretkey, "s3.amazonaws.com");
        var fileStream = new MemoryStream(HelperMethods.GetBytes(zipPath));
        var md5val = HelperMethods.GetMD5HashFromStream(fileStream);
        fileStream.Position = 0;
        amazonService.UploadFile(s3.bucket, fileStream, zipName, md5val);

        var sqsDoc = DeliveryXml(md5val, s3.bucket, "Test job");

        amazonService.SendSQSMessage(sqsDoc.ToString(), s3.postqueue);
    }

MD5 哈希方法：

此方法用于从我的内存流中创建 MD5 哈希：

    public static string GetMD5HashFromStream(Stream stream)
    {

        MD5 md5 = new MD5CryptoServiceProvider();
        byte[] retVal = md5.ComputeHash(stream);

        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < retVal.Length; i++)
        {
            sb.Append(retVal[i].ToString("x2"));
        }
        return sb.ToString();
    }

编辑：

只是在整体方法中添加了 fileStream.Position = 0。但仍然是完全相同的问题。

Answer 1

我怀疑问题可能是在计算了流的哈希值之后，流被留在了数据的 end...是没有数据。尝试在调用GetMD5HashFromStream之后添加这个

fileStream.Position = 0;

这将“倒回”流，以便您可以再次读取它。

编辑：刚刚查看了文档，虽然上面是 a 问题，但它不是唯一的问题。目前，您正在附加 MD5 哈希的 hex 表示 - 但 documentation 声明：

Content-MD5：消息的 base64 编码的 128 位 MD5 摘要

注意“base64 编码”部分。因此，您需要将 MD5 代码更改为：

public static string GetMD5HashFromStream(Stream stream)
{
    using (MD5 md5 = MD5.Create())
    {
        byte[] hash = md5.ComputeHash(stream);
        return Convert.ToBase64String(hash);
    }
}

Answer 2

如果您使用的是 AWS .NET SDK，您实际上可以只使用他们自己的实用程序类来执行此操作。看看这个测试https://github.com/aws/aws-sdk-net/blob/f2823205eba5e5cfcaa7a98ce59bfda3f9749a0e/sdk/test/Services/S3/IntegrationTests/ObjectLockConfigurationTests.cs 并搜索“MD5Digest”你会看到这种东西

var putObjectRequest = new PutObjectRequest()
{
    BucketName = bucketName,
    Key = key,
    ContentBody = content,
    MD5Digest = AmazonS3Util.GenerateChecksumForContent(content, true),
};

if (retainUntilDate.HasValue)
{
    putObjectRequest.ObjectLockMode = ObjectLockMode.Governance;
    putObjectRequest.ObjectLockRetainUntilDate = retainUntilDate.Value;
}