【发布时间】:2018-08-18 02:24:26
【问题描述】:
我正在尝试以块的形式从 Azure blob 下载数据,然后尝试将相同的块上传到 aws s3 存储桶。
上传时我收到"Your proposed upload is smaller than the minimum allowed size"exception。我注意到一件事,在上传响应中,我得到的内容长度为 0。我正在尝试的数据大小超过 300MB。
任何指针这里可能有什么问题?
下面是我的代码 sn-p :
var remainingLength = blob.Properties.Length;
long startPosition = 0;
List<UploadPartResponse> uploadResponses = new List<UploadPartResponse>();
int i = 1;
string uploadId = string.Empty;
//Step 1: build and send a multi upload request
var initiateRequest = new InitiateMultipartUploadRequest
{
BucketName = existingBucketName,
Key = "firstobj"
};
var initResponse = client.InitiateMultipartUpload(initiateRequest);
uploadId = initResponse.UploadId;
do
{
var blockSize = Math.Min(segmentSize, remainingLength);
using (var ms = new MemoryStream())
{
blob.DownloadRangeToStream(ms, startPosition, blockSize);
//Step 2: upload each chunk (this is run for every chunk unlike the other steps which are run once)
var uploadRequest = new UploadPartRequest
{
BucketName = existingBucketName,
Key = "firstobj",
UploadId = uploadId,
PartNumber = i,
PartSize = ms.Length,
InputStream = ms
};
// Upload part and add response to our list.
var temp = client.UploadPart(uploadRequest);
uploadResponses.Add(temp);
}
//Step 3: build and send the multipart complete request
if (blockSize < segmentSize)
{
var completeRequest = new CompleteMultipartUploadRequest
{
BucketName = existingBucketName,
Key = "firstobj",
UploadId = uploadId,
};
completeRequest.AddPartETags(uploadResponses);
client.CompleteMultipartUpload(completeRequest);
}
startPosition += blockSize;
remainingLength -= blockSize;
i++;
}
while (remainingLength > 0);
【问题讨论】:
-
您尝试在 S3 中上传的分段大小/块大小是多少?
-
@GauravMantri 50 MB
-
代码中的哪一行出现错误?
-
@GauravMantri client.CompleteMultipartUpload(completeRequest);抛出错误
-
PartSize = ms.Length?这似乎不对。这似乎应该是PartSize = blockSize,不是吗?
标签: amazon-web-services azure amazon-s3