使用 Java 和 OkHttpClient 将大文件分块上传到 AWS 存储桶答案

【问题标题】：Upload large files to AWS bucket in chuncks using Java and OkHttpClient使用 Java 和 OkHttpClient 将大文件分块上传到 AWS 存储桶
【发布时间】：2022-08-16 08:10:47
【问题描述】：

我需要使用 Java 和 OkHttpClient3 将大文件上传到 AWS 存储桶。

我能够以单个块上传文件并且一切正常。但是，我需要在多个块中进行，例如 500mb 的块。如何部分发送文件片段？

这适用于单个块......


import okhttp3.*;

OkHttpClient client = new OkHttpClient().newBuilder()
        .connectTimeout(2400, TimeUnit.SECONDS)
        .writeTimeout(2400, TimeUnit.SECONDS)
        .readTimeout(2400, TimeUnit.SECONDS)
        .build();
RequestBody body = new MultipartBody.Builder().setType(MultipartBody.FORM)
        .addFormDataPart(\"Content-Type\", _responseInit.getMultipart_params().getContentType())
        .addFormDataPart(\"Policy\", _responseInit.getMultipart_params().getPolicy())
        .addFormDataPart(\"X-Amz-Signature\", _responseInit.getMultipart_params().getXamzSignature())
        .addFormDataPart(\"acl\", _responseInit.getMultipart_params().getAcl())
        .addFormDataPart(\"key\", _responseInit.getMultipart_params().getKey() + \"/p1\")
        .addFormDataPart(\"success_action_status\", _responseInit.getMultipart_params().getSuccess_action_status())
        .addFormDataPart(\"x-amz-algorithm\", _responseInit.getMultipart_params().getXamzAlgorithm())
        .addFormDataPart(\"x-amz-credential\", _responseInit.getMultipart_params().getXamzCredential())
        .addFormDataPart(\"x-amz-date\", _responseInit.getMultipart_params().getXamzDate())
        .addFormDataPart(\"name\", _asset.getName())
        .addFormDataPart(\"chunk\", \"1\")
        .addFormDataPart(\"chunks\", \"1\")
        .addFormDataPart(\"Filename\", _responseInit.getMultipart_params().getKey() + \"/p1\")
        .addFormDataPart(\"file\", _asset.getName(),
                RequestBody.create(MediaType.parse(\"application/octet-stream\"), _asset))
        .build();
Request request = new Request.Builder()
        .url(_endPoint)
        .method(\"POST\", body)
        .addHeader(\"Content-Type\", \"multipart/form-data\")
        .build();
Response response = client.newCall(request).execute();

每次上传一个块后，我需要注册这个块。所以我需要多次调用这段代码......

标签： java okhttp

【解决方案1】：

我找到了解决方案。使用字节数组分块读取文件，然后将字节数组作为参数传递给客户端对象...

    int filePosition = 0;
    long currentPosition = 0;
    
    String partSize = "104857600"; // Chuncks of 100 MB.
    Integer partSizeInt = Integer.valueOf(partSize);

    byte[] buffer = new byte[partSizeInt ];
    FileInputStream fileToUpload = null;
    
    File myFile = new File("FULL PATH TO FILE");


    try {
        fileToUpload = new FileInputStream(myFile);

        int read = 0;

        while (currentPosition < myFile.length())
        {
            read = fileToUpload.read(buffer, filePosition, buffer.length);

            if (read == -1) {
                break;
            }else{
                byte[] bytesRead = Arrays.copyOf(buffer, read);

                ...

                .addFormDataPart("file", _asset.getName(),
            RequestBody.create(MediaType.parse("application/octet-stream"), bytesRead ))

                ...

                currentPosition = currentPosition + read;
            }
        }

    } catch (Exception e) {
        // crap
    } finally {
        try {
            fileToUpload.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

【讨论】：