通过 AWS SDK 从 AWS S3 存储桶下载文件会损坏文件

Question

我正在使用 java 代码和 aws sdk 从我的 aws s3 中的存储桶下载文件 (zip)。 但是，下载的文件已损坏。 手动下载文件。

我比较了文件的内容并注意到损坏的文件包含某种未编码的字符 看到这个：

用于下载的代码如下：

public boolean downloadFile(String bucketName,  String fileNameOnServer, String localFileName )
    {
        S3Object object =null;
        InputStream objectData =null;
        try
        {
            object = s3.getObject(new GetObjectRequest(bucketName, fileNameOnServer));
            objectData = object.getObjectContent();
        }
        catch(Exception e)
        {
            LoggingService.writeToLog("Error001 downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName, e, LogModule.CommonUtils,LogLevel.Error);
            return false;
        }
        try
        {
            FileWriter fw = new FileWriter(localFileName, true);
            BufferedWriter bw = new BufferedWriter(fw);
            try
            {
                BufferedReader reader = new BufferedReader(new InputStreamReader(object.getObjectContent()));
                String line;                                                
                while( (line = reader.readLine() ) !=null)
                {
                    bw.write(line);
                    bw.newLine();
                }
                LoggingService.writeToLog("file from "+bucketName+"/"+fileNameOnServer+" "+" downloaded to "+bucketName + " successfully",  LogModule.CommonUtils,LogLevel.Info);
                return true;
            }
            catch(IOException e)
            {
                LoggingService.writeToLog("Error downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName, e, LogModule.CommonUtils,LogLevel.Error);
                return false;
            }
            finally
            {
                objectData.close();
                bw.close();
                object.close();
            }

        }
        catch(IOException e)
        {
            LoggingService.writeToLog("Error opening local file "+localFileName+" for writing ", e, LogModule.CommonUtils,LogLevel.Error);
            return false;
        }
    }

Answer 1

InputStreamReader 的文档包括：

InputStreamReader 是从字节流到字符流的桥梁：它读取字节并使用指定的字符集将它们解码为字符。它使用的字符集可以由名称指定，也可以明确给出，或者可以接受平台的默认字符集。

换句话说，它试图将数据视为文本数据。对于像 zip 文件这样的二进制数据，这几乎肯定会损坏数据。相反，如果您直接读取和写入字节，您将在不更改它们的情况下传递它们：

package com.exampleapp.app;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectInputStream;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.IOException;
import java.io.OutputStream;

public class App
{
    public static void main(String[] args)
    {
        downloadFile("example-bucket", "example-key.zip", "local-file.zip");
    }

    static AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(Regions.DEFAULT_REGION).build();

    public static boolean downloadFile(String bucketName,  String fileNameOnServer, String localFileName )
    {
        S3Object object = null;
        InputStream objectData = null;
        InputStream reader = null;
        OutputStream writer = null;
        try
        {
            object = s3.getObject(new GetObjectRequest(bucketName, fileNameOnServer));
            objectData = object.getObjectContent();
        }
        catch(Exception e)
        {
            System.out.println("Error001 downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName);
            return false;
        }
        try
        {
            File file = new File(localFileName);
            try
            {
                reader = new BufferedInputStream(object.getObjectContent());
                writer = new BufferedOutputStream(new FileOutputStream(file));
                int read = -1;
                while ( ( read = reader.read() ) != -1 )
                {
                    writer.write(read);
                }
                System.out.println("file from "+bucketName+"/"+fileNameOnServer+" "+" downloaded to "+bucketName + " successfully");
                return true;
            }
            catch(IOException e)
            {
                System.out.println("Error downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName);
                return false;
            }
            finally
            {
                object.close();
                writer.flush();
                writer.close();
                reader.close();
            }

        }
        catch(IOException e)
        {
            System.out.println("Error opening local file "+localFileName+" for writing ");
            return false;
        }
    }
}

Answer 2

Photo Spring BOOT 应用现在支持下载图像。它完美地工作。您应该使用适用于 Java V2 的 AWS 开发工具包。这是示例应用程序：

当我打开下载的图像时 - 它是有效的并且没有损坏：

从 Amazon S3 存储桶下载此图像的代码位于 Spring 控制器中：

  @RequestMapping(value = "/downloadphoto", method = RequestMethod.GET)
    void buildDynamicReportDownload(HttpServletRequest request, HttpServletResponse response) {
        try {

            //Get the photo object name
            String photoKey = request.getParameter("photoKey");
            byte[] photoBytes = s3Client.getObjectBytes("myphotobucket", photoKey) ;
            InputStream is = new ByteArrayInputStream(photoBytes);

            //define the required information to download the image
            response.setContentType("image/png");
            response.setHeader("Content-disposition", "attachment; filename="+photoKey);
            org.apache.commons.io.IOUtils.copy(is, response.getOutputStream());
            response.flushBuffer();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

V2 S3 代码在这里。注意它读取一个对象并返回一个字节[]。

public byte[] getObjectBytes (String bucketName, String keyName) {

        s3 = getClient();

        try {
            // create a GetObjectRequest instance
            GetObjectRequest objectRequest = GetObjectRequest
                    .builder()
                    .key(keyName)
                    .bucket(bucketName)
                    .build();

            // get the byte[] from this AWS S3 object
            ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
            byte[] data = objectBytes.asByteArray();
            return data;

        } catch (S3Exception e) {
            System.err.println(e.awsErrorDetails().errorMessage());
            System.exit(1);
        }
        return null;
    }

最后是 Javascript 调用

function DownloadImage(){

    //Post the values to the controller
    var photo =  $('#photo').val();
    window.location="../downloadphoto?photoKey=" + photo ;
}

此示例将被更新，因此所有这些代码都在示例文档中。

https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2/usecases/creating_photo_analyzer_app