【问题标题】:Upload File to Google Cloud Storage via appengine通过 appengine 将文件上传到 Google Cloud Storage
【发布时间】:2016-07-26 13:17:36
【问题描述】:

我正在尝试将文件上传到 Google Cloud Storage。我的 Servlet 代码是

public class UploadFile extends HttpServlet {

    private final String BUCKET = "XXXXXXXXX";
       private boolean isMultipart;
       private String filePath;
       private int maxFileSize = 5 * 1024 * 1024;
       private int maxMemSize = 5 * 1024 * 1024;
       private File file ;

       public void init( ){
          // Get the file location where it would be stored.
          filePath = 
                 getServletContext().getInitParameter("file-upload"); 
       }
       public void doPost(HttpServletRequest request, 
                   HttpServletResponse response)
                  throws ServletException, java.io.IOException {
          // Check that we have a file upload request
          isMultipart = ServletFileUpload.isMultipartContent(request);
          response.setContentType("text/html");
          java.io.PrintWriter out = response.getWriter( );
          if( !isMultipart ){
             out.println("<html>");
             out.println("<head>");
             out.println("<title>Servlet upload</title>");  
             out.println("</head>");
             out.println("<body>");
             out.println("<p>No file uploaded</p>"); 
             out.println("</body>");
             out.println("</html>");
             return;
          }
          DiskFileItemFactory factory = new DiskFileItemFactory();
          // maximum size that will be stored in memory
          factory.setSizeThreshold(maxMemSize);
          // Location to save data that is larger than maxMemSize.
          factory.setRepository(new File("/temp/image/"));

          // Create a new file upload handler
          ServletFileUpload upload = new ServletFileUpload(factory);
          // maximum file size to be uploaded.
          upload.setSizeMax( maxFileSize );

          try{ 
          // Parse the request to get file items.
          List fileItems = upload.parseRequest(request);

          // Process the uploaded file items
          Iterator i = fileItems.iterator();

          out.println("<html>");
          out.println("<head>");
          out.println("<title>Servlet upload</title>");  
          out.println("</head>");
          out.println("<body>");
          while ( i.hasNext () ) 
          {
             FileItem fi = (FileItem)i.next();
             if ( !fi.isFormField () )  
             {
                // Get the uploaded file parameters
                String fieldName = fi.getFieldName();
                String fileName = fi.getName();
                String contentType = fi.getContentType();
                boolean isInMemory = fi.isInMemory();
                long sizeInBytes = fi.getSize();
                // Write the file
                if( fileName.lastIndexOf("\\") >= 0 ){
                   file = new File( filePath + 
                   fileName.substring( fileName.lastIndexOf("\\"))) ;
                }else{
                   file = new File( filePath + 
                   fileName.substring(fileName.lastIndexOf("\\")+1)) ;
                }

               String path = Events.uploadFile ( fileName, "image/*", file, BUCKET );

                // fi.write( file ) ;
                out.println("Uploaded Filename: " + fileName + "<br>"+ " File Path:"+ path);
             }
          }
          out.println("</body>");
          out.println("</html>");
       }catch(Exception ex) {
           System.out.println(ex);
       }
       }
       public void doGet(HttpServletRequest request, 
                           HttpServletResponse response)
            throws ServletException, java.io.IOException {

            throw new ServletException("GET method used with " +
                    getClass( ).getName( )+": POST method required.");
       } 
    }

web.xml

<servlet>
        <servlet-name>UploadFile</servlet-name>
        <servlet-class>XXXXXXXXXX.UploadFile</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>UploadFile</servlet-name>
        <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
    </servlet-mapping>

fileUpload 函数将文件保存到 GCS

public static String uploadFile ( String name, String contentType, File file, String bucketName )
            throws IOException, GeneralSecurityException
    {

        InputStreamContent contentStream = new InputStreamContent ( contentType, new FileInputStream ( file ) );

        // Setting the length improves upload performance
        contentStream.setLength ( file.length () );

        StorageObject objectMetadata = new StorageObject ()
                // Set the destination object name
                .setName ( name )
                // Set the access control list to publicly read-only
                .setAcl ( Arrays.asList ( new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );

        // Do the insert
        Storage client = StorageFactory.getService ();
        Storage.Objects.Insert insertRequest = client.objects ().insert ( bucketName, objectMetadata, contentStream );

        insertRequest.execute ();
        return "https://storage.cloud.google.com/" + BUCKET + "/" + file.getName ();
    }

但是当我尝试使用一些 API 测试客户端进行测试时,它给出了错误

org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found

在将它与 Angular 中的 UI 集成并在本地测试之后,我面临这个问题

Cross-Origin Request Blocked Reason: CORS header 'Access-Control-Allow-Origin' missing

我试图解决这个问题,但没有找到与 google appengine 对应的解决方案。


最初我试图通过此代码上传图像,但在不久的将来,相同的代码将用于将 .pdf 和 .html 文件上传到 GCS。

供参考: 我正在使用 Google Endpoints 来满足我与客户端的其他数据通信需求。 Client End 是一个在 Angular 中构建的 webapp,但它将扩展到 android 和 ios。

任何帮助将不胜感激。

谢谢

2016 年 1 月 8 日更新

现在我正在服务器上获取文件,但我不知道在将文件发送到 Google 云存储之前我必须临时保存文件的位置。关于将文件存储在

war\WEB-INI\

我面临的例外是

 java.security.AccessControlException: access denied ("java.io.FilePermission" "\war\WEB-INI\profile.png" "read")

【问题讨论】:

    标签: java google-app-engine google-cloud-storage google-cloud-endpoints


    【解决方案1】:

    我终于可以通过 appengine 将文件从客户端上传到 Google Cloud Storage。

    我假设在执行这些步骤之前,您已经准备好了以下内容

    • 来自您的服务帐户的 JSON 文件。
    • 已创建默认存储桶。

    第 1 步:制作这样的 Servlet

    package XXXXXXXXX;
    
    import java.io.IOException;
    import java.io.InputStream;
    import java.security.GeneralSecurityException;
    import java.util.Arrays;
    
    import javax.servlet.ServletException;
    import javax.servlet.http.HttpServlet;
    import javax.servlet.http.HttpServletRequest;
    import javax.servlet.http.HttpServletResponse;
    
    import com.google.api.client.http.InputStreamContent;
    import com.google.api.services.storage.Storage;
    import com.google.api.services.storage.model.ObjectAccessControl;
    import com.google.api.services.storage.model.StorageObject;
    
    import XXXXXXXXXXXXX.StorageFactory;
    
    //@author Umesh Chauhan
    
    /**
     * Save File to GCS
     *
     * @param fileName        File Name with format
     * @header Content-Type   "*/*"
     * @return file path
     * @throws Exception      Any Error during upload
     */
    public class UploadFile extends HttpServlet
    {
    
        private static final long serialVersionUID = 1L;
        private final String BUCKET = "YOUR BUCKET NAME";
        private int maxFileSize = 6 * 1024 * 1024;
    
        @Override
        protected void doOptions ( HttpServletRequest req, HttpServletResponse resp ) throws ServletException, IOException
        {
            // pre-flight request processing
            resp.setHeader ( "Access-Control-Allow-Origin", "*" );
            resp.setHeader ( "Access-Control-Allow-Methods", "*" );
            resp.setHeader ( "Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept" );
        }
    
        @Override
        public void doPost ( HttpServletRequest request, HttpServletResponse response )
                throws ServletException, java.io.IOException
        {
    
            try
            {
                String path = uploadFile ( request.getParameter ( "fileName" ), request.getContentType (),
                        request.getInputStream (), BUCKET, request.getInputStream ().available () );
                // Sending Response
                response.setStatus ( HttpServletResponse.SC_OK );
                response.getWriter ().write ( path );
                response.getWriter ().flush ();
                response.getWriter ().close ();
    
            }
            catch ( GeneralSecurityException e )
            {
                e.printStackTrace ();
            }
        }
    
        public String uploadFile ( String name, String contentType, InputStream input, String bucketName,
                int contentLength ) throws IOException, GeneralSecurityException
        {
    
            InputStreamContent contentStream = new InputStreamContent ( contentType, input );
    
            if ( contentLength < maxFileSize )
            {
    
                // It is done Automatically.
                /*
                 * // Setting the length improves upload performance
                 * contentStream.setLength ( contentLength );
                 */
    
                StorageObject objectMetadata = new StorageObject ()
                        // Set the destination object name
                        .setName ( name )
                        // Set the access control list to publicly read-only
                        .setAcl ( Arrays.asList (
                                new ObjectAccessControl ().setEntity ( "allUsers" ).setRole ( "READER" ) ) );
    
                // Do the insert
                Storage client = StorageFactory.getService ();
    
                Storage.Objects.Insert insertRequest = client.objects ()
                        .insert ( bucketName, objectMetadata, contentStream );
    
                insertRequest.execute ();
    
                return "https://storage.cloud.google.com/" + BUCKET + "/" + name;
            }
            else
            {
                throw new GeneralSecurityException ( "File size canot be more then 6 MB !" );
            }
        }
    
        public void doGet ( HttpServletRequest request, HttpServletResponse response )
                throws ServletException, java.io.IOException
        {
            throw new ServletException ( "GET method used with " + getClass ().getName () + ": POST method required." );
        }
    
    }
    

    第 2 步:您的存储工厂

    package XXXXXXXXXXXX;
    
    import java.io.IOException;
    import java.net.URL;
    import java.security.GeneralSecurityException;
    import java.util.Collection;
    
    import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
    import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
    import com.google.api.client.http.HttpTransport;
    import com.google.api.client.json.JsonFactory;
    import com.google.api.client.json.jackson2.JacksonFactory;
    import com.google.api.services.storage.Storage;
    import com.google.api.services.storage.StorageScopes;
    
    //@author Umesh Chauhan 
    
    public class StorageFactory
    {
    
        private static Storage instance = null;
    
        public static synchronized Storage getService () throws IOException, GeneralSecurityException
        {
            if ( instance == null )
            {
                instance = buildService ();
            }
            return instance;
        }
    
        private static Storage buildService () throws IOException, GeneralSecurityException
        {
    
            HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport ();
            JsonFactory jsonFactory = new JacksonFactory ();
    
            GoogleCredential credential = GoogleCredential.fromStream (
                    new URL ( "HERE GOES THE URL FOR YOUR SERVICE ACCOUNT JSON - I USED GOOGLE DRIVE DIRECT DOWNLOAD LINK TO MY JSON FILE" )
                            .openStream () );
    
            // Depending on the environment that provides the default credentials
            // (for
            // example: Compute Engine, App Engine), the credentials may require us
            // to
            // specify the scopes we need explicitly. Check for this case, and
            // inject
            // the Cloud Storage scope if required.
            if ( credential.createScopedRequired () )
            {
                Collection<String> scopes = StorageScopes.all ();
                credential = credential.createScoped ( scopes );
            }
    
            return new Storage.Builder ( transport, jsonFactory, credential ).setApplicationName ( "YOUR PROJECT NAME" ).build ();
        }
    }
    

    第 3 步:更新您web.xml

        <servlet>
            <servlet-name>UploadFile</servlet-name>
            <servlet-class>PACKAGE.UploadFile</servlet-class>
        </servlet>
    
        <servlet-mapping>
            <servlet-name>UploadFile</servlet-name>
            <url-pattern>/uploadManager/UploadFile</url-pattern>  //Based on your original URL
        </servlet-mapping>
    

    【讨论】:

    • “这里是您的服务帐户 JSON 的 URL - 我使用 GOOGLE DRIVE DIRECT 下载链接到我的 JSON 文件”如何生成服务帐户 json?你能举一个结构的例子吗?
    • @user5980143 你必须从谷歌控制台下载它。
    • 如果我已经在谷歌应用引擎中运行,我不需要“GoogleCreditial”对吗?
    【解决方案2】:

    这里是根据docs上传到存储的正确方法

    import com.google.appengine.tools.cloudstorage.*;
    
    public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
      String buffer = "the data to be saved in GCS bla bla";
      GcsFileOptions instance = GcsFileOptions.getDefaultInstance();
      GcsFilename fileName = new GcsFilename("bucketName", "fileName");
      GcsOutputChannel outputChannel;
      gcsService.createOrReplace(fileName, instance, ByteBuffer.wrap(buffer.getBytes()));
    }
    

    你会找到完整的代码here

    【讨论】:

    • 这是 100% 正确的方法,最初的问题是他的应用程序正在“谷歌应用程序引擎”中运行!
    • 我发布了另一个选项 stackoverflow.com/questions/38590971/… com.google.cloud.storage 库比 com.google.appengine 维护得更好
    【解决方案3】:

    使用com.google.cloud.storage

    • 在 appengine 上运行时 - 无需身份验证
    • 在本地运行时运行gcloud auth application-default login 命令 见authentication

      import com.google.cloud.storage.*;
      
      
      public static void upload(String bucketName, String fileId, String content) throws IOException {
          Storage storage = StorageOptions.newBuilder().build().getService();
      
          BlobInfo fileInfo = BlobInfo.newBuilder(bucketName, fileId)
                  .build();
      
          InputStream fileIS = IOUtils.toInputStream(content, "UTF-8");
          storage.create(fileInfo, fileIS);
      
      }
      

    【讨论】:

      【解决方案4】:

      首先您需要在飞行前请求处理中解决 CORS 问题,您需要在后端进行:在 Google App Engine 上通过添加 doOptions 方法来完成,例如:

      @Override
      protected void doOptions(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException 
      { 
          resp.setHeader("Access-Control-Allow-Origin", "*");
          resp.setHeader("Access-Control-Allow-Methods", "*");
          resp.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
      }
      

      那么您需要确保您发送的请求带有像Content-Type: multipart/form-data 这样的标头,否则您的文件将被错误地编码。在 angular2 中,请求标头在您的发布请求中设置为第三个(可选)参数,例如:

      let headers = new Headers();
      headers.append('content-type', 'multipart/form-data');
      http.post(url, body, {
                        headers:headers
                      })
      

      【讨论】:

        【解决方案5】:

        很抱歉,这并没有直接解决您的问题,但我还是想指出这一点。我建议使用您的普通端点之一来生成临时上传 URL,您的 Angular 客户端可以使用该 URL 将文件直接发送到 Cloud Storage,而无需通过您的应用程序。一种方法是通过 blobstore API,如 here 所述。您可以按照here 所述(在同一页面下方)通过该 API 上传到云存储。

        这减少了您在服务器上所需的上传代码量,不受 GAE 请求的 32MB 限制,并且符合 Google 的建议(如上面的文档链接所示)。

        【讨论】:

          猜你喜欢
          • 2014-01-05
          • 2015-08-28
          • 2018-12-23
          • 1970-01-01
          • 2019-06-04
          • 1970-01-01
          • 2014-06-02
          • 1970-01-01
          • 1970-01-01
          相关资源
          最近更新 更多