ApplicationError: 10 using Google Cloud Storage Java API to read a big zip file

Question

我正在使用 Google Cloud Storage Java API 从 Google App Engine 应用程序读取存储在 Google Cloud Storage 存储桶中的 zip 文件。 它适用于小文件，但是当我尝试读取我需要的真实文件（25Mb 压缩）时，我得到一个 ApplicationError: 10 异常。测试是在 GAE 上部署的应用程序完成的。

我可能已经理解 ApplicationError: 10 当文件打开时间过长（+ 超过 30 秒）时抛出，请参阅：https://groups.google.com/forum/?fromgroups#!topic/google-appengine-java/5XBVaVDKvvs，但我仍在努力让这个东西正常工作。

感谢您的帮助。

有我使用的测试代码：

FileService fileService = FileServiceFactory.getFileService();
AppEngineFile readableFile = new AppEngineFile("/gs/xxxx-my-bucket/xxx-bigfile.zip");
FileReadChannel readChannel = fileService.openReadChannel(readableFile, false);
ZipInputStream zis = new ZipInputStream(Channels.newInputStream(readChannel));
ZipEntry ze = null;
while ((ze = zis.getNextEntry()) != null) {
    resp.getOutputStream().println("Unzipping " + ze.getName());
    for (int c = zis.read(); c != -1; c = zis.read()) {
        resp.getOutputStream().print((char) c);
    }
    zis.closeEntry();
} 
zis.close();

我得到的堆栈跟踪是：

java.io.IOException
    at com.google.appengine.api.files.FileServiceImpl.translateException(FileServiceImpl.java:617)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:590)
    at com.google.appengine.api.files.FileServiceImpl.read(FileServiceImpl.java:539)
    at com.google.appengine.api.files.FileServiceImpl.read(FileServiceImpl.java:410)
    at com.google.appengine.api.files.FileReadChannelImpl.read(FileReadChannelImpl.java:73)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
    at java.io.FilterInputStream.read(FilterInputStream.java:133)
    at java.io.PushbackInputStream.read(PushbackInputStream.java:186)
    at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
    at java.util.zip.ZipInputStream.read(ZipInputStream.java:163)
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:122)
    at fr.heliorama.skylinewebapp.servlets.TestGSZip.doGet(TestGSZip.java:30)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:102)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:477)
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
    at java.lang.Thread.run(Thread.java:679)
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 10: 
    at java.lang.Thread.getStackTrace(Thread.java:1495)
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:237)
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:68)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:182)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:180)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:180)
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:68)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:101)
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:50)
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:586)
    ... 48 more
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 10: 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.failure(ApiProxyImpl.java:546)
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:784)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:824)
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.failure(RpcStub.java:815)
    at com.google.net.rpc3.impl.client.RpcClientInternalContext.runCallbacks(RpcClientInternalContext.java:895)
    at com.google.net.rpc3.impl.client.RpcClientInternalContext.finishRpcAndNotifyApp(RpcClientInternalContext.java:798)
    at com.google.net.rpc3.impl.client.RpcNetChannel.afterFinishingActiveRpc(RpcNetChannel.java:1054)
    at com.google.net.rpc3.impl.client.RpcNetChannel.finishRpc(RpcNetChannel.java:902)
    at com.google.net.rpc3.impl.client.RpcNetChannel.handleResponse(RpcNetChannel.java:2250)
    at com.google.net.rpc3.impl.client.RpcNetChannel.messageReceived(RpcNetChannel.java:2057)
    at com.google.net.rpc3.impl.client.RpcNetChannel.access$2000(RpcNetChannel.java:143)
    at com.google.net.rpc3.impl.client.RpcNetChannel$TransportCallback.receivedMessage(RpcNetChannel.java:3112)
    at com.google.net.rpc3.impl.client.RpcChannelTransportData$TransportCallback.receivedMessage(RpcChannelTransportData.java:599)
    at com.google.net.rpc3.impl.wire.RpcBaseTransport.receivedMessage(RpcBaseTransport.java:417)
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.parseOneMessage(RpcClientTcpTransport.java:750)
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.parseMessages(RpcClientTcpTransport.java:634)
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.access$100(RpcClientTcpTransport.java:37)
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport$1.dataReceived(RpcClientTcpTransport.java:295)
    at com.google.net.async3.SocketConnection.handleNetworkReadEvent(SocketConnection.java:900)
    at com.google.net.async3.SocketConnection.access$400(SocketConnection.java:43)
    at com.google.net.async3.SocketConnection$NetworkReadHandlerImpl.run(SocketConnection.java:929)
    at com.google.net.eventmanager.AbstractFutureTask$Sync.innerRun(AbstractFutureTask.java:260)
    at com.google.net.eventmanager.AbstractFutureTask.run(AbstractFutureTask.java:121)
    at com.google.net.eventmanager.EventManagerImpl.runTask(EventManagerImpl.java:575)
    at com.google.net.eventmanager.EventManagerImpl.internalRunWorkerLoop(EventManagerImpl.java:999)
    at com.google.net.eventmanager.EventManagerImpl.runWorkerLoop(EventManagerImpl.java:881)
    at com.google.net.eventmanager.WorkerThreadInfo.runWorkerLoop(WorkerThreadInf...(length 8282)

Answer 1

好的，当使用谷歌应用引擎和谷歌云商店时，你会受到几个时间限制：

• 在 GS 上读取文件需要 30 秒。
• 使用 GAE URLFetch 服务时需要 10 秒
• GAE 上的每个请求 60 秒。

在我的特定情况下，我无法解决这些限制。