【发布时间】：2015-08-14 15:27:43
【问题描述】：
我正在开发一个网络应用程序，它需要一个 zip 文件，由用户上传，在服务器上解压缩并处理文件。当 zip 文件不是太大（20-25MB）时，它就像一个魅力，但如果文件大约或超过（50MB），它会产生 OutOfMemoryError。
我尝试通过添加来增加 java 最大内存分配池
export CATALINA_OPTS="-Xmx1024M" tomcat7中的startup.sh，但错误依旧存在。
AFAIK，问题在于解压缩 .zip 文件。 top 显示在提取 50MB 文件期间，tomcat 使用了 800MB 内存。是否有任何解决方案可以在有效利用可用内存的同时实现高达 ~200MB 的上传？
解压代码如下：
package user;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
public class unzip {
public void unzipFile(String filePath, String oPath)
{
FileInputStream fis = null;
ZipInputStream zipIs = null;
ZipEntry zEntry = null;
try {
fis = new FileInputStream(filePath);
zipIs = new ZipInputStream(new BufferedInputStream(fis));
while((zEntry = zipIs.getNextEntry()) != null){
try{
byte[] tmp = new byte[8*1024];
FileOutputStream fos = null;
String opFilePath = oPath+zEntry.getName();
System.out.println("Extracting file to "+opFilePath);
fos = new FileOutputStream(opFilePath);
int size = 0;
while((size = zipIs.read(tmp)) != -1){
fos.write(tmp, 0 , size);
}
fos.flush();
fos.close();
}catch(Exception ex){
}
}
zipIs.close();
fis.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
错误代码如下：
HTTP Status 500 - javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
type Exception report
message javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
description The server encountered an internal error that prevented it from fulfilling this request.
exception
org.apache.jasper.JasperException: javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:549)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:455)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
root cause
javax.servlet.ServletException: java.lang.OutOfMemoryError: Java heap space
org.apache.jasper.runtime.PageContextImpl.doHandlePageException(PageContextImpl.java:916)
org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:845)
org.apache.jsp.Upload_jsp._jspService(Upload_jsp.java:369)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
root cause
java.lang.OutOfMemoryError: Java heap space
org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:322)
org.apache.commons.io.output.DeferredFileOutputStream.getData(DeferredFileOutputStream.java:213)
org.apache.commons.fileupload.disk.DiskFileItem.getSize(DiskFileItem.java:289)
org.apache.jsp.Upload_jsp._jspService(Upload_jsp.java:159)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:390)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:334)
javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
note The full stack trace of the root cause is available in the Apache Tomcat/7.0.52 (Ubuntu) logs.
Apache Tomcat/7.0.52 (Ubuntu)
令人惊讶的是，catalina.out 文件中没有关于此异常的任何内容。
提前致谢。
编辑 Upload.jsp 中 DiskFileItem 的代码
//necessary imports go here
File file ;
int maxFileSize = 1000 * 1000 * 1024;
int maxMemSize = 1000 * 1024;
ServletContext context = pageContext.getServletContext();
String filePath = context.getInitParameter("file-upload");
String contentType = request.getContentType();
if(contentType != null)
{
if ((contentType.indexOf("multipart/form-data") >= 0))
{
DiskFileItemFactory factory = new DiskFileItemFactory();
factory.setSizeThreshold(maxMemSize);
factory.setRepository(new File("/tmp/"));
ServletFileUpload upload = new ServletFileUpload(factory);
upload.setSizeMax( maxFileSize );
try{
List fileItems = upload.parseRequest(request);
Iterator i = fileItems.iterator();
while (i.hasNext ())
{
FileItem fi = (FileItem)i.next();
if ( !fi.isFormField () )
{
String fieldName = fi.getFieldName();
String fileName = fi.getName();
if(fileName.endsWith(".zip")||fileName.endsWith(".pdf")||fileName.endsWith(".doc")||fileName.endsWith(".docx")||fileName.endsWith(".ppt")||fileName.endsWith(".pptx")||fileName.endsWith(".html")||fileName.endsWith(".htm")||fileName.endsWith(".epub")||fileName.endsWith(".djvu"))
{
boolean isInMemory = fi.isInMemory();
long sizeInBytes = fi.getSize();
new File(filePath+fileName).mkdir();
filePath = filePath+fileName+"/";
file = new File( filePath + fileName.substring( fileName.lastIndexOf("/"))) ;
fi.write(file);
String fileExtension = FilenameUtils.getExtension(fileName);
if(fileExtension.equals("zip"))
{
System.out.println("In zip.");
unzip mfe = new unzip();
mfe.unzipFile(filePath+fileName,filePath);
File zip = new File(filePath+fileName);
zip.delete();
}
File corePath = new File(filePath);
int count=0;
//some more processing
}
}
}
}
catch(Exception e)
{
//exception handling goes here
}
}
}
【问题讨论】：
-
看来您使用的是 Java 7。Java 8 自行处理此类问题，无需用户进行任何额外配置。
-
您应该真正处理内部循环中的异常。如果发生不好的事情，您甚至不会关闭文件
-
您正在尝试处理大文件，因此出现此类内存错误。
-
@mmc18 处理大文件可能会消耗大量内存；但它不一定必须。！这在很大程度上取决于 what 和 how 事情是如何完成的。长话短说-您的评论没有帮助；除此之外：做出如此大胆的声明根本不是基于事实。