生成报告时通过jenkins Out of Memory错误运行Jmeter脚本

Question

我在通过 Jenkins 运行 Jmeter 脚本时遇到内存不足问题。该过程是在框架中编写代码，将生成的 XML 文件结果转换为 CSV，然后是 HTML 以在仪表板中发布报告。

我已经尝试将 Jenkins 服务器的堆空间从 32 GB 增加到 25 GB。似乎最初需要 1 Gb，但经过一段时间后它会抛出错误，即使堆中仍有 24 GB pf 内存可用，我为此运行 free -h。

还尝试通过增加 Jmeter 内存 设置 HEAP=-Xms1g -Xmx8g -XX:MaxMetaspaceSize=512m

脚本在 Windows 上的 Jnekins 服务器中执行良好。但是当 jenkin 服务器在 Linux 模式下时，它会抛出错误。

听到的是我的错误日志。

/var/lib/jenkins/workspace/ITT2_Execution/Resources/csvReportPath/ITT2_Unicast_Broker_Download_count.xml/var/lib/jenkins/workspace/ITT2_Execution/Resources//Configuration/transformGaurav.xsl/var/lib/jenkins/workspace/ITT2_Execution/jmeter_reports/ITT2_Unicast_Broker_Download_2_Oct_2019_19_3_52_Count.html{titleReport=ITT2_Unicast_Broker_DownloadCountReport, dateReport=2-Oct-2019 22:02:38}
Finished Parsing
/var/lib/jenkins/workspace/ITT2_Execution/Resources/csvReportPath/AutomationReport_5.2.4.2018.20_2_Oct_2019_19_3_52_count.xml/var/lib/jenkins/workspace/ITT2_Execution/Resources//Configuration/transformGaurav.xsl/var/lib/jenkins/workspace/ITT2_Execution/jmeter_reports/AutomationReport_5.2.4.2018.20_2_Oct_2019_19_3_52.html{titleReport=nullCountReport, dateReport=2-Oct-2019 22:02:39}
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2019/10/02 18:33:10 - please wait.
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2019/10/02 18:33:10 - please wait.
JVMDUMP032I JVM requested Heap dump using '/var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0001.phd' in response to an event
JVMDUMP010I Heap dump written to /var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0001.phd
JVMDUMP032I JVM requested System dump using '/var/lib/jenkins/workspace/ITT2_Execution/core.20191002.183310.40181.0002.dmp' in response to an event
JVMDUMP010I System dump written to /var/lib/jenkins/workspace/ITT2_Execution/core.20191002.183310.40181.0002.dmp
JVMDUMP032I JVM requested Java dump using '/var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0003.txt' in response to an event
JVMDUMP010I Java dump written to /var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0003.txt
JVMDUMP032I JVM requested Snap dump using '/var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0005.trc' in response to an event
JVMDUMP010I Snap dump written to /var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0005.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
JVMDUMP032I JVM requested Heap dump using '/var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0004.phd' in response to an event
JVMDUMP010I Heap dump written to /var/lib/jenkins/workspace/ITT2_Execution/heapdump.20191002.183310.40181.0004.phd
JVMDUMP032I JVM requested Java dump using '/var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0006.txt' in response to an event
JVMDUMP010I Java dump written to /var/lib/jenkins/workspace/ITT2_Execution/javacore.20191002.183310.40181.0006.txt
JVMDUMP032I JVM requested Snap dump using '/var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0007.trc' in response to an event
JVMDUMP010I Snap dump written to /var/lib/jenkins/workspace/ITT2_Execution/Snap.20191002.183310.40181.0007.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
[WARNING] 
java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
    at java.lang.reflect.Method.invoke(Method.java:508)
    at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)
    at java.lang.Thread.run(Thread.java:811)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at org.apache.xerces.xni.XMLString.toString(Unknown Source)
    at org.apache.xerces.parsers.AbstractDOMParser.characters(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanCharReference(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
    at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
    at utils.APIReportProcessing.fetchAPIReportDetailModuleWise(APIReportProcessing.java:110)
    at jmeterRun.RunProcess.prepareFinalResultsMerged(RunProcess.java:228)
    at jmeterRun.ControllerJMeter.main(ControllerJMeter.java:139)
    ... 6 more
[WARNING] Attempt to (de-)serialize anonymous class hudson.maven.reporters.MavenArtifactArchiver$2; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/
[WARNING] Attempt to (de-)serialize anonymous class hudson.maven.reporters.MavenFingerprinter$1; see: https://jenkins.io/redirect/serialization-of-anonymous-classes/

谢谢 比贝克

Answer 1

JMeter 进程在各种设置和参数下在 Java 虚拟机 (JVM) 中运行。 Java 堆空间（如错误消息中所指）是 JVM 从底层操作系统获取的内存，用于为创建必要的对象分配空间。

JMeter 的默认配置（参见 Windows 的 jmeter.bat 或非 Windows 系统脚本的 jmeter）假定堆空间仅为 512 兆字节。考虑到许多现代智能手机的数量高达四倍，这实际上相当低！如果您的测试运行超过 512Mb 的海量对象，您将收到 OOM 错误并且您的测试将失败。

幸运的是，有一个简单的解决方案。只需将最大堆大小增加到总可用物理 RAM 的约 80%。为此，请在 JMeter 启动脚本中找到以下行：

HEAP="-Xms1g -Xmx25g"

现在相应地更改 -Xmx 值。例如：如果要将最大堆大小设置为 25 GB，则需要将行更改为：

HEAP="-Xms1g -Xmx25g"

要应用更改，您需要重新启动 JMeter。

Answer 2

根据Understand the OutOfMemoryError Exception文章：

线程 thread_name 中的异常：java.lang.OutOfMemoryError：Java 堆空间

原因：详细信息Java heap space 表示无法在Java heap 中分配对象。此错误不一定意味着内存泄漏。问题可以像配置问题一样简单，其中指定的堆大小（或默认大小，如果未指定）对于应用程序来说是不够的。 在其他情况下，特别是对于长期存在的应用程序，该消息可能表明应用程序无意中持有对对象的引用，这可以防止对象被垃圾收集。这是内存泄漏的 Java 语言等价物。注意：应用程序调用的 API 也可能无意中持有对象引用。

此错误的另一个潜在来源是过度使用终结器的应用程序。如果一个类有一个 finalize 方法，那么该类型的对象在垃圾回收时不会回收它们的空间。相反，在垃圾回收之后，对象会排队等待最终确定，这将在稍后发生。在 Oracle Sun 实现中，终结器由为终结队列提供服务的守护线程执行。如果终结器线程无法跟上终结队列，那么 Java 堆可能会填满，并且会抛出这种类型的 OutOfMemoryError 异常。可能导致这种情况的一种情况是，当应用程序创建高优先级线程时，会导致终结队列的增长速度快于终结器线程为该队列提供服务的速度。

行动：您可以在Monitor the Objects Pending Finalization. 中找到有关如何监控待定对象的更多信息

所以使用Java Memory Map 工具来调查最大的对象以及它们属于什么进程。

另外，您似乎正在 Linux 或其他类 Unix 操作系统上运行 Jenkins，而 SET 命令是特定于 Windows 的。此外，Jenkins 不尊重HEAP 环境变量，而是使用JAVA_ARGS 和/或JENKINS_JAVA_OPTIONS。

对于 JMeter：

HEAP="-Xms1g -Xmx8g" && export HEAP

对于詹金斯：

JENKINS_JAVA_OPTIONS="-Xms1g -Xmx8g" && export JENKINS_JAVA_OPTIONS

更多信息：

• How to add Java arguments to Jenkins?
• 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure