Java GC Overhead Limit Error Using JUnit 5 TestFactory with large stream of dynamicTest (DynamicContainer / DynamicNode)

Question

我在尝试使用 maven-surefire-plugin 2.21.0 创建和运行大量 JUnit 5 动态测试时遇到以下问题

[ERROR] 超出 GC 开销限制 [错误] org.apache.maven.surefire.booter.SurefireBooterForkException：分叉进程中出现错误 [错误] 超出 GC 开销限制 [错误] 在 org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:673) [错误] 在 org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:535) [错误] 在 org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:280) [错误] 在 org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245) [错误] 在 org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1124) [错误] 在 org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:954) [错误] 在 org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:832) [错误] 在 org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) [错误] 在 org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) [错误] 在 org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154) [错误] 在 org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146) [错误] 在 org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117) [错误] 在 org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81) [错误] 在 org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) [错误] 在 org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

下面是我的代码，它是一个数据库比较工具。基本上我正在做的是，从两个数据库流式传输数据并比较记录，如果两条记录中的任何一条存在差异，则测试失败，因此将在数据库中创建尽可能多的动态测试（~14M），相当大，不确定是否要在这种规模上使用动态测试。非常感谢这方面的任何帮助:)

 @TestFactory
Stream<DynamicNode> dynamicTestsWithContainers() throws SQLException {
    return tableNameProvider()//List of db tables ~100
            .map(tableName -> dynamicContainer(tableName, dynamicNodeStream(tableName)));
}

private Stream<DynamicNode> dynamicNodeStream(String tableName) {

    try {
        System.out.println("Testing " + tableName);
        Stream<Row> rows = sourceRepo.rows(tableName);
        Stream<List<Row>> batchRows = batch(rows, 10000);
        Optional<TableSchema> tableInfo = sourceRepo.getTableSchema(tableName);


        final Stream<DynamicNode> dynamicNodeStream = batchRows
                .flatMap(batch -> {

                    Map<String, Row> sourceRowsMap = buildRowMap(batch, tableInfo.get());//HashMap with 10000 Objects
                    Map<String, Row> targetRowsMap = targetRepo.getTargetDBRows(sourceRowsMap, tableInfo.get());//HashMap with 10000 Objects
                    Set<String> commonKeys = Sets.intersection(sourceRowsMap.keySet(), targetRowsMap.keySet());

                    final Stream<DynamicTest> dynamicTestStream = Streams.concat(
                            Stream.of(
                                    dynamicTest("All source records should be present in target DB", () -> assertThat(targetRowsMap.keySet())
                                            .as("Comparing " + sourceRepo.getServerName() + " against " + targetRepo.getServerName())
                                            .hasSameElementsAs(sourceRowsMap.keySet()))
                            ),
                            commonKeys
                                    .stream()
                                    .map(rowKey -> dynamicTest(tableName + " Row with #" + rowKey + " should be same in target DB",
                                            () -> assertThat(targetRowsMap.get(rowKey).getRowData())
                                                    .isEqualToComparingFieldByFieldRecursively(sourceRowsMap.get(rowKey).getRowData())
                                    ))

                    );

                    return dynamicTestStream;
                });

        return dynamicNodeStream;
    } catch (Exception e) {
        throw new RuntimeException("Error running tests on table " + tableName, e);
    }
}

Answer 1

试试：

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <argLine>-XX:+UseConcMarkSweepGC</argLine>
            </configuration>
        </plugin>

在您的构建配置文件中允许并发垃圾收集。