当测试无法运行时，确保 Maven 构建失败

Question

我注意到，有时在 Jenkins 上运行 maven 构建时，运行的 Jbehave 测试的数量因运行而异。在分析日志时，我看到以下 sn-p：

    Failed to run story stories/cancel.story
java.lang.InterruptedException: stories/cancel.story
    at org.jbehave.core.embedder.StoryRunner$RunContext.interruptIfCancelled(StoryRunner.java:616)
    at org.jbehave.core.embedder.StoryRunner.runStepsWhileKeepingState(StoryRunner.java:514)
    at org.jbehave.core.embedder.StoryRunner.runScenarioSteps(StoryRunner.java:479)
    at org.jbehave.core.embedder.StoryRunner.runStepsWithLifecycle(StoryRunner.java:445)
    at org.jbehave.core.embedder.StoryRunner.runCancellable(StoryRunner.java:305)
    at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:220)
    at org.jbehave.core.embedder.StoryRunner.run(StoryRunner.java:181)
    at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:235)
    at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:207)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

问题是，当测试被跳过或以这种方式运行失败时，构建仍然被认为是成功的。

是否有一个 maven surefire 插件配置可以确保每当测试失败时运行构建都会导致失败？以下是 maven surefire 构建配置

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.11</version>
            <configuration>
                <skip>true</skip>
            </configuration>
        </plugin>
        <plugin>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.11</version>
            <configuration>
                <includes>
                    <include>**/*TestSuite.java</include>
                </includes>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>integration-test</goal>
                        <goal>verify</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-javadoc-plugin</artifactId>
            <version>2.9</version>
        </plugin>
        <plugin>
            <groupId>net.thucydides.maven.plugins</groupId>
            <artifactId>maven-thucydides-plugin</artifactId>
            <version>${thucydides.version}</version>
            <executions>
                <execution>
                    <id>thucydides-reports</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>aggregate</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <dependency>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                    <version>1.2.17</version>
                </dependency>
            </dependencies>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.2</version>
            <configuration>
                <reportPlugins>
                    <plugin>
                        <groupId>net.thucydides.maven.plugins</groupId>
                        <artifactId>maven-thucydides-plugin</artifactId>
                        <version>${thucydides.version}</version>
                    </plugin>
                </reportPlugins>
            </configuration>
        </plugin>
    </plugins>
</build>

Answer 1

您的maven-surefire-plugin 设置为完全跳过测试（使用<skip>true</skip>），因此使用maven-failsafe-plugin 运行测试。该插件应该不会在integration-test 期间因失败而停止，然后仅在verify 阶段失败。

所以如果你真的想回答这个问题：

是否有一个 maven surefire 插件配置可以确保每当测试失败时运行构建都会导致失败？

即：您希望maven-surefire-plugin 运行测试，而不是maven-failsafe-plugin，那么答案是：删除

        <configuration>
            <skip>true</skip>
        </configuration>

来自您的 POM。在这种情况下，您也不需要maven-failsafe-plugin 配置，因为它只会让您的测试运行两次。

但如果您的目标是让maven-failsafe-plugin 工作，那么我认为您可能会遇到以下问题之一：

1. 没有实现正确的目标。作为插件help states，你应该调用它

   mvn verify
   

2. 一个老插件，与你正在使用的测试框架不兼容（当前版本为2.19.1）

3. 或者这个帮助推荐：

   对于非常复杂的构建，最好将集成测试和验证目标的执行分开：

   <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-failsafe-plugin</artifactId>
     <version>2.19.1</version>
     <executions>
       <execution>
         <id>integration-test</id>
         <goals>
           <goal>integration-test</goal>
         </goals>
       </execution>
       <execution>
         <id>verify</id>
         <goals>
           <goal>verify</goal>
         </goals>
       </execution>
     </executions>
   </plugin>