【问题标题】:Retry test case using TestNG while using ITestContext with test method在使用带有测试方法的 ITestContext 时使用 TestNG 重试测试用例
【发布时间】:2017-08-05 21:30:58
【问题描述】:

我正在使用 dataprovider 方法和一个测试方法(在测试方法中带有 ITestContext 参数)。一个简化的例子如下:

@DataProvider(name="Dataprovider")
public Object[][] dataprovider(){

    return new Object[][]{{1},{2,},{3}};
}

@Test(dataProvider="Dataprovider")
public void test(int data, ITestContext itx){

    System.out.println(data);
    org.testng.Assert.assertEquals(data, 3);
}

我的 Retry 类和 RetryListener 类如下:

public class RetryListener implements IAnnotationTransformer {

@Override
public void transform(ITestAnnotation testannotation, Class testClass,
        Constructor testConstructor, Method testMethod) {

    IRetryAnalyzer retry = testannotation.getRetryAnalyzer();

    if (retry == null)  {
        testannotation.setRetryAnalyzer(Retry.class);
    }
}
}

public class Retry implements IRetryAnalyzer {

private static int retryCount = 0;
private int maxRetryCount = 1;

// Below method returns 'true' if the test method has to be retried else 'false' 
//and it takes the 'Result' as parameter of the test method that just ran
    public boolean retry(ITestResult result) {
        if (retryCount < maxRetryCount) {
            System.out.println("Retrying test " + result.getName() + " with status "
                    + getResultStatusName(result.getStatus()) + " for the " + (retryCount+1) + " time(s).");
            retryCount++;
            return true;
        }
        retryCount = 0;
        return false;
    }

    public String getResultStatusName(int status) {
        String resultName = null;
        if(status==1)
            resultName = "SUCCESS";
        if(status==2)
            resultName = "FAILURE";
        if(status==3)
            resultName = "SKIP";
        return resultName;
    }

}

预期:当测试失败时,TestNG调用重试,然后数据提供者应该返回相同的值给测试方法。

已观察到:Dataprovider 返回相同的值,但测试方法未运行,重试终止并开始下一个测试(dataprovider 现在将返回新值)

但是我的重试没有进入测试方法(它不期望 (int data, ITestContext itx) 用于测试方法)。如果我删除 ITestContext,则重试有效。

ITestContext 是维护测试用例上下文的必要条件。那么如何执行重试以及将 ITestContext 保留在测试方法中。

【问题讨论】:

  • "我的重试没有进入测试方法"->什么意思?
  • 当retry被调用时,数据提供者被再次调用以相同的值返回到测试方法,这次它不运行测试方法,测试只是通过(重试通过)和进入下一个测试
  • 由于附加参数ITestContext itx,目前无法编译。
  • @GrzegorzGórkiewicz 你的意思是,它不会“运行”正确吗?
  • 好的,所以没有办法做到这一点吗?我必须删除 ITestContext 才能运行测试方法,是吗?

标签: java testing automation testng


【解决方案1】:

在由数据提供者提供支持的@Test 方法中,我认为很难将ITestContext 作为显式参数并期望它由TestNG 注入,因为TestNG 不知道如何注入它(尤其是在哪个位置)。

请查看以下内容是否适合您(如果您注意到,我现在还明确地通过数据提供者传递 ITestContext

import org.testng.IRetryAnalyzer;
import org.testng.ITestContext;
import org.testng.ITestResult;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class RetryWithDataProvider {
    @DataProvider(name = "Dataprovider")
    public Object[][] dataprovider(ITestContext ctx) {

        return new Object[][]{{1, ctx}, {2, ctx}, {3, ctx}};
    }

    @Test(dataProvider = "Dataprovider", retryAnalyzer = Retry.class)
    public void test(int data, ITestContext itx) {
        System.out.println(data);
        System.out.println("Current Test name is : " + itx.getName());
        org.testng.Assert.assertEquals(data, 3);
    }

    public static class Retry implements IRetryAnalyzer {

        private static int retryCount = 0;
        private int maxRetryCount = 1;

        public boolean retry(ITestResult result) {
            if (retryCount < maxRetryCount) {
                System.out.println("Retrying test " + result.getName() + " with status "
                        + getResultStatusName(result.getStatus()) + " for the " + (retryCount + 1) + " time(s).");
                retryCount++;
                return true;
            }
            retryCount = 0;
            return false;
        }

        String getResultStatusName(int status) {
            String resultName = null;
            if (status == 1)
                resultName = "SUCCESS";
            if (status == 2)
                resultName = "FAILURE";
            if (status == 3)
                resultName = "SKIP";
            return resultName;
        }
    }
}

这是我的控制台输出

[TestNG] Running:
  /Users/krmahadevan/Library/Caches/IntelliJIdea2016.3/temp-testng-customsuite.xml
1
Retrying test test with status FAILURE for the 1 time(s).

Test ignored.
1

java.lang.AssertionError: expected [3] but found [1]
Expected :3
Actual   :1
 <Click to see difference>


    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:513)
    at org.testng.Assert.assertEqualsImpl(Assert.java:135)
    at org.testng.Assert.assertEquals(Assert.java:116)
    at org.testng.Assert.assertEquals(Assert.java:389)
    at org.testng.Assert.assertEquals(Assert.java:399)
    at com.rationaleemotions.stackoverflow.RetryWithDataProvider.test(RetryWithDataProvider.java:20)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
    at org.testng.internal.Invoker.retryFailed(Invoker.java:998)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1200)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at org.testng.TestRunner.privateRun(TestRunner.java:756)
    at org.testng.TestRunner.run(TestRunner.java:610)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
    at org.testng.SuiteRunner.run(SuiteRunner.java:289)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
    at org.testng.TestNG.runSuites(TestNG.java:1133)
    at org.testng.TestNG.run(TestNG.java:1104)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:127)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

2
Retrying test test with status FAILURE for the 1 time(s).

Test ignored.
2

java.lang.AssertionError: expected [3] but found [2]
Expected :3
Actual   :2
 <Click to see difference>


    at org.testng.Assert.fail(Assert.java:94)
    at org.testng.Assert.failNotEquals(Assert.java:513)
    at org.testng.Assert.assertEqualsImpl(Assert.java:135)
    at org.testng.Assert.assertEquals(Assert.java:116)
    at org.testng.Assert.assertEquals(Assert.java:389)
    at org.testng.Assert.assertEquals(Assert.java:399)
    at com.rationaleemotions.stackoverflow.RetryWithDataProvider.test(RetryWithDataProvider.java:20)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
    at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
    at org.testng.internal.Invoker.retryFailed(Invoker.java:998)
    at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1200)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
    at org.testng.TestRunner.privateRun(TestRunner.java:756)
    at org.testng.TestRunner.run(TestRunner.java:610)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
    at org.testng.SuiteRunner.run(SuiteRunner.java:289)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1293)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1218)
    at org.testng.TestNG.runSuites(TestNG.java:1133)
    at org.testng.TestNG.run(TestNG.java:1104)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:127)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)

3

===============================================
Default Suite
Total tests run: 5, Failures: 2, Skips: 2
===============================================

【讨论】:

  • 感谢您的回复。看起来不错。
猜你喜欢
  • 2018-12-31
  • 1970-01-01
  • 2016-12-05
  • 1970-01-01
  • 2015-08-14
  • 2021-06-13
  • 1970-01-01
  • 2020-01-21
  • 1970-01-01
相关资源
最近更新 更多