【问题标题】:Integration test for Spring Batch failing because of thread execution由于线程执行,Spring Batch 的集成测试失败
【发布时间】:2021-06-20 11:52:39
【问题描述】:

我有一个 Spring 批处理项目。目标是对 Spring Job 中存在的各个步骤执行集成测试。

我正在使用 JobLauncherTestUtils 来启动该步骤。但是,当此实用程​​序启动这些步骤时,它会在单独的线程中运行它。线程完成执行后,应该为jobExecution.getStepExecutions() 分配一些值。

问题: 出于某种原因,甚至在线程完成执行之前,测试进入下一行List<StepExecution> actualStepExecutions = new ArrayList<>(jobExecution.getStepExecutions()),其中jobExecution.getStepExecutions() 当前为空,因此测试失败并显示@987654324 @ 和 Index 0 out of bounds for length 0 - 标记在下面的测试类中。

问题:有什么优雅的方法可以等待 Step 执行 THREAD 完成,然后进入测试中的下一行进行验证?

代码:

集成测试类:

@Slf4j
@SpringBatchTest
@SpringBootTest
@ActiveProfiles({"test", "master"})
@ContextConfiguration(classes = {InhouseClass3.class, InhouseClass1.class, InhouseClass2.class})
public class BatchJobIntegrationTest {

    private static final String Param1 = "someParam";

    @Autowired
    @Qualifier("hikariDatasource")
    DataSource hikariDatasource;

    @Autowired
    Job BatchJob;

    @Autowired
    JobLauncher jobLauncher;

    JobExecution jobExecution;

    @Autowired
    CreateDirectoryTasklet createDirectoryTasklet;

    JobParameters jobParameters;

    JobLauncherTestUtils jobLauncherTestUtils;

    @BeforeEach
    void setUp() {
        String startTimestamp = Timestamp.from(Instant.now()).toString();
        jobParameters = new JobParametersBuilder()
                .addString(Param1, startTimestamp)
                .toJobParameters();

        jobLauncherTestUtils = new JobLauncherTestUtils();
        jobLauncherTestUtils.setJob(BatchJob);
        jobLauncherTestUtils.setJobLauncher(jobLauncher);
    }


    @SneakyThrows
    @Test
    void TaskletToTest_Test() {
        jobExecution = jobLauncherTestUtils.launchStep("loadTaskletToTest", jobParameters);

        List<StepExecution> actualStepExecutions = new ArrayList<>(jobExecution.getStepExecutions());
        // ERROR: jobExecution.getStepExecutions() is NULL. 

        ExitStatus actualJobExitStatus = actualStepExecutions.get(0).getExitStatus();
        // ERROR: Index 0 out of bounds for length 0

        assertEquals("loadGdxClaims", actualStepExecutions.get(0).getStepName());
        assertEquals(ExitStatus.COMPLETED, actualJobExitStatus);
    }

    @SneakyThrows
    @Test
    // This is my workaround to make my above test run. 
    // I added a sleep for 2 seconds. But this doesn't look like an ideal way, coz what 
    // if the launchstep thread running the tasklet took more than 2 seconds? 
    void loadGDXClaimTaskletTest_Working_() {
        jobExecution = jobLauncherTestUtils.launchStep("createDirectory", jobParameters);
        boolean counter = true;
        while(counter) {
            if (jobExecution.getStepExecutions().size()!=0 ) {
                List<StepExecution> actualStepExecutions = new ArrayList<>(jobExecution.getStepExecutions());

                ExitStatus actualJobExitStatus = actualStepExecutions.get(0).getExitStatus();
                log.info("------- step executions : {}", actualStepExecutions);
                assertEquals("createDirectory", actualStepExecutions.get(0).getStepName());
                assertEquals(ExitStatus.COMPLETED, actualJobExitStatus);
                counter = false;
            } else {
                TimeUnit.SECONDS.sleep(2);
            }
        }
    }



}

Tasklet 测试步骤:

@Slf4j
@Component
public class TaskletToTest implements Tasklet {

    private final InhouseService inhouseService;

    public TaskletToTest(InhouseService inhouseService) {
        this.inhouseService = inhouseService;
    }

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws InterruptedException, IllegalJobNameException, JSchException, IOException {
        log.info("TaskletToTest before");
        inhouseService.retry();
        log.info("TaskletToTest after");
        return RepeatStatus.FINISHED;
    }
}

包含需要测试的步骤列表的批处理作业:

@Slf4j
@Profile("master")
@Configuration
public class MasterConfig extends DefaultBatchConfigurer {
    @Bean(name = "BatchJob")
    public Job remoteChunkingJob(TaskletStep someOtherTasklet,
                                 TaskletStep loadTaskletToTest,
                                 JobExecutionListener jobExecutionListener) {
        return this.jobBuilderFactory.get("extract gdx load")
                .incrementer(new RunIdIncrementer())     
                .listener(jobExecutionListener)
                .start(someOtherTasklet) 
                .next(loadTaskletToTest)
                .build();
    }

    @Bean
    TaskletStep loadTaskletToTest(TaskletToTest taskletToTest) {
        return this.stepBuilderFactory.get("loadTaskletToTest").tasklet(taskletToTest).build();
    }
}

【问题讨论】:

    标签: java spring spring-boot spring-batch


    【解决方案1】:

    我正在使用 JobLauncherTestUtils 来启动该步骤。然而,当这个工具启动这些步骤时,它会在一个单独的线程中运行它。

    问题:由于某种原因,甚至在线程完成执行之前,测试就进入下一行

    JobLauncherTestUtils 使用JobLauncher 来启动作业和步骤。因此,根据您使用的 JobLauncher 实现,您可以在当前线程或单独的线程中运行作业/步骤。

    您没有在测试中分享哪个JobLauncher 是自动装配的,但您似乎已经定义了一个基于异步TaskExecutor 实现的作业启动器。这就是您的工作/步骤在后台执行的原因:

    // This returns immediately with an asynchrnous TaskExecutor
    // However, with a synchronous TaskExectuor it will block waiting for the step to finish 
    jobExecution = jobLauncherTestUtils.launchStep("loadTaskletToTest", jobParameters);
    
    List<StepExecution> actualStepExecutions = new ArrayList<>(jobExecution.getStepExecutions());
            
    

    因此,您需要检查哪个JobLauncher(通常是带有同步或异步TaskExecutorSimpleJobLauncher)在您的测试中自动连接。

    【讨论】:

    • 在这种情况下如何使 TaskExecutor 同步?
    • 您需要在您的JobLauncher 上设置一个同步的TaskExecutor 实现,例如默认使用的SyncTaskExecutor,请参阅此处的Javadocs:docs.spring.io/spring-batch/docs/4.3.x/api/org/springframework/…。您正在测试类中自动装配 JobLauncher,但确实显示了它是如何创建的以及它是在哪个上下文中定义的。所以你需要确保注入的 JobLauncher 是基于同步任务执行器的。
    • 我相信我已经回答了你的问题,所以请接受它:stackoverflow.com/help/someone-answers(如果你没有足够的 stackoverflow 声誉,请支持它)。
    猜你喜欢
    • 2020-06-09
    • 1970-01-01
    • 1970-01-01
    • 2018-09-27
    • 2015-03-24
    • 2019-12-26
    • 2017-07-02
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多