使用 Spring Batch JpaItemWriter 写入数据不完整答案

【问题标题】：Writing Data using Spring Batch JpaItemWriter incomplete使用 Spring Batch JpaItemWriter 写入数据不完整
【发布时间】：2016-08-02 15:49:31
【问题描述】：

我正在尝试使用 Spring Batch (3.0.6) 执行一些数据维护操作。我正在从 JPA（Hibernate 5.0.1 实现）读取数据，对其进行更改并尝试再次使用 JPA 将其保存回数据库。

作业执行成功，但是，在查看了我看到的数据库之后，实际上只有少数条目被更改（所有条目都应该被更改）。似乎数据未正确刷新/事务未提交，日志显示所有项目都在适当的处理器和适当的写入器中处理。

可能我在 transactionManager 方面遗漏了一些东西，但我很难弄清楚，因为工作环境对我来说看起来很理智：

<beans xmlns="http://www.springframework.org/schema/beans" 
    xmlns:batch="http://www.springframework.org/schema/batch" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-3.0.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd">

    <bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />

    <bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
        <property name="jobRepository" ref="jobRepository" />
    </bean>

    <bean id="jobListener" class="batch.enricher.JobListener" />

    <job id="hibernateJob" restartable="false"
    xmlns="http://www.springframework.org/schema/batch">
        <step id="step1">
            <tasklet transaction-manager="transactionManager">
                <chunk reader="hibernateItemReader" processor="enrichmentProcessor"
                writer="entityWriter" commit-interval="50">
                </chunk>
            </tasklet>
        </step>
        <batch:listeners>
            <batch:listener ref="jobListener" />
        </batch:listeners>
    </job>

    <bean id="entityManagerFactoryRegistry" class="database.EntityManagerFactoryRegistry" />

    <bean id="enrichmentProcessor" class="batch.enricher.EnrichmentProcessor" />

    <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
        <property name="entityManagerFactory" value="#{entityManagerFactoryRegistry.emf}" />
    </bean>

    <bean id="entityWriter" class="batch.enricher.EntityWriter">
        <property name="entityManagerFactory" value="#{entityManagerFactoryRegistry.emf}" />
    </bean>

    <bean id="hibernateItemReader"
    class="org.springframework.batch.item.database.JpaPagingItemReader">
        <property name="queryString" value="from Relation" />
        <property name="entityManagerFactory" value="#{entityManagerFactoryRegistry.emf}" />
    </bean>
</beans>

我猜这些实现再简单不过了：

public class EntityWriter extends JpaItemWriter<Relation> {
}

public class EnrichmentProcessor implements ItemProcessor<Relation, Relation>
{
    @Override
    public Relation process(Relation item) throws Exception
    {
        // Do something with EACH item...
        // Logging revealed that all items are correctly processed
        return item;
    }
}

public class JobListener implements JobExecutionListener
{
    @Override
    public void beforeJob(JobExecution jobExecution)
    {
        // Perform some trivial initializing...
    }

    @Override
    public void afterJob(JobExecution jobExecution)
    {
    }
}

我已经尝试覆盖EntityWriter（doWrite and write）中适当的写入方法，但没有显着成功。我正在使用CommandLineJobRunner 运行代码。

编辑：对于总共 64 项，前 60 项已正确更新，使用提交间隔和页面大小 10。我对日志进行了 grep 以仅包含有趣的语句。我只包含了最后两个页面读取（第 50 - 64 项）

[DEBUG] 2016-04-13 11:29:54.753 [main] JpaTransactionManager - Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.753 [main] JpaTransactionManager - Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@377a58f6] for JPA transaction
[DEBUG] 2016-04-13 11:29:54.754 [main] JpaPagingItemReader - Reading page 5
[DEBUG] 2016-04-13 11:29:54.831 [main] JpaItemWriter - Writing to JPA with 10 items.
[DEBUG] 2016-04-13 11:29:54.889 [main] JpaItemWriter - 10 entities merged.
[DEBUG] 2016-04-13 11:29:54.889 [main] JpaItemWriter - 0 entities found in persistence context.
[DEBUG] 2016-04-13 11:29:54.890 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.892 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.892 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@3d7c7168]
[DEBUG] 2016-04-13 11:29:54.892 [main] ResourcelessTransactionManager - Resuming suspended transaction after completion of inner transaction
[DEBUG] 2016-04-13 11:29:54.892 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.898 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.899 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@337df06d]
[DEBUG] 2016-04-13 11:29:54.899 [main] ResourcelessTransactionManager - Resuming suspended transaction after completion of inner transaction
[DEBUG] 2016-04-13 11:29:54.899 [main] JpaTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.899 [main] JpaTransactionManager - Committing JPA transaction on EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@377a58f6]
[DEBUG] 2016-04-13 11:29:54.900 [main] JpaTransactionManager - Closing JPA EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@377a58f6] after transaction
[DEBUG] 2016-04-13 11:29:54.901 [main] JpaTransactionManager - Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.902 [main] JpaTransactionManager - Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@4bc82040] for JPA transaction
[DEBUG] 2016-04-13 11:29:54.904 [main] JpaPagingItemReader - Reading page 6
[DEBUG] 2016-04-13 11:29:54.959 [main] JpaItemWriter - Writing to JPA with 4 items.
[DEBUG] 2016-04-13 11:29:54.986 [main] JpaItemWriter - 4 entities merged.
[DEBUG] 2016-04-13 11:29:54.986 [main] JpaItemWriter - 0 entities found in persistence context.
[DEBUG] 2016-04-13 11:29:54.987 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.988 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.988 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@1d8fdd4f]
[DEBUG] 2016-04-13 11:29:54.988 [main] ResourcelessTransactionManager - Resuming suspended transaction after completion of inner transaction
[DEBUG] 2016-04-13 11:29:54.988 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.991 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.991 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@1b446bed]
[DEBUG] 2016-04-13 11:29:54.992 [main] ResourcelessTransactionManager - Resuming suspended transaction after completion of inner transaction
[DEBUG] 2016-04-13 11:29:54.992 [main] JpaTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.992 [main] JpaTransactionManager - Committing JPA transaction on EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@4bc82040]
[DEBUG] 2016-04-13 11:29:54.994 [main] JpaTransactionManager - Closing JPA EntityManager [org.hibernate.jpa.internal.EntityManagerImpl@4bc82040] after transaction
[DEBUG] 2016-04-13 11:29:54.995 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:54.996 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:54.997 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@5c2dd133]
[DEBUG] 2016-04-13 11:29:54.997 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:55.002 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:55.002 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@2bb9da60]
[DEBUG] 2016-04-13 11:29:55.003 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.updateExecutionContext]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:55.005 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:55.005 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@358cce2e]
[DEBUG] 2016-04-13 11:29:55.008 [main] ResourcelessTransactionManager - Creating new transaction with name [org.springframework.batch.core.repository.support.SimpleJobRepository.update]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
[DEBUG] 2016-04-13 11:29:55.012 [main] ResourcelessTransactionManager - Initiating transaction commit
[DEBUG] 2016-04-13 11:29:55.012 [main] ResourcelessTransactionManager - Committing resourceless transaction on [org.springframework.batch.support.transaction.ResourcelessTransactionManager$ResourcelessTransaction@5875c7d8]

【问题讨论】：

标签： jpa transactions spring-batch spring-data-jpa

【解决方案1】：

您是否在事务中调用JpaItemWriter.write(List)？

ItemWriter 使用 JPA EntityManagerFactory 来合并不属于持久性上下文的任何实体。 需要在事务内部调用 write(List)。阅读器必须配置有能够参与 Spring 托管事务的 EntityManagerFactory。写入器在设置属性后是线程安全的（正常的单例行为），因此它可以用于写入多个并发事务。

Reference

【讨论】：

看看我最近的编辑。我添加了适当的日志语句，表明使用了事务。