多个 Spring Batch 作业同时执行导致死锁

Question

我正在使用 spring batch 4.0.0、spring boot 2.2.0、java jdk 12.0.2、db sql server 2016。

并发执行的多个 Spring Batch 作业导致 Spring Batch 元数据表中的死锁。

我尝试了不同的解决方案：

1. 我在元数据表上设置了身份

ALTER TABLE [OWN].[BATCH_JOB_EXECUTION_SEQ]
ADD CONSTRAINT [PK_BATCH_JOB_EXECUTION_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

ALTER TABLE [OWN].[BATCH_JOB_SEQ]
ADD CONSTRAINT [PK_BATCH_JOB_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

ALTER TABLE [OWN].[BATCH_STEP_EXECUTION_SEQ]
ADD CONSTRAINT [PK_BATCH_STEP_EXECUTION_SEQ] PRIMARY KEY CLUSTERED ([ID] ASC);

1. 我确实覆盖了 JobRepository 和 JobExplorer 来配置隔离级别

    @Override
    protected JobRepository createJobRepository() throws Exception {
        JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
        factory.setDatabaseType(DATABASE_TYPE);
        factory.setIsolationLevelForCreate(ISOLATION_REPEATABLE_READ);
        factory.setDataSource(dataSource);
        factory.setTransactionManager(getTransactionManager());
        factory.setTablePrefix(TABLE_PREFIX);
        factory.afterPropertiesSet();
        return factory.getObject();
    }

    @Override
    protected JobExplorer createJobExplorer() throws Exception {
        JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
        jobExplorerFactoryBean.setDataSource(dataSource);
        jobExplorerFactoryBean.afterPropertiesSet();
        jobExplorerFactoryBean.setTablePrefix(TABLE_PREFIX);
        return jobExplorerFactoryBean.getObject();
    }

但是当我开始多份工作时，我经常/总是遇到同样的错误：

无法增加身份；嵌套异常是 com.microsoft.sqlserver.jdbc.SQLServerException: Transaction was deadlocked on lock resources with another process，并被选为死锁牺牲品。重新运行事务。

Answer 1

我找到了解决方案并将其发布给遇到相同问题的任何人；我从来没有找到一个完整而全面的解决方案，所以我想解释一下如何解决它以及它为什么会发生，让我们开始吧。 此死锁是由 BATCH_STEP_EXECUTION_SEQ、BATCH_JOB_EXECUTION_SEQ 和 BATCH_JOB_SEQ 表的更新和删除引起的，但为什么会发生呢？ 也许是因为事情没有得到妥善管理，但真正的问题是 sql server 开始使用 sql server 2012 的序列，所以 spring batch 不得不使用这个表来管理序列，我没有为那些使用旧的人提供解决方案sql server 版本低于 2012，本指南仅适用于 2012 年以后的版本。

1.在您的数据库中创建序列。

请注意从尚未处理的 ID 开始序列，否则如果从 1 开始，则必须清理使用作业 ID 的弹簧表和可能的表。

DROP TABLE [OWN].[BATCH_JOB_SEQ]
GO


DROP TABLE [OWN].[BATCH_JOB_EXECUTION_SEQ]
GO


DROP TABLE [OWN].[BATCH_STEP_EXECUTION_SEQ]
GO


CREATE SEQUENCE [OWN].[BATCH_JOB_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO


CREATE SEQUENCE [OWN].[BATCH_JOB_EXECUTION_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO

CREATE SEQUENCE [OWN].[BATCH_STEP_EXECUTION_SEQ] 
 AS [bigint]
 START WITH 1000
 INCREMENT BY 1
 MINVALUE -9223372036854775808
 MAXVALUE 9223372036854775807
 CACHE 
GO

2。也覆盖 JobRepository 方法和 JobExplorer（我个人必须添加它，否则我会遇到关于 spring 批处理表前缀的问题）。

将隔离级别设置为 REPEATABLE_READ，否则您将在 DB 上遇到新的并发错误（我个人在 BATCH_JOB_EXECUTION 或 BATCH_JOB_INSTANCE 上遇到问题，我现在不记得了）。

@Component
public class SQLServerConfig extends DefaultBatchConfigurer{

  @Autowired
  private DataSource dataSource;

  @Value("${spring.batch.tablePrefix}")
  private String tablePrefix;

  @Override
  protected JobRepository createJobRepository() throws Exception {
      JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
      factory.setDatabaseType(DatabaseType.SQLSERVER.name());
      factory.setDataSource(dataSource);
      factory.setTransactionManager(getTransactionManager());
      factory.setTablePrefix(tablePrefix);
      factory.setIncrementerFactory(new CustomDataFieldMaxValueIncrementerFactory(dataSource));
      factory.setIsolationLevelForCreate("ISOLATION_REPEATABLE_READ");
      factory.afterPropertiesSet();
      return factory.getObject();
  }

  @Override
  protected JobExplorer createJobExplorer() throws Exception {
      JobExplorerFactoryBean jobExplorerFactoryBean = new JobExplorerFactoryBean();
      jobExplorerFactoryBean.setDataSource(dataSource);
      jobExplorerFactoryBean.afterPropertiesSet();
      jobExplorerFactoryBean.setTablePrefix(tablePrefix);
      return jobExplorerFactoryBean.getObject();
  }

}

3.设置 CustomDataFieldIncrementerFactory 因为现在它开始从它管理序列。

 public class CustomDataFieldMaxValueIncrementerFactory extends DefaultDataFieldMaxValueIncrementerFactory {


  private DataSource dataSource;


  public CustomDataFieldMaxValueIncrementerFactory(DataSource dataSource) {
      super(dataSource);
      this.dataSource = dataSource;
  }


  @Override
  public DataFieldMaxValueIncrementer getIncrementer(String incrementerType, String incrementerName) {
      DatabaseType databaseType = DatabaseType.valueOf(incrementerType.toUpperCase());
      DataFieldMaxValueIncrementer dataFieldMaxValueIncrementer = null;
      if (databaseType == DatabaseType.SQLSERVER) {
          dataFieldMaxValueIncrementer =  new CustomSqlServerMaxValueIncrementer(dataSource, incrementerName);
      } else {
          dataFieldMaxValueIncrementer = super.getIncrementer(incrementerType, incrementerName);
      }
      return dataFieldMaxValueIncrementer;
  }
}

就是这样，现在运行良好！！！

我希望它对某人有用。