【发布时间】:2026-02-07 00:30:01
【问题描述】:
在 Spring Boot 中,我尝试使用 Spring Batch 读取具有 1000 个条目的 JSON 文件,进行一些处理,然后将输出写入 CSV 文件。每当我使用 TaskExecutor 启用多线程步骤时,就会出现此问题。 我实现 TaskExecutor 的代码
@Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(2);
executor.setMaxPoolSize(2);
executor.setQueueCapacity(500);
executor.setThreadNamePrefix("GithubLookup-");
executor.initialize();
return executor;
}
我的步骤实现代码
@SuppressWarnings({ "unchecked" })
@Bean
public Step humanStep() throws MalformedURLException {
return stepBuilderFactory
.get("humanStep")
.<Human, Human>chunk(20)
.reader(jsonItemReader())
.processor(processor())
.writer(humanItemWriter())
.taskExecutor(taskExecutor())
.build();
}
我的 ItemReader 实现代码
@SuppressWarnings({ "rawtypes", "unchecked" })
@Bean
public JsonItemReader jsonItemReader() throws MalformedURLException {
Resource input_resource1 = new FileSystemResource(inputPath);
return new JsonItemReaderBuilder()
.jsonObjectReader(new JacksonJsonObjectReader(Human.class))
.resource(input_resource1)
.name("humanJsonItemReader")
.build();
}
我的JSON文件格式如下
[
{"id":1,"first_name":"Hendrika","last_name":"Glossop","email":"hglossop0@samsung.com","gender":"Female","car":"SALFR2BGXFH882836"},
{"id":2,"first_name":"Batsheva","last_name":"Drysdell","email":"bdrysdell1@example.com","gender":"Female","car":"2C3CA5CGXBH086890"},
{"id":3,"first_name":"Xenos","last_name":"Evered","email":"xevered2@independent.co.uk","gender":"Male","car":"2C3CCAEG5FH423486"},
{"id":4,"first_name":"Worthington","last_name":"Oleksinski","email":"woleksinski3@unicef.org","gender":"Male","car":"3VWML7AJ7EM658490"},
{"id":5,"first_name":"Dee","last_name":"Weston","email":"dweston4@tripod.com","gender":"Female","car":"WBA3B3C59FJ781374"},
.
.
.
]
抛出的错误如下:
org.springframework.batch.item.ParseException: Unable to read next JSON object
at org.springframework.batch.item.json.JacksonJsonObjectReader.read(JacksonJsonObjectReader.java:84) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.json.JsonItemReader.doRead(JsonItemReader.java:103) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.read(AbstractItemCountingItemStreamItemReader.java:93) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.doRead(SimpleChunkProvider.java:99) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.read(SimpleChunkProvider.java:180) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider$1.doInIteration(SimpleChunkProvider.java:126) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:375) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:145) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.SimpleChunkProvider.provide(SimpleChunkProvider.java:118) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:71) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:407) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:331) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140) ~[spring-tx-5.2.8.RELEASE.jar:5.2.8.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:273) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:82) ~[spring-batch-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.batch.repeat.support.TaskExecutorRepeatTemplate$ExecutingRunnable.run(TaskExecutorRepeatTemplate.java:262) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:1.8.0_261]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:1.8.0_261]
at java.lang.Thread.run(Unknown Source) ~[na:1.8.0_261]
Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('_' (code 95)): was expecting comma to separate Object entries
at [Source: (sun.nio.ch.ChannelInputStream); line: 1, column: 24]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1851) ~[jackson-core-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:707) ~[jackson-core-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:632) ~[jackson-core-2.11.2.jar:2.11.2]
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:741) ~[jackson-core-2.11.2.jar:2.11.2]
at org.springframework.batch.item.json.JacksonJsonObjectReader.read(JacksonJsonObjectReader.java:80) ~[spring-batch-infrastructure-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 19 common frames omitted
谁能告诉我我做错了什么?
【问题讨论】:
-
“意外字符('_'(代码 95)):期望用逗号分隔对象条目”听起来可能 JSON 格式不正确?
-
当我从 stepBuilder 中省略 taskExecutor 行时它可以工作。
-
我不熟悉 StepBuilder,但是,听起来它可能是跨线程共享阅读器。阅读器不能跨线程共享。第二个线程可能正在尝试从底层流中读取,而另一个线程也在读取并且可能会导致问题。在实践中,你不能真正并行化 json 解析(除非你有一个专门的线程从流中读取段,然后分发给可能进一步解析它的线程。这实际上可能会因为线程开销而变慢。)
标签: java json spring spring-boot spring-batch