从 sFtp 服务器读取文件

Question

我正在从 sftp 服务器读取文件并使用 Spring Batch Framework 将该文件加载到数据库中，下面是我的代码，但我遇到了错误

代码：

<bean id="cvsFileItemReaderMeta" class="org.springframework.batch.item.file.FlatFileItemReader">
    <!-- Read a csv file -->

    <property name="resource" value= "ftp://scmuser:scmuser%40123@172.18.228.32:22/home/scmuser/CSV/Meta.csv" />

错误：

org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:142)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96)
    at org.springframework.batch.core.step.item.ChunkMonitor.open(ChunkMonitor.java:115)
    at org.springframework.batch.item.support.CompositeItemStream.open(CompositeItemStream.java:96)
    at org.springframework.batch.core.step.tasklet.TaskletStep.open(TaskletStep.java:306)
    at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:192)
    at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:137)
    at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64)
    at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60)
    at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:152)
    at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:131)
    at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:135)
    at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:301)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:134)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49)
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:127)
    at com.tcs.ceg.iforesee.service.DataLoader.loadMetaMapping(DataLoader.java:127)
    at com.tcs.ceg.iforesee.service.UploadFile.uploadFile(UploadFile.java:77)
    at com.tcs.ceg.iforesee.service.UploadFile.main(UploadFile.java:22)
Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): URL [ftp://username:password@hostname.com:22/home/scmuser/CSV/Meta.csv]
    at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:251)
    at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:139)

Answer 1

原因：java.lang.IllegalStateException：输入资源必须存在（阅读器处于“严格”模式）：URL [ftp://username:password@hostname.com:22/home/scmuser/CSV/Meta.csv]

显然该资源不存在。您要么必须在尝试打开资源之前确保资源存在，要么使用FlatFileItemReader#setStrict 取消设置阅读器的严格模式并传递false

在严格模式下，如果输入资源不存在，阅读器将在 AbstractItemCountingItemStreamItemReader.open(org.springframework.batch.item.ExecutionContext) 上抛出异常。

Answer 2

实现目标并更好地处理错误的另一种方法是将下载+导入分为 2 步：

1. 从 FTP 检索文件（您可以使用 dedicer/exit-status 处理文件丢失、文件下载不完整等）
2. 从本地导入文件，无需担心 ftp 传输错误（例如，在数据导入过程中连接断开）