由于参数无效，SPRING XD -SQOOP 作业失败

Question

我们正在尝试使用 sqoop 作业作为 spring xd 的模块将数据从 sql server 摄取到 hdfs。

当我尝试使用 xd - shell of spring xd 工具运行以下命令时：

sqoop --command=import --args='--connect 'jdbc:sqlserver://localhost/mydb'  
--username root --password p@ss --query select * from tablename  
--target-dir /path/to/hdfs/file'

从 spring xd shell 作为 sqoop 作业启动作业时出现以下错误：

sqoop.errors

Try --help for usage instructions.
Exception in thread "main" java.lang.RuntimeException: Sqoop failed - return code 1
at org.springframework.xd.sqoop.SqoopRunner.main(SqoopRunner.java:81)

更多错误

19:38:40,556 ERROR main tool.BaseSqoopTool - Error parsing arguments for import:
19:38:40,556 ERROR main tool.BaseSqoopTool - Unrecognized argument: '--connect
19:38:40,556 ERROR main tool.BaseSqoopTool - Unrecognized argument: 'jdbc:sqlserver://localhost/mydb'
19:38:40,556 ERROR main tool.BaseSqoopTool - Unrecognized argument: --username
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: root
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: --password
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: p@ss
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: --query
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: select
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: *
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: from
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: tablename
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: --target-dir
19:38:40,557 ERROR main tool.BaseSqoopTool - Unrecognized argument: /path/to/hdfs/file'  

退出说明：

java.lang.IllegalStateException: Step execution failed - Exception in thread "main" java.lang.RuntimeException: Sqoop failed - return code 1
at org.springframework.batch.step.tasklet.x.AbstractProcessBuilderTasklet.execute(AbstractProcessBuilderTasklet.java:209)

我也尝试将= 放在参数之后，但得到了相同的结果。

为什么？

Answer 1

您在单引号内有单引号，这会将您的命令拆分为

sqoop --command=import --args='--connect '
jdbc:sqlserver://localhost/mydb
'--username root --password p@ss --query select * from tablename --target-dir /path/to/hdfs/file'

由于 --connect 本身不是一个有效的参数，因此您会遇到错误。

您可以转义单引号，如下所示：sqoop --command=import --args='--connect \'jdbc:sqlserver://localhost/mydb\' --username root --password p@ss --query select * from tablename --target-dir /path/to/hdfs/file'

或者您也应该能够在 args 部分周围使用双引号而不是单引号，例如：sqoop --command=import --args="--connect 'jdbc:sqlserver://localhost/mydb' --username root --password p@ss --query select * from tablename --target-dir /path/to/hdfs/file"