【问题标题】:hadoop start-xxx.sh command not found and syntax errorhadoop start-xxx.sh 命令未找到且语法错误
【发布时间】:2019-06-11 12:56:38
【问题描述】:

我在我的 MacOS 上安装了 Hadoop。我都尝试按照 Hadoop 文档和 Brew 发布的方式进行操作。但每当我尝试运行start-dfs.shstart-yarn.shstart-all.sh 时,它只会抛出以下错误:

如果您不想点击屏幕截图的图像,终端的错误将在此处粘贴在文本中:

Starting resourcemanager
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token `<'
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: `  done < <(for text in "${input[@]}"; do'
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 70: hadoop_deprecate_envvar: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 87: hadoop_bootstrap: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 104: hadoop_parse_args: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 105: shift: : numeric argument required
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 265: hadoop_need_reexec: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 273: hadoop_verify_user_perm: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/yarn: line 232: hadoop_validate_classname: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/yarn: line 233: hadoop_exit_with_usage: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 293: hadoop_add_client_opts: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 300: hadoop_subcommand_opts: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 303: hadoop_generic_java_subcmd_handler: command not found
Starting nodemanagers
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: syntax error near unexpected token `<'
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-functions.sh: line 398: `  done < <(for text in "${input[@]}"; do'
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 70: hadoop_deprecate_envvar: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 87: hadoop_bootstrap: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 104: hadoop_parse_args: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 105: shift: : numeric argument required
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 265: hadoop_need_reexec: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/../libexec/hadoop-config.sh: line 273: hadoop_verify_user_perm: command not found
/usr/local/Cellar/hadoop/3.1.1/libexec/bin/yarn: line 232: hadoop_validate_classname: command not found

我已经完成了Hadoop tutorial的每一步。
而且我还尝试了在 Stackoverflow 上找到的解决方案,例如使用 bash 而不是 sh 来运行脚本和使用 sudo。

我的 MacOS 版本是:10.14
Hadoop版本是:3.1.1

我确实看到了similar question on stackoverflow
但是在那篇文章中没有回答任何可行的解决方案,提出这个问题的人说他已经解决了这个问题。我没有足够的声誉来评论和联系该用户 masterX。

【问题讨论】:

  • 在这里查看我的答案stackoverflow.com/a/52531871/2308683
  • 我检查了你的答案。我按照您的步骤操作,但在执行 start-dfs.sh 时仍然遇到同样的错误。你介意给我更多信息吗?
  • 因为你得到syntax error,这表明你没有从 Bash shell 运行命令。而command not found 表示这些脚本的$PATH 由于某种原因没有正确设置。不过,正如我在那里的回答所示,Brew 中安装了相同版本的 Hadoop,它对我有用……您可以做的是编辑脚本以启用调试 tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html 并查找它开始失败的位置。并且不要使用sh start-dfs.sh,你可以使用chmod +x start-dfs.sh,那么它只是来自那个目录的./start-dfs.sh
  • 你解决了吗?我遇到了同样的问题

标签: macos hadoop syntax-error


【解决方案1】:

导致该错误的原因可能有很多:

  1. 可能是由于 Hadoop 环境变量配置错误。试试export HADOOP_HOME=/usr/local/Cellar/hadoop/3.1.1/
  2. 可能需要权限:试试sudo chown YOURUSERNAME:YOURUSERNAME /usr/local/Cellar/hadoop/3.1.1/ -R

【讨论】:

    【解决方案2】:

    检查启动文件的权限。可能存在权限问题。 您是否为 Hadoop 作业创建了其他用户?

    【讨论】:

    • 我需要检查哪些权限?我是否必须添加其他用户以及如何添加?你介意给我更多提示吗?抱歉,我是 hadoop 的新手。
    猜你喜欢
    • 2019-06-20
    • 2014-07-21
    • 2018-08-19
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多