作业跟踪器未运行

Question

我在本地机器上安装了单节点 hadoop。除作业跟踪器外，所有进程都在运行。我想从 Ozzie 运行我的工作，因此我需要运行 Job Tracker。

JPS 输出

12545 Jps
12371 SecondaryNameNode
12180 DataNode
12046 NameNode

start-dfs.sh 输出

19/09/15 18:47:24 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hduser-namenode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hduser-datanode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hduser-secondarynamenode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
19/09/15 18:47:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

start-yarn.sh 输出

starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-hduser-resourcemanager-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-hduser-nodemanager-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out

cat mapred-site.xml

<configuration>
 <property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
 </property>
    <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
    </property>
    <property>
            <name>yarn.app.mapreduce.am.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
    <property>
            <name>mapreduce.map.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
    <property>
            <name>mapreduce.reduce.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
</configuration>

cat yarn-site.xml

<configuration>
    <property>
            <name>yarn.acl.enable</name>
            <value>0</value>
    </property>

    <property>
            <name>yarn.resourcemanager.hostname</name>
            <value>103.101.71.26</value>
    </property>

    <property>
            <name>yarn.nodemanager.aux-services</name>
            <value>mapreduce_shuffle</value>
    </property>
</configuration>

根据mapred-site.xml 文件作业跟踪器应该在localhost:54311 上运行，但是当我在网络浏览器中运行JPS 时它不会作为服务运行。

请问我可以做些什么来运行它..？

谢谢

Answer 1

JobTracker 是 Hadoop 1 的概念，已被 YARN ResourceManager 取代。 Oozie 也使用 YARN

不清楚您是按照什么设置配置的，但官方安装指南中未提及此属性。

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html#YARN_on_a_Single_Node

事实上，大多数以mapred.* 开头的属性已被弃用，您可以在此处找到有效的 mapred-site XML 属性 - https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml