Hadoop ResourceManager HA 在 /0.0.0.0:8032 连接到 ResourceManager答案

【问题标题】：Hadoop ResourceManager HA Connecting to ResourceManager at /0.0.0.0:8032Hadoop ResourceManager HA 在 /0.0.0.0:8032 连接到 ResourceManager
【发布时间】：2016-01-29 19:42:19
【问题描述】：

扩展其中一个问题： Hadoop: Connecting to ResourceManager failed

Hadoop 2.6.1

我确实配置了 ResourceManager HA。

当我杀死“本地”ResourceManager（检查集群）时，会发生故障转移，并且其他服务器上的 ResourceManager 会变为活动状态。不幸的是，当我尝试使用“本地”实例节点管理器运行作业时，它不会“故障转移”到活动 ResourceManager 的请求。

yarn@stg-hadoop106:~$ jps
26738 Jps
23463 DataNode
23943 DFSZKFailoverController
24297 NodeManager
25690 ResourceManager
23710 JournalNode
23310 NameNode

#kill and start ResourceManager, so the failover occur
yarn@stg-hadoop106:~$ kill -9 25690
~/hadoop/sbin/yarn-daemon.sh  start resourcemanager

yarn@stg-hadoop106:~$ ~/hadoop/bin/yarn  rmadmin -getServiceState rm1
standby
yarn@stg-hadoop106:~$ ~/hadoop/bin/yarn  rmadmin -getServiceState rm2
active

#run my class:

14:56:51.476 [main] INFO  o.apache.samza.job.yarn.ClientHelper - trying to connect to RM 0.0.0.0:8032
2015-10-29 14:56:51 RMProxy [INFO] Connecting to ResourceManager at /0.0.0.0:8032
14:56:51.572 [main] DEBUG o.a.h.s.a.util.KerberosName - Kerberos krb5 configuration not found, setting default realm to empty
2015-10-29 14:56:51 NativeCodeLoader [WARN] Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14:56:51.575 [main] DEBUG o.a.hadoop.util.PerformanceAdvisory - Falling back to shell based
2015-10-29 14:56:52 Client [INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2015-10-29 14:56:53 Client [INFO] Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)

yarn-site.xml

 <property>
     <name>yarn.resourcemanager.ha.enabled</name>
     <value>true</value>
 </property>
 <property>
     <name>yarn.resourcemanager.cluster-id</name>
     <value>clusterstaging</value>
 </property>
 <property>
     <name>yarn.resourcemanager.ha.rm-ids</name>
     <value>rm1,rm2,rm3</value>
 </property>
 <property>
     <name>yarn.resourcemanager.hostname.rm1</name>
     <value>stg-hadoop106</value>
 </property>
 <property>
     <name>yarn.resourcemanager.hostname.rm2</name>
     <value>stg-hadoop107</value>
 </property>
 <property>
     <name>yarn.resourcemanager.hostname.rm3</name>
     <value>stg-hadoop108</value>
 </property>
 <property>
     <name>yarn.resourcemanager.zk-address</name>
     <value>A:2181,B:2181,C:2181</value>
 </property>

我没有配置

<name>yarn.resourcemanager.hostname</name>

因为它应该“按原样”工作 - 如果我错了，请纠正我 :)

我试过了

<name>yarn.client.failover-proxy-provider</name>

但没有成功

有什么想法吗？也许我错误地期望客户端找到活动的 RM 节点？

您知道如何在“自动故障转移”选项中切换节点活动/备用吗？

~/hadoop/bin/yarn  rmadmin -failover rm1 rm2
    Exception in thread "main" java.lang.UnsupportedOperationException: RMHAServiceTarget doesn't have a corresponding ZKFC address

~/hadoop/bin/yarn  rmadmin -transitionToActive rm1 rm2
    Automatic failover is enabled for org.apache.hadoop.yarn.client.RMHAServiceTarget@2b72cb8a
    Refusing to manually manage HA state, since it may cause

【问题讨论】：

标签： hadoop high-availability failover resourcemanager

【解决方案1】：

如果您在自动故障转移模式下启用 HA-RM，则无法触发活动到备用或反之亦然。并且您应该提供yarn.client.failover-proxy-provider 参数，这是客户端用于故障转移到活动 RM 的类。以及配置yarn.resourcemanager.hostname 来识别RM（即rm1、rm2）。

如果未启用自动故障转移，您可以使用以下触发 yarn rmadmin -transitionToStandby rm1

请做以上修改并给出结果

【讨论】：