Hadoop - 未能指定服务器的 Kerberos 主体名称答案

【问题标题】：Hadoop - failed to specify server's Kerberos principal nameHadoop - 未能指定服务器的 Kerberos 主体名称
【发布时间】：2018-11-26 15:31:33
【问题描述】：

错误 - 无法指定服务器的 Kerberos 主体名称

我正在尝试使用 Kerberos 设置 Hadoop 集群。在开始 Kerberos 配置之前，我设法让集群与 Spark 和 Yarn 一起工作。目前我的主节点和三个节点正在运行，但我在纱线日志中遇到错误。

错误：

java.io.IOException: Failed on local exception: java.io.IOException: java.lang.IllegalArgumentException : Failed to specify server's Kerberos principal name

核心站点.xml

<configuration>

    <property>
        <name>fs.default.name</name>
        <value>hdfs://hadoopmaster:9000</value>
    </property>

    <!--Kerberos configuration-->

    <property>
        <name>hadoop.security.authentication</name>
        <value>kerberos</value>
    </property>

    <property>
        <name>hadoop.security.authorization</name>
        <value>true</value>
    </property>

    <property>
      <name>hadoop.security.auth_to_local</name>
      <value>
        RULE:[2:$1@$0](hdfs/.*@.*EXAMPLEREALM.COM)s/.*/hdfs/
        RULE:[2:$1@$0](HTTP/.*@.*EXAMPLEREALM.COM)s/.*/hdfs/
        RULE:[2:$1@$0](yarn/.*@.*EXAMPLEREALM.COM)s/.*/yarn/
        DEFAULT
      </value>
    </property>

</configuration>

hdfs-site.xml

<configuration>

    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/home/hadoop/data/namenode</value>
    </property>

    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/home/hadoop/data/datanode</value>
    </property>

    <property>
        <name>dfs.replication<name>
        <value>2</value>
    </property>

    <!-- General HDFS security config -->
    <property>
      <name>dfs.block.access.token.enable</name>
      <value>true</value>
    </property>

    <!-- NameNode security config -->
    <property>
      <name>dfs.namenode.keytab.file</name>
      <value>/etc/security/keytabs/hdfs.service.keytab</value> <!-- path to the HDFS keytab -->
    </property>
    <property>
      <name>dfs.namenode.kerberos.principal</name>
      <value>hdfs/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>
    <property>
      <name>dfs.namenode.kerberos.internal.spnego.principal</name>
      <value>HTTP/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>

    <!-- Secondary NameNode security config -->
    <property>
      <name>dfs.secondary.namenode.keytab.file</name>
      <value>/etc/security/keytabs/hdfs.service.keytab</value> <!-- path to the HDFS keytab -->
    </property>
    <property>
      <name>dfs.secondary.namenode.kerberos.principal</name>
      <value>hdfs/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>
    <property>
      <name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name>
      <value>HTTP/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>

    <!-- DataNode security config -->
    <property>
      <name>dfs.datanode.data.dir.perm</name>
      <value>700</value> 
    </property>
    <property>
      <name>dfs.datanode.address</name>
      <value>0.0.0.0:1004</value>
    </property>
    <property>
      <name>dfs.datanode.http.address</name>
      <value>0.0.0.0:1006</value>
    </property>
    <property>
      <name>dfs.datanode.keytab.file</name>
      <value>/etc/security/keytabs/hdfs.service.keytab</value> <!-- path to the HDFS keytab -->
    </property>
    <property>
      <name>dfs.datanode.kerberos.principal</name>
      <value>hdfs/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>

    <!-- Web Authentication config -->
    <property>
      <name>dfs.web.authentication.kerberos.principal</name>
      <value>HTTP/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
     </property>

 </configuration>

yarn-site.xml

<configuration>

    <property>
        <name>yarn.acl.enable</name>
        <value>0</value>
    </property>

    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>hadoopmaster</value>
    </property>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>

    <property>
        <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
        <value>org.apache.hadoop.mapred.ShuffleHandler</value>
    </property>

    <property>
      <name>yarn.nodemanager.principal</name>
      <value>yarn/hadoopslave1.examplerealm.com@EXAMPLEREALM.COM</value>
    </property>

    <property>
      <name>yarn.nodemanager.keytab</name>
      <value>/etc/security/keytabs/yarn.service.keytab</value>
    </property>

</configuration>

【问题讨论】：

标签： hadoop hadoop-yarn kerberos

【解决方案1】：

您是否在所有节点上安装了krb5-libs 和krb5-workstation？

在 Centos 上：

yum install krb5-server  krb5-libs
yum install  krb5-libs krb5-workstation

在这种情况下，尝试这样做可能会对您有所帮助：

systemctl enable krb5kdc 
systemctl start krb5kdc
systemctl enable kadmin 
systemctl start kadmin

同时检查：
https://community.hortonworks.com/questions/176262/failed-to-specify-servers-kerberos-principal-name.html

【讨论】：