kube2iam - 无效角色：与带注释的角色不匹配

Question

我有一个完全正常工作的 AWS EKS 集群 (1.12)，我将 kube2iam 用于两个生产服务。但是，当我尝试向我们的集群添加另一个服务时收到错误消息，这让我很困惑。我正在运行 kube2iam 0.10.0。谢谢。

kube2iam 日志中的错误

time="2019-04-29T21:21:57Z" level=info msg="GET /latest (200) took 654374 ns" req.method=GET req.path=/latest req.remote=172.18.3.44 res.duration=654374 res.status=200
time="2019-04-29T21:21:57Z" level=info msg="GET /latest/meta-data/iam/security-credentials/ (200) took 18485 ns" req.method=GET req.path=/latest/meta-data/iam/security-credentials/ req.remote=172.18.3.44 res.duration=18485 res.status=200
time="2019-04-29T21:21:57Z" level=info msg="GET /latest/meta-data/iam/security-credentials/ui-eb-instance (200) took 37702 ns" req.method=GET req.path=/latest/meta-data/iam/security-credentials/ui-eb-instance req.remote=172.18.3.44 res.duration=37702 res.status=200
time="2019-04-29T21:21:57Z" level=info msg="GET /latest/dynamic/instance-identity/document (200) took 869383 ns" req.method=GET req.path=/latest/dynamic/instance-identity/document req.remote=172.18.3.44 res.duration=869383 res.status=200
time="2019-04-29T21:21:57Z" level=info msg="GET /latest/meta-data/iam/info (200) took 574052 ns" req.method=GET req.path=/latest/meta-data/iam/info req.remote=172.18.3.44 res.duration=574052 res.status=200
time="2019-04-29T21:21:57Z" level=error msg="Invalid role: does not match annotated role" ns.name=apollo-ui params.iam.role=e2e3-XXXXXXXXXX pod.iam.role="arn:aws:iam::XXXXXXXXXX:role/ui-eb-instance" req.method=GET req.path=/latest/meta-data/iam/security-credentials/e2e3-XXXXXXXXXX req.remote=172.18.3.44
time="2019-04-29T21:21:57Z" level=info msg="GET /latest/meta-data/iam/security-credentials/e2e3-XXXXXXXXXX (403) took 36262 ns" req.method=GET req.path=/latest/meta-data/iam/security-credentials/e2e3-XXXXXXXXXX req.remote=172.18.3.44 res.duration=36262 res.status=403

我的 Kubernetes 部署

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2019-04-29T20:13:44Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: ui
    app.kubernetes.io/managed-by: Tiller
    app.kubernetes.io/name: ui
    helm.sh/chart: ui-0.1.0
  name: ui
  namespace: apollo-ui
  resourceVersion: "3267240"
  selfLink: /apis/extensions/v1beta1/namespaces/apollo-ui/deployments/ui
  uid: 4a3cf375-6abb-11e9-b606-063dfc51737e
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: ui
      app.kubernetes.io/name: ui
  strategy:
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 50%
    type: RollingUpdate
  template:
    metadata:
      annotations:
        iam.amazonaws.com/role: ui-eb-instance
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: ui
        app.kubernetes.io/name: ui
    spec:
      containers:
      - env:
        ...
        ...
        ...
        image: XXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/ui:709870106
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /health_check
            port: http
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: ui
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /health_check
            port: http
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  conditions:
  - lastTransitionTime: "2019-04-29T20:13:44Z"
    lastUpdateTime: "2019-04-29T20:13:44Z"
    message: Deployment does not have minimum availability.
    reason: MinimumReplicasUnavailable
    status: "False"
    type: Available
  - lastTransitionTime: "2019-04-29T21:21:33Z"
    lastUpdateTime: "2019-04-29T21:21:33Z"
    message: ReplicaSet "ui-756cf57f9b" is progressing.
    reason: ReplicaSetUpdated
    status: "True"
    type: Progressing
  observedGeneration: 1
  replicas: 2
  unavailableReplicas: 2
  updatedReplicas: 2

AWS 角色“ui-eb-instance”角色

{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Action": "sts:AssumeRole",
                    "Principal": {
                        "Service": "ec2.amazonaws.com"
                    },
                    "Effect": "Allow",
                    "Sid": ""
                },
                {
                    "Action": "sts:AssumeRole",
                    "Principal": {
                        "AWS": "arn:aws:iam::XXXXXXXXXX:role/e2e3-XXXXXXXXXX"
                    },
                    "Effect": "Allow",
                    "Sid": ""
                }
            ]
        },
        "MaxSessionDuration": 3600,
        "RoleId": "XXXXXXXXXX",
        "CreateDate": "2018-04-20T18:06:01Z",
        "RoleName": "ui-eb-instance",
        "Path": "/",
        "Arn": "arn:aws:iam::XXXXXXXXXX:role/ui-eb-instance"
    }
}

我的 AWS EKS 工作节点 e2e3-XXXXXXXXXX 角色

{
    "Role": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Action": "sts:AssumeRole",
                    "Principal": {
                        "Service": "ec2.amazonaws.com"
                    },
                    "Effect": "Allow",
                    "Sid": "EKSWorkerAssumeRole"
                }
            ]
        },
        "MaxSessionDuration": 3600,
        "RoleId": "XXXXXXXXXX",
        "CreateDate": "2019-04-10T23:13:31Z",
        "RoleName": "e2e3-XXXXXXXXXX",
        "Path": "/",
        "Arn": "arn:aws:iam::XXXXXXXXXX:role/e2e3-XXXXXXXXXX"
    }
}

我的 kube2iam 守护进程

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  annotations:
    flux.weave.works/antecedent: infra:helmrelease/kube2iam
  creationTimestamp: "2019-04-16T18:37:54Z"
  generation: 3
  labels:
    app: kube2iam
    chart: kube2iam-0.9.1
    heritage: Tiller
    release: kube2iam
  name: kube2iam
  namespace: infra
  resourceVersion: "3252943"
  selfLink: /apis/extensions/v1beta1/namespaces/infra/daemonsets/kube2iam
  uid: bf258ef8-6076-11e9-b606-063dfc51737e
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: kube2iam
      release: kube2iam
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: kube2iam
        release: kube2iam
    spec:
      containers:
      - args:
        - --host-interface=eni+
        - --node=$(NODE_NAME)
        - --host-ip=$(HOST_IP)
        - --iptables=true
        - --auto-discover-base-arn=true
        - --auto-discover-default-role=true
        - --app-port=8181
        env:
        - name: HOST_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: jtblin/kube2iam:0.10.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 8181
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 5
          successThreshold: 1
          timeoutSeconds: 1
        name: kube2iam
        ports:
        - containerPort: 8181
          hostPort: 8181
          protocol: TCP
        resources: {}
        securityContext:
          privileged: true
          procMount: Default
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      hostNetwork: true
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: kube2iam
      serviceAccountName: kube2iam
      terminationGracePeriodSeconds: 30
  templateGeneration: 3
  updateStrategy:
    type: OnDelete
status:
  currentNumberScheduled: 2
  desiredNumberScheduled: 2
  numberAvailable: 2
  numberMisscheduled: 0
  numberReady: 2
  observedGeneration: 3
  updatedNumberScheduled: 2

在有问题的 ui pod 中验证 kube2iam 角色

bash-4.4#  curl http://169.254.169.254/latest/meta-data/iam/security-credentials/
ui-eb-instance

Answer 1

我能想到的几点：

1. 这可能是您的节点角色e2e3-XXXXXXXXXX 无法承担ui-eb-instance 角色的问题。这两个角色之间确实存在信任关系，但您是否将任何权限策略附加到e2e3-XXXXXXXXXX？

   您可以使用put 或attach 权限策略，并且可以从允许所有资源的AssumeRole 权限的策略开始：

   {
     "Version": "2012-10-17",
      "Statement": [
       {
         "Action": [
           "sts:AssumeRole"
         ],
         "Effect": "Allow",
         "Resource": "*"
       }
     ]
   }
   

   kube2iam docs 中也对此进行了描述。

2. 也可能是您的基本角色 arn 没有被 --auto-discover-base-arn 自动发现，因此您也可以尝试：--base-role-arn=arn:aws:iam::xxxxxxx:role/