集群范围内 API 组中的禁止资源

Question

我无法确定我的设置权限的确切问题，如下所示。我已经查看了所有类似的质量保证，但仍然无法解决问题。目的是部署 Prometheus 并让它抓取 /metrics 端点，我在集群中的其他应用程序可以正常暴露。

Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: endpoints is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"endpoints\" in API group \"\" at the cluster scope"
Failed to watch *v1.Pod: failed to list *v1.Pod: pods is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"pods\" in API group \"\" at the cluster scope"
Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"services\" in API group \"\" at the cluster scope"
...
...

下面的命令将no返回给所有的服务、节点、pod等

kubectl auth can-i get services --as=system:serviceaccount:default:default -n default

Minikube

$ minikube start --vm-driver=virtualbox --extra-config=apiserver.Authorization.Mode=RBAC

????  minikube v1.14.2 on Darwin 11.2
✨  Using the virtualbox driver based on existing profile
????  Starting control plane node minikube in cluster minikube
????  Restarting existing virtualbox VM for "minikube" ...
????  Preparing Kubernetes v1.19.2 on Docker 19.03.12 ...
    ▪ apiserver.Authorization.Mode=RBAC
????  Verifying Kubernetes components...
????  Enabled addons: storage-provisioner, default-storageclass, dashboard
????  Done! kubectl is now configured to use "minikube" by default

角色

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole

metadata:
  name: monitoring-cluster-role

rules:
  - apiGroups: [""]
    resources: ["nodes", "services", "pods", "endpoints"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get"]
  - apiGroups: ["extensions"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]

apiVersion: v1
kind: ServiceAccount

metadata:
  name: monitoring-service-account
  namespace: default

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding

metadata:
  name: monitoring-cluster-role-binding

roleRef:
  kind: ClusterRole
  name: monitoring-cluster-role
  apiGroup: rbac.authorization.k8s.io

subjects:
  - kind: ServiceAccount
    name: monitoring-service-account
    namespace: default

普罗米修斯

apiVersion: v1
kind: ConfigMap
 
metadata:
  name: prometheus-config-map
  namespace: default
 
data:
  prometheus.yml: |
    global:
      scrape_interval: 15s
    scrape_configs:
      - job_name: 'kubernetes-service-endpoints'
        kubernetes_sd_configs:
        - role: endpoints
        relabel_configs:
        - action: labelmap
          regex: __meta_kubernetes_service_label_(.+)
        - source_labels: [__meta_kubernetes_namespace]
          action: replace
          target_label: kubernetes_namespace
        - source_labels: [__meta_kubernetes_service_name]
          action: replace
          target_label: kubernetes_name 

apiVersion: apps/v1
kind: Deployment
 
metadata:
  name: prometheus-deployment
  namespace: default
  labels:
    app: prometheus
 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus:latest
          ports:
            - name: http
              protocol: TCP
              containerPort: 9090
          volumeMounts:
            - name: config
              mountPath: /etc/prometheus/
            - name: storage
              mountPath: /prometheus/
      volumes:
        - name: config
          configMap:
            name: prometheus-config-map
        - name: storage
          emptyDir: {}

apiVersion: v1
kind: Service
 
metadata:
  name: prometheus-service
  namespace: default
 
spec:
  type: NodePort
  selector:
    app: prometheus
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 9090

Answer 1

用户“system:serviceaccount:default:default”无法在集群范围内的 API 组“”中列出资源“端点”

用户“system:serviceaccount:default:default”无法在集群范围内的 API 组“”中列出资源“pods”

用户“system:serviceaccount:default:default”无法在集群范围内的 API 组“”中列出资源“服务”

在命名空间 default 中使用 ServiceAccount default 运行的东西正在做它没有权限的事情。

apiVersion: v1
kind: ServiceAccount
metadata:
  name: monitoring-service-account

在这里您创建一个特定的 ServiceAccount。您还可以授予它一些集群范围的权限。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
  namespace: default

您在命名空间 default 中运行 Prometheus，但未指定特定的 ServiceAccount，因此它将使用 ServiceAccount default 运行。

我认为您的问题是您应该设置在 Prometheus 的部署清单中创建的 ServiceAccount。