无法在 Kubernetes 集群中设置服务 DNS答案

【问题标题】：Unable to setup service DNS in Kubernetes cluster无法在 Kubernetes 集群中设置服务 DNS
【发布时间】：2017-12-20 16:35:21
【问题描述】：

Kubernetes 版本 --> 1.5.2

我第一次为 Kubernetes 服务设置 DNS，我遇到了 SkyDNS。所以按照文档，我的skydns-svc.yaml 文件是：

apiVersion: v1
kind: Service
spec:
  clusterIP: 10.100.0.100
  ports:
  - name: dns
    port: 53
    protocol: UDP
    targetPort: 53
  - name: dns-tcp
    port: 53
    protocol: TCP
    targetPort: 53
  selector:
    k8s-app: kube-dns
  sessionAffinity: None
  type: ClusterIP

我的skydns-rc.yaml 文件是：

apiVersion: v1
kind: ReplicationController
spec:
  replicas: 1
  selector:
    k8s-app: kube-dns
    version: v18
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: kube-dns
        kubernetes.io/cluster-service: "true"
        version: v18
    spec:
      containers:
      - args:
        - --domain=kube.local
        - --dns-port=10053
        image: gcr.io/google_containers/kubedns-amd64:1.6
        imagePullPolicy: IfNotPresent
        name: kubedns
        ports:
        - containerPort: 10053
          name: dns-local
          protocol: UDP
        - containerPort: 10053
          name: dns-tcp-local
          protocol: TCP
        resources:
          limits:
            cpu: 100m
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 100Mi
        terminationMessagePath: /dev/termination-log
      - args:
        - --cache-size=1000
        - --no-resolv
        - --server=127.0.0.1#10053
        image: gcr.io/google_containers/kube-dnsmasq-amd64:1.3
        imagePullPolicy: IfNotPresent
        name: dnsmasq
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
      - args:
        - -cmd=nslookup kubernetes.default.svc.kube.local 127.0.0.1 >/dev/null &&
          nslookup kubernetes.default.svc.kube.local 127.0.0.1:10053 >/dev/null
        - -port=8080
        - -quiet
        image: gcr.io/google_containers/exechealthz-amd64:1.0
        imagePullPolicy: IfNotPresent
        name: healthz
        ports:
        - containerPort: 8080
          protocol: TCP
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi

在我的奴才上，我更新了/etc/systemd/system/multi-user.target.wants/kubelet.service 文件并在ExecStart 部分下添加了以下内容：

ExecStart=/usr/bin/kubelet \
        $KUBE_LOGTOSTDERR \
        $KUBE_LOG_LEVEL \
        $KUBELET_API_SERVER \
        $KUBELET_ADDRESS \
        $KUBELET_PORT \
        $KUBELET_HOSTNAME \
        $KUBE_ALLOW_PRIV \
        $KUBELET_POD_INFRA_CONTAINER \
        $KUBELET_ARGS \
            --cluster-dns=10.100.0.100 \
            --cluster-domain=kubernetes \

完成所有这些并成功启动 rc 和 svc ：

[root@kubernetes-master DNS]# kubectl get po | grep dns
kube-dns-v18-hl8z6                                                3/3       Running             0          6s
[root@kubernetes-master DNS]# kubectl get svc | grep dns
kube-dns                            10.100.0.100     <none>        53/UDP,53/TCP                                                    20m

这就是我从配置的角度得到的全部信息。现在为了测试我的设置，我下载了busybox并测试了nslookup

[root@kubernetes-master DNS]# kubectl get svc | grep kubernetes
kubernetes                          10.100.0.1       <none>        443/TCP 

[root@kubernetes-master DNS]# kubectl exec busybox -- nslookup kubernetes
nslookup: can't resolve 'kubernetes'
Server:    10.100.0.100
Address 1: 10.100.0.100

有什么我错过的吗？

编辑 ::

查看日志，我看到了一些可以解释为什么这不起作用的东西：

kubectl logs $(kubectl get pods -l k8s-app=kube-dns -o name) -c kubedns
.
.
.
E1220 17:44:48.403976       1 reflector.go:216] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.100.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided
E1220 17:44:48.487169       1 reflector.go:216] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.100.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided
I1220 17:44:48.487716       1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.100.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying.
E1220 17:44:49.410311       1 reflector.go:216] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.100.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided
I1220 17:44:49.492338       1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.100.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying.
E1220 17:44:49.493429       1 reflector.go:216] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.100.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided
.
.
.

看起来kubedns 无法授权 K8S 主节点。我什至尝试进行手动调用：

curl -k https://10.100.0.1:443/api/v1/endpoints?resourceVersion=0
Unauthorized

【问题讨论】：

您可能想试一试coredns.io，它在我的一个集群上就像魅力一样。除此之外，您的部署似乎遗漏了一些零碎的东西，请查看此template 以获得良好的起点
是的，它在所有（master 和 minions）上运行/usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://master:8080
我想我找到了问题所在。请阅读我的编辑。但鉴于kubedns 需要授权，我不确定现在如何进行。有什么想法吗？

标签： dns kubernetes skydns

【解决方案1】：

看起来kube-dns pod 无法通过 kubernetes api 服务器进行身份验证。我在 kube-dns pod 的 YAML 文件中没有看到任何 secret 和 serviceaccount。

我建议执行以下操作：

使用kubectl create secret 为具有正确证书文件ca.crt 和token 的kube-dns pod 创建一个k8s secret：

$ kubectl get secrets -n=kube-system | grep dns
kube-dns-token-66tfx     kubernetes.io/service-account-token   3      1d

使用kubectl create serviceaccount 为kube-dns pod 创建一个k8s serviceaccount：

$ kubectl get serviceaccounts -n=kube-system | grep dns
kube-dns               1         1d`

在 YAML 文件的 kube-dns 容器内将 secret 挂载到 /var/run/secrets/kubernetes.io/serviceaccount：

...
kind: Pod
...
spec:
  ...
  containers:
    ...
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: kube-dns-token-66tfx
      readOnly: true
  ...
  volumes:
  - name: kube-dns-token-66tfx
    secret:
      defaultMode: 420
      secretName: kube-dns-token-66tfx

以下是为 pod 创建 serviceaccounts 的链接：

https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

https://kubernetes.io/docs/admin/service-accounts-admin/

【讨论】：