创建 pod 并出现 CrashLoopBackOff 时无法“启动容器”答案

【问题标题】：Failed to "Start Container" while creating pod and gettting CrashLoopBackOff创建 pod 并出现 CrashLoopBackOff 时无法“启动容器”
【发布时间】：2019-12-04 15:45:41
【问题描述】：

我对 kubernetes 非常陌生，正在尝试使用现有的 docker 映像创建一个 pod。 Pod 已创建，但附加到它的容器未运行。你能帮我解决这个问题吗？

在stackoverflow上参考之前的答案，但无法得到答案。

Docker 文件：

FROM selenium/standalone-firefox-debug:3.141.59-selenium
USER seluser
CMD /opt/bin/entry_point.sh

使用 Docker 文件成功创建映像。

容器创建成功，我可以使用 vnc 查看器访问容器。

docker run -d -p 4444:4444 -p 5901:5900 --name=checkcont -it testImage

注意：我是否需要在 pod.yml 中提及 5901:5900，如果是，请您举个例子说明如何提及 vnc 的不同端口

使用以下文件创建 pod 时出现错误“CrashLoopBackOff”

---
 apiVersion: v1
 kind: Pod
 metadata:
   name: hello-test
 spec:
   containers:
   - name: hello-test-cont
     image: testImage
     ports:
     - containerPort: 4444

...

日志

2019-11-21T09:07:15.834731+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:15.999653+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:15+00:00 ==
2019-11-21T09:07:16.000062+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:16.148533+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:16.149823+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:17.509070+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:17.510418+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:17+00:00 ==
2019-11-21T09:07:20.515166+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:20.677541+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:20+00:00 ==
2019-11-21T09:07:20.677850+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:20.814736+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:20.819174+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:22.169192+00:00 machine-id systemd[1]: Started libcontainer container 94e63e7fc516e13fd2f68d93ad8b27033578ad6064a4d67b656e4075bc816453.
2019-11-21T09:07:22.171186+00:00 machine-id systemd[1]: Starting libcontainer container 94e63e7fc516e13fd2f68d93ad8b27033578ad6064a4d67b656e4075bc816453.
2019-11-21T09:07:22.187229+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:22.188861+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:22+00:00 ==
2019-11-21T09:07:22.234828+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:22.233811907Z" level=warning msg="Unknown healthcheck type 'NONE' (expected 'CMD') in container 94e63e7fc516e13fd2f68d93ad8b27033578ad6064a4d67b656e4075bc816453"
2019-11-21T09:07:22.243520+00:00 machine-id kubelet[17918]: W1121 09:07:22.243239   17918 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for default/hello-test through plugin: invalid network status for
2019-11-21T09:07:22.260932+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:22.260836014Z" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version 1.0.0-rc2\nspec: 1.0.0-rc2-dev\n"
2019-11-21T09:07:22.261189+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:22.260890502Z" level=warning msg="failed to retrieve docker-init version"
2019-11-21T09:07:22.281343+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:22.281151057Z" level=error msg="containerd: deleting container" error="exit status 1: \"container 94e63e7fc516e13fd2f68d93ad8b27033578ad6064a4d67b656e4075bc816453 does not exist\\none or more of the container deletions failed\\n\""
2019-11-21T09:07:22.286116+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:22.286007527Z" level=warning msg="94e63e7fc516e13fd2f68d93ad8b27033578ad6064a4d67b656e4075bc816453 cleanup: failed to unmount secrets: invalid argument"
2019-11-21T09:07:23.258400+00:00 machine-id kubelet[17918]: W1121 09:07:23.258309   17918 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for default/hello-test through plugin: invalid network status for
2019-11-21T09:07:23.265928+00:00 machine-id kubelet[17918]: E1121 09:07:23.265868   17918 pod_workers.go:191] Error syncing pod d148d8b9-83c2-4fa3-aeef-aa52063568f2 ("hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"), skipping: failed to "StartContainer" for "hello-test-cont" with CrashLoopBackOff: "back-off 20s restarting failed container=hello-test-cont pod=hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"
2019-11-21T09:07:24.271600+00:00 machine-id kubelet[17918]: W1121 09:07:24.271511   17918 docker_sandbox.go:394] failed to read pod IP from plugin/docker: Couldn't find network status for default/hello-test through plugin: invalid network status for
2019-11-21T09:07:24.276093+00:00 machine-id kubelet[17918]: E1121 09:07:24.276039   17918 pod_workers.go:191] Error syncing pod d148d8b9-83c2-4fa3-aeef-aa52063568f2 ("hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"), skipping: failed to "StartContainer" for "hello-test-cont" with CrashLoopBackOff: "back-off 20s restarting failed container=hello-test-cont pod=hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"
2019-11-21T09:07:25.193657+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:25.342440+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:25+00:00 ==
2019-11-21T09:07:25.342767+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:25.483283+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:25.490946+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:26.855714+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:26.857423+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:26+00:00 ==
2019-11-21T09:07:30.861720+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:31.012558+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:31+00:00 ==
2019-11-21T09:07:31.012884+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:31.163171+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:31.170232+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:31.726380+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:31.726164882Z" level=warning msg="failed to retrieve docker-runc version: unknown output format: runc version 1.0.0-rc2\nspec: 1.0.0-rc2-dev\n"
2019-11-21T09:07:31.726701+00:00 machine-id dockerd-current[18605]: time="2019-11-21T09:07:31.726218553Z" level=warning msg="failed to retrieve docker-init version"
2019-11-21T09:07:32.545552+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:32.547033+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:32+00:00 ==
2019-11-21T09:07:35.552154+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:35.725415+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:35+00:00 ==
2019-11-21T09:07:35.725775+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:35.874046+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:35.877087+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:37.236668+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:37.238603+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:37+00:00 ==
2019-11-21T09:07:37.265979+00:00 machine-id kubelet[17918]: E1121 09:07:37.264872   17918 pod_workers.go:191] Error syncing pod d148d8b9-83c2-4fa3-aeef-aa52063568f2 ("hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"), skipping: failed to "StartContainer" for "hello-test-cont" with CrashLoopBackOff: "back-off 20s restarting failed container=hello-test-cont pod=hello-test_default(d148d8b9-83c2-4fa3-aeef-aa52063568f2)"
2019-11-21T09:07:40.243249+00:00 machine-id dockerd-current[18605]: INFO: Leader election disabled.
2019-11-21T09:07:40.395096+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon ensure completed at 2019-11-21T09:07:40+00:00 ==
2019-11-21T09:07:40.395435+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with deprecated label ==
2019-11-21T09:07:40.538812+00:00 machine-id dockerd-current[18605]: error: no objects passed to apply
2019-11-21T09:07:40.544479+00:00 machine-id dockerd-current[18605]: INFO: == Reconciling with addon-manager label ==
2019-11-21T09:07:41.902469+00:00 machine-id dockerd-current[18605]: serviceaccount/storage-provisioner unchanged
2019-11-21T09:07:41.904341+00:00 machine-id dockerd-current[18605]: INFO: == Kubernetes addon reconcile completed at 2019-11-21T09:07:41+00:00 ==
"

【问题讨论】：

你能包含更多的日志吗？您所显示的内容听起来容器在退出后无法自行清理，但这实际上并不能说明退出原因。
@DavidMaze，请参考详细日志。
PodSandBox 由于网络问题而未创建 docker_sandbox.go:394] failed to read pod IP from plugin/docker Couldn't find network status for default/hello-test through plugin: invalid network status 这是运行时的错误。好像你的 k8s 集群不能正常工作

标签： docker kubernetes

【解决方案1】：

你可以像这样添加端口

apiVersion: v1
kind: Pod
metadata:
   name: hello-test
spec:
   containers:
     image: imagename:tag
     ports:
    - containerPort: 4444
      name: hello-test-cont
      protocol: TCP
    - containerPort: 5900
      name: testport
      protocol: TCP

还要确保提供完整的图像名称。如果您的机器上已经存在该图像，请同时提及标签名称。我能够创建具有相似文件格式的 pod 和容器。希望它对你有用。

【讨论】：

谢谢@vish。用文件提及标签名称解决了我的问题。 Pod 和容器现在正在运行。因为我正在使用图像 Selenium/standalone-firefox-debug:3.141.59-selenium 并且需要使用 vnc 进行连接。能否请您也提供相应的步骤。
@rajjj，为 vnc 创建一个单独的问题。

【解决方案2】：

您可以通过以下方式在deployment 中指定多个端口：

        ports:
        - containerPort: 7000
          name: one-node
        - containerPort: 7001
          name: two-node
        - containerPort: 7199
          name: bnm
        - containerPort: 9042
          name: qaz

这是cassandra-statefulset.yaml 的示例，它在部署中使用了多个端口。

至于selenium，有关于如何在Kubernetes上运行的示例here。

Kubernetesselenium-node-firefox-deployment 如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: selenium-node-firefox
  labels:
    app: selenium-node-firefox
spec:
  replicas: 2
  selector:
    matchLabels:
      app: selenium-node-firefox
  template:
    metadata:
      labels:
        app: selenium-node-firefox
    spec:
      volumes:
      - name: dshm
        emptyDir:
          medium: Memory
      containers:
      - name: selenium-node-firefox
        image: selenium/node-firefox-debug:3.141
        ports:
          - containerPort: 5900
        volumeMounts:
          - mountPath: /dev/shm
            name: dshm
        env:
          - name: HUB_HOST
            value: "selenium-hub"
          - name: HUB_PORT
            value: "4444"
        resources:
          limits:
            memory: "1000Mi"
            cpu: ".5"

【讨论】：

以上都没有回答这个问题。

【解决方案3】：

根据docs 关于容器端口：

...在这里暴露一个端口会给系统有关网络连接的附加信息容器使用，但主要是信息性的。未指定端口这里不会阻止该端口被暴露。

根据您发布的日志，您遇到的问题似乎是容器无法启动。

查看事件（例如kubectl get events --sort-by=.metadata.creationTimestamp）可能会显示更多信息，但问题可能出在图像上。更具体地说 - Kubernetes 找不到它，因为它试图从公共 Docker Hub 中提取它。原因如下：

Kubernetes 从 Docker Registry 中提取容器镜像。根据doc：

您创建 Docker 映像并将其推送到注册表之前在 Kubernetes pod 中引用它。

此外：

容器的image 属性支持与 docker 命令可以，包括私有注册表和标签。

您必须将自定义映像推送到公共或私有 Docker 注册表并在 image 属性中引用该映像。

This answer 应该可以帮助您建立一个本地 Docker 注册表以与本地 Kubernetes 集群一起使用。

【讨论】：