【发布时间】:2016-06-23 09:22:19
【问题描述】:
我的kubectl version
Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.4", GitCommit:"3eed1e3be6848b877ff80a93da3785d9034d0a4f", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.4", GitCommit:"3eed1e3be6848b877ff80a93da3785d9034d0a4f", GitTreeState:"clean"}
我关注了Creating Multi-Container Pods
启动 pod 后,一个容器已启动,但其他容器未启动。
kubectl get pods
NAME READY STATUS RESTARTS AGE
redis-django 1/2 CrashLoopBackOff 9 22m
然后我做了kubectl describe redis-django。在底部我看到Error syncing pod, skipping 错误
31m <invalid> 150 {kubelet 172.25.30.21} spec.containers{frontend} Warning BackOff Back-off restarting failed docker container
25m <invalid> 121 {kubelet 172.25.30.21} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "frontend" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=frontend pod=redis-django_default(9f35ffcd-391e-11e6-b160-0022195df673)"
我该如何解决这个错误?任何帮助!
谢谢!
操作系统:Ubuntu 14
更新
之前我在Creating Multi-Container Pods找到的yaml文件下面使用
apiVersion: v1
kind: Pod
metadata:
name: redis-django
labels:
app: web
spec:
containers:
- name: key-value-store
image: redis
ports:
- containerPort: 6379
- name: frontend
image: django
ports:
- containerPort: 8000
frontend 容器未启动。然后我将 yaml 文件更改为两个具有不同名称和端口的 redis 容器。但是,结果是一样的(Getting Error syncing pod, skipping)
后来我将 yaml 文件更改为,只有一个 django 容器。此 pod 状态 CrashLoopBackOff 和 Error syncing pod, skipping
UPDATE-2
我tail -f /var/log/upstart/kublet.log,这给出了同样的错误。 Kubelet 一直在尝试启动容器,但不是!
I0623 12:15:13.943046 445 manager.go:2050] Back-off 5m0s restarting failed container=key-value-store pod=redis-django_default(94683d3c-392e-11e6-b160-0022195df673)
E0623 12:15:13.943100 445 pod_workers.go:138] Error syncing pod 94683d3c-392e-11e6-b160-0022195df673, skipping: failed to "StartContainer" for "key-value-store" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=key-value-store pod=redis-django_default(94683d3c-392e-11e6-b160-0022195df673)"
UPDATE-3
root@vm1:~/kubernetes/cluster/ubuntu/binaries# kubectl describe pod redis-django
Name: redis-django
Namespace: default
Node: 192.168.1.10/192.168.1.10
Start Time: Thu, 23 Jun 2016 22:58:03 -0700
Labels: app=web
Status: Running
IP: 172.16.20.2
Controllers: <none>
Containers:
key-value-store:
Container ID: docker://8dbdd6826c354243964f0306427082223d3da49bf2aaf30e15961ea00362fe42
Image: redis
Image ID: docker://sha256:4465e4bcad80b5b43cef0bace96a8ef0a55c0050be439c1fb0ecd64bc0b8cce4
Port: 6379/TCP
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Running
Started: Thu, 23 Jun 2016 22:58:10 -0700
Ready: True
Restart Count: 0
Environment Variables:
frontend:
Container ID: docker://9c89602739abe7331b3beb3a79e92a7cc42e2a7e40e11618413c8bcfd0afbc16
Image: django
Image ID: docker://sha256:0cb63b45e2b9a8de5763fc9c98b79c38b6217df718238251a21c8c4176fb3d68
Port: 8000/TCP
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 23 Jun 2016 22:58:41 -0700
Finished: Thu, 23 Jun 2016 22:58:41 -0700
Last State: Terminated
Reason: Completed
Exit Code: 0
Started: Thu, 23 Jun 2016 22:58:22 -0700
Finished: Thu, 23 Jun 2016 22:58:22 -0700
Ready: False
Restart Count: 2
Environment Variables:
Conditions:
Type Status
Ready False
Volumes:
default-token-0oq7p:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-0oq7p
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
49s 49s 1 {default-scheduler } Normal Scheduled Successfully assigned redis-django to 192.168.1.10
48s 48s 1 {kubelet 192.168.1.10} spec.containers{key-value-store} Normal Pulling pulling image "redis"
43s 43s 1 {kubelet 192.168.1.10} spec.containers{key-value-store} Normal Pulled Successfully pulled image "redis"
43s 43s 1 {kubelet 192.168.1.10} spec.containers{key-value-store} Normal Created Created container with docker id 8dbdd6826c35
42s 42s 1 {kubelet 192.168.1.10} spec.containers{key-value-store} Normal Started Started container with docker id 8dbdd6826c35
37s 37s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Started Started container with docker id 3872ceae75d4
37s 37s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Created Created container with docker id 3872ceae75d4
30s 30s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Created Created container with docker id d97b99b6780c
30s 30s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Started Started container with docker id d97b99b6780c
29s 29s 1 {kubelet 192.168.1.10} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "frontend" with CrashLoopBackOff: "Back-off 10s restarting failed container=frontend pod=redis-django_default(9d0a966a-39d0-11e6-9027-000c293d51ab)"
42s 16s 3 {kubelet 192.168.1.10} spec.containers{frontend} Normal Pulling pulling image "django"
11s 11s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Started Started container with docker id 9c89602739ab
38s 11s 3 {kubelet 192.168.1.10} spec.containers{frontend} Normal Pulled Successfully pulled image "django"
11s 11s 1 {kubelet 192.168.1.10} spec.containers{frontend} Normal Created Created container with docker id 9c89602739ab
29s 10s 2 {kubelet 192.168.1.10} spec.containers{frontend} Warning BackOff Back-off restarting failed docker container
10s 10s 1 {kubelet 192.168.1.10} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "frontend" with CrashLoopBackOff: "Back-off 20s restarting failed container=frontend pod=redis-django_default(9d0a966a-39d0-11e6-9027-000c293d51ab)"
对于容器frontend:不显示任何日志消息
root@vm1:~/kubernetes/cluster/ubuntu/binaries# kubectl logs redis-django -p -c frontend
root@vm1:~/kubernetes/cluster/ubuntu/binaries# kubectl logs redis-django -p -c key-value-store
Error from server: previous terminated container "key-value-store" in pod "redis-django" not found
root@vm1:~/kubernetes/cluster/ubuntu/binaries# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8dbdd6826c35 redis "docker-entrypoint.sh" 2 minutes ago Up 2 minutes k8s_key-value-store.f572c2d_redis-django_default_9d0a966a-39d0-11e6-9027-000c293d51ab_11101aea
8995bbf9f4f4 gcr.io/google_containers/pause:2.0 "/pause" 2 minutes ago Up 2 minutes k8s_POD.48e5231f_redis-django_default_9d0a966a-39d0-11e6-9027-000c293d51ab_c00025b0
root@vm1:~/kubernetes/cluster/ubuntu/binaries#
【问题讨论】:
-
kubelet.log 是否提供任何线索?它应该在节点机器的 /var/log/ 下。
-
刚刚检查了日志。那里也显示相同的错误。(您可以在我的问题
UPDATE-2中看到) -
我也检查了我的笔记本电脑(Ubuntu-单节点 Kubernetes)。在
logs和kubectl describe pod <POD>中出现相同的错误:-( -
日志说它处于崩溃循环中。我认为这条信息已经重复了很多次,在第一个信息中应该说明原因。你能粘贴整个
kubectl describe输出吗? -
正如@caesarxuchao 提到的,
kubectl describe <pod_name>的输出应该包括最后一个容器的终止原因。除此之外,您还可以使用kubectl logs <pod_name> -p -c <contianer_name>查看上次终止容器的日志