NGINX 在 Kubernetes 上执行滚动重启时返回 502答案

【问题标题】：NGINX return 502 when performing rolling restart on KubernetesNGINX 在 Kubernetes 上执行滚动重启时返回 502
【发布时间】：2022-08-09 18:48:34
【问题描述】：

我有一个带有多个容器（节点应用程序）的 pod，它们前面有一个 NGINX 反向代理。我已经为所有这些 pod 定义了就绪和活跃度探测。但是，在执行滚动重启时，我仍然会收到丢弃的请求。

我正在使用端点来监视返回状态。

while sleep 1; do http -h get http://workstation1.factory1.domain.com/healthcheck | head -n 1 | awk -F\" \" \'{print $2}\'; done

这是我的部署定义。

apiVersion: apps/v1
kind: Deployment
metadata:
  name: portal-deployment
  labels:
    app: portal
spec:
  replicas: 2
  selector:
    matchLabels:
      app: portal
  template:
    metadata:
      labels:
        app: portal
    spec:
      volumes:
      - configMap:
          name: nginx-conf
        name: nginx-conf
      containers:
      - name: nginx
        image: nginx:stable-alpine
        ports:
        - containerPort: 80
        volumeMounts:
        - mountPath: /etc/nginx/nginx.conf
          name: nginx-conf
          subPath: nginx.conf
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 80
          initialDelaySeconds: 20
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 80
          initialDelaySeconds: 60
          periodSeconds: 3
        resources:
          requests:
            memory: \"64Mi\"
            cpu: \"250m\"
          limits:
            memory: \"128Mi\"
            cpu: \"500m\"
        lifecycle:
          preStop:
            exec:
              command: [
                # Gracefully shutdown nginx
                \"/usr/sbin/nginx\", \"-s\", \"quit\"
              ]

      - name: portal-api
        image: XXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com/portal-api
        imagePullPolicy: Always
        envFrom:
          - configMapRef: 
              name: portal-config
        ports:
        - containerPort: 8080
        resources:
          requests:
            memory: \"64Mi\"
            cpu: \"250m\"
          limits:
            memory: \"128Mi\"
            cpu: \"500m\"
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 8080
          initialDelaySeconds: 20
          periodSeconds: 3

      - name: cutter-api-2
        image: XXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com/cutter-api
        imagePullPolicy: Always
        resources:
          requests:
            memory: \"64Mi\"
            cpu: \"250m\"
          limits:
            memory: \"128Mi\"
            cpu: \"500m\"
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 8002
          initialDelaySeconds: 20
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 8002
          initialDelaySeconds: 10
          periodSeconds: 3
        envFrom:
          - configMapRef: 
              name: cell-config
        env:
        - name: PORT
          value: \"8002\"
        - name: SLOT
          value: \"2\"
        - name: OPCUA_ENDPOINT
          value: \"XXXXXXXXXX\"
        - name: OPCUA_USERNAME
          value: \"XXXXXXXXXX\"
        - name: OPCUA_PASSWORD
          value: \"XXXXXXXXXX\"
        ports:
          - name: http
            containerPort: 8002
      
      - name: cutter-api-13
        image: XXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com/cutter-api
        imagePullPolicy: Always
        resources:
          requests:
            memory: \"64Mi\"
            cpu: \"250m\"
          limits:
            memory: \"128Mi\"
            cpu: \"500m\"
        livenessProbe:
          httpGet:
            path: /healthcheck
            port: 8013
          initialDelaySeconds: 20
          periodSeconds: 3
        readinessProbe:
          httpGet:
            path: /healthcheck
            port: 8013
          initialDelaySeconds: 10
          periodSeconds: 3
        envFrom:
          - configMapRef: 
              name: cell-config
        env:
        - name: PORT
          value: \"8013\"
        - name: SLOT
          value: \"13\"
        - name: OPCUA_ENDPOINT
          value: \"XXXXXXXXXX\"
        - name: OPCUA_USERNAME
          value: \"XXXXXXXXXX\"
        - name: OPCUA_PASSWORD
          value: \"\"
        ports:
          - name: http
            containerPort: 8013

      imagePullSecrets:
        - name: aws-ecr
---
apiVersion: v1
kind: Service
metadata:
  name: portal-service
spec:
  selector:
    app: portal
  ports:
    - port: 9080
      targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: workstation-1
spec:
  rules:
    - host: workstation1.factory1.domain.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: portal-service
                port:
                  number: 9080

我认为正在发生的事情是请求进入但被 NGINX 丢弃被终止并返回 502？

当 Kubernetes 迁移到新部署时，我只收到 502 响应。

标签： nginx kubernetes kubernetes-pod

【解决方案1】：

检查 Pod 和容器是否正常运行 $ kubectl get pods 如果它们没有运行，请重新启动它们或强制 kubernetes 重新安排它。
检查容器是否正在侦听所需的端口。 kubectl describe pod [pod-name] 如果那里缺少任何东西，请添加它。
检查服务是否处于活动状态。 kubectl get svc 如果您在列表中看不到任何必需的服务，则需要使用 Kubectl 创建。
检查服务映射是否正确。 kubectl describe svc [service-name]。
使用以下命令检查入口是否存在：get ing，如果缺少，则需要使用kubectl apply -f [ingress-config].yaml. 创建一个
检查入口规则和后端。 kubectl describe ingress [ingress-name] 如果入口未正确映射或不健康，您需要使用 kubectl apply -f [ingress-config].yaml. 修复它

【讨论】：