kubernetes 入口 websockets (socket.io)答案

【问题标题】：kubernetes ingress websockets (socket.io)kubernetes 入口 websockets (socket.io)
【发布时间】：2019-11-26 09:22:01
【问题描述】：

我在尝试打开 websocket 时不断收到错误请求 (400)，我正在尝试找出原因。后端建立在带有 flask-socketio 的烧瓶上。这是我用于后端的 Docker 容器：

FROM alpine:edge

RUN apk update
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio

ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui

CMD ["gunicorn", "-k", "gevent", "-w", "1", "--bind", "0.0.0.0:3001", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]

这是我的 yaml 文件：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: my-global-ip
    networking.gke.io/managed-certificates: my-certificate
    nginx.ingress.kubernetes.io/add-base-url: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/websocket-services: "my-service-web"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
spec:
  rules:
  - http:
      paths:
      - path: /grafana/*
        backend:
          serviceName: my-service-web
          servicePort: grafana-port
      - path: /*
        backend:
          serviceName: my-service-web
          servicePort: web-app-port

apiVersion: v1
kind: Service
metadata:
  name: my-service-web

spec:
  ports:
    - port: 3000
      name: grafana-port
      targetPort: grafana-port
      protocol: TCP
    - port: 3001
      name: web-app-port
      targetPort: web-app-port
      protocol: TCP
  selector:
    app: my-cloud
  type: NodePort

从日志中可以看出，请求到达了后端容器：

10.166.0.42 - - [26/Nov/2019:08:52:22 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:23 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:25 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:28 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:34 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:39 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.41 - - [26/Nov/2019:08:52:44 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.166.0.42 - - [26/Nov/2019:08:52:49 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"
10.4.2.1 - - [26/Nov/2019:08:52:54 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0"

我想不通的是如何配置 kubernetes 入口以遵循 nginx 的 flask-socketio 配置：

    location /socket.io {
        include proxy_params;
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass http://127.0.0.1:5000/socket.io;
    }

如何在 kubernetes ingress 中升级到 websocket 的连接？

更新：我在启用日志的情况下实例化了 SocketIO，并得到了以下结果：

6f3a03945f174b039b033e887079b97d: Sending packet OPEN data {'sid': '6f3a03945f174b039b033e887079b97d', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 25000}
6f3a03945f174b039b033e887079b97d: Sending packet MESSAGE data 0
6f3a03945f174b039b033e887079b97d: Received request to upgrade to websocket
10.166.0.43 - - [26/Nov/2019:10:19:45 +0000] "GET /socket.io/?EIO=3&transport=websocket HTTP/1.1" 400 11 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:70.0) Gecko/20100101 Firefox/70.0

【问题讨论】：

ingress-nginx 默认启用 websockets kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/…，所以我会查看应用程序，而不是 k8s。您也可以尝试绕过 ingress-nginx 直接连接到 nodeport。
你有多少网络工作者？如果你使用多个，那么你需要在你的 nginx 配置中使用粘性会话。
@Miguel 嗨，米格尔。我正在使用 1 名工人。问题是我不知道如何配置 kubernetes ingress nginx 来升级到 websocket 的连接。如果我在客户端应用程序中将传输更改为“轮询”，那么一切正常，但这不是纯 websockets（我现在可以使用轮询）。我有我自己的带有 nginx 的虚拟机（配置正确），其中 websockets 在flask-socketio 的文档之后工作得很好。我正在努力将该虚拟机移动到 kubernetes 上的容器中。
你完成that了吗？也看看here。
@mWatney 我刚刚尝试了它们在这些链接中显示的内容，但它不起作用。我的 websocket 连接仅在轮询模式下工作（我使用的是 socket.io）。但是升级到101失败了。您是否在 kubernetes（flask-socketio 后端）上启动并运行了 websocket？

标签： flask kubernetes kubernetes-ingress flask-socketio

【解决方案1】：

我几乎从一开始就做到了。我将 Dockerfile 中的 CMD 更改为（根据 Flask-socketio 文档）：

gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 module:app

这是完整的Dockerfile：

FROM alpine:edge

RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update && apk upgrade
RUN apk add python3 py3-cffi py3-bcrypt libc-dev py3-psycopg2 py3-gevent-websocket
RUN pip3 install --upgrade pip
RUN pip3 install flask flask-restful flask-jwt-extended gunicorn requests flask-sqlalchemy flask-socketio

ADD ./rest-api /root/rest-api
ADD ./ui/dist/ui /root/ui

CMD ["gunicorn", "-k", "geventwebsocket.gunicorn.workers.GeventWebSocketWorker", "-w", "1", "--bind", "0.0.0.0:3001", "--timeout", "180", "--access-logfile", "-", "--chdir", "/root/rest-api/", "app:app"]

【讨论】：