如何在 Apache 反向代理后面使用 websocket 和主节点连接 Jenkins 从代理

Question

我正在尝试使用 websocket 连接一个新的 jenkins 从属节点，从从属节点启动代理对 wss 调用的响应是 400 Bad request，即从代理收到的“握手错误”。
通过我的低级测试和分析，我发现在 HTTP/1.1 中的 Apache 反向代理中的传入调用在转发到目标节点时会降级为 HTTP/1.0，然后在响应返回给调用者时升级到 HTTP/1.1。我没有找到任何配置来避免这种情况。

详细说明

架构

• Apache 反向代理，版本 2.4.41，安装在 RHEL7.3，IP 192.168.1.2
  Apache 它位于公共防火墙后面，只允许端口 443，https 它在 apache 虚拟主机中被卸载。这 Apache 和 Jenkins Master 节点之间的通信是在 http on 8080端口
• Jenkins Master 节点版本 2.263.1，安装在一个 RHEL7.6, Openjdk11, IP 192.168.1.10:8080
  (NO "看来你的 反向代理设置已损坏”消息显示在 jenkins 中）
• Jenkins Slave 节点，代理安装在 Windows server 2016，Openjdk11，IP 192.168.1.11
  （主从节点在同一个子网，理想情况下不需要通过公网连接。我没有 找到了一种方法，只允许通过设置 jnlp hudson.TcpSlaveAgentListener.hostName 系统属性，参考： https://stackoverflow.com/a/39965700 和 https://stackoverflow.com/a/39136802)

反向代理配置

RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"

SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerName off

ProxyRequests Off
AllowEncodedSlashes NoDecode

<Location /jenkins>
    ProxyPreserveHost On
    ProxyPass http://192.168.1.10:8080/jenkins nocanon
    ProxyPassReverse http://192.168.1.10:8080/jenkins
    ProxyPassReverse https://my.public.fqdn.com/jenkins

    RequestHeader set X-Forwarded-Host "my.public.fqdn.com"
</Location>

<Location /jenkins/wsagents>
    ### Configurations tested and commented, not working ###
    #SetEnv force-no-vary 1
    #SetEnv force-proxy-request-1.0 1
    #SetEnv proxy-nokeepalive 1
    #RequestHeader unset Expect early

    ProxyPreserveHost On
    ProxyPass ws://192.168.1.10:8080/jenkins/wsagents
    ProxyPassReverse ws://192.168.1.10:8080/jenkins/wsagents
</Location>

行为
在詹金斯中从“管理节点和云”中创建一个新节点，并从“启动方法”中选择“通过将其连接到主节点来启动代理”，然后选择“使用 Websocket”， 当我尝试从从节点启动代理时，我遇到了 400 Bad request response fall in handshake 错误：

java -jar agent.jar -jnlpUrl https://my.public.fqdn.com/jenkins/computer/Slave%20Windows/slave-agent.jnlp -secret @secret-file -workDir "E:\JenkinsSlave"

Sending handshake request:
> GET wss://my.public.fqdn.com/jenkins/wsagents/
> Connection: Upgrade
> Host: my.public.fqdn.com
> Node-Name: Slave Windows
> Origin: my.public.fqdn.com
> Sec-WebSocket-Key: FFGODSDcF0TTP4q/usk9Bw==
> Sec-WebSocket-Version: 13
> Secret-Key: 123123123123
> Upgrade: websocket
> X-Remoting-Capability: rO0ABXNyABpod4ucmVtbpbmcuQ2FwYWJpbGl0eQAAAAAAAAABAgABSgAEbWFza3hwAAAAf4=

2021-01-14T22:48:44.776+0100 FINE io.jenkins.remoting.shaded.org.glassfish.tyrus.core.DebugContext flush: < Session 36d3c733-37f3-4f6e-bea1-920e8cf6f3da [128 ms]: Received handshake response: 
< 400
< Cache-Control: must-revalidate, no-cache, no-store
< Content-Length: 533
< Content-Type: text/html;charset=iso-8859-1
< Cookie: 52d4f682f884de63b52ae34622c7f3968acfc365d02327e2eec34f1f8e1
< Server: Jetty(9.4.33.v20201020)
< X-Content-Type-Options: nosniff
< X-Remoting-Capability: rO0ABXNyABpod4ucmVtbpbmcuQ2FwYWJpbGl0eQAAAAAAAAABAgABSgAEbWFza3hwAAAAf4=
< X-Remoting-Minimum-Version: 3.14

SEVERE hudson.remoting.jnlp.Main$CuiListener error: Handshake error.
io.jenkins.remoting.shaded.javax.websocket.DeploymentException: Handshake error.
    at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.ClientManager$3$1.run(ClientManager.java:674)
    at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.ClientManager$3.run(ClientManager.java:712)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.ClientManager$SameThreadExecutorService.execute(ClientManager.java:866)
    at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
    at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:511)
    at io.jenkins.remoting.shaded.org.glassfish.tyrus.client.ClientManager.connectToServer(ClientManager.java:355)
    at hudson.remoting.Engine.runWebSocket(Engine.java:628)
    at hudson.remoting.Engine.run(Engine.java:470)
Caused by: io.jenkins.remoting.shaded.org.glassfish.tyrus.core.HandshakeException: Response code was not 101: 400.

在我发现的 Jenkins 主节点日志中（系统日志级别设置为 ALL）：

警告 o.e.j.w.s.WebSocketServerFactory#isUpgradeRequest: 不是 'HTTP/1.1' 请求（原为 [HTTP/1.0]）

问题是
为什么 apache 反向代理将 HTTP/1.1 切换到 HTTP/1.0 以便 websocket 与后端通信，以及如何修复它？
我在 apache 中尝试了许多配置，但都没有运气（即 force-no-vary）

参考
Apache responses with http/1.0 even if request is http/1.1
Jenkins: How to configure Jenkins behind Nginx reverse proxy for JNLP slaves to connect

Answer 1

对于第二部分（如何修复它），对 apache 配置的以下添加对我有用。它使用了这个问题的答案：WebSocket through SSL with Apache reverse proxy

具体来说：

# allow for upgrading to websockets
RewriteEngine On
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /(.*)           ws://localhost:8080/$1 [P,L]
RewriteCond %{HTTP:Upgrade} !=websocket [NC]
RewriteRule /(.*)           http://localhost:8080/$1 [P,L]

ProxyPass "/jenkins/wsagents" "ws://localhost:8080/jenkins/wsagents"
ProxyPassReverse "/jenkins/wsagents" "ws://localhost:8080/jenkins/wsagents"

启用 mod_rewrite 和 mod_proxy_wstunnel。