【发布时间】：2019-12-13 05:23:45
【问题描述】：
我的 Jenkins 在 Kubernetes 中运行，服务类型：LoadBalancer，并在 azure 注释下方添加以获取内部子网私有 IP 地址以在内部公开服务。
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
service.beta.kubernetes.io/azure-load-balancer-internal-subnet: subnetName
我有一个安装了 Apache 的 ubuntu VM。创建自签名证书并在 apache 配置中终止，我可以使用 HTTPS 访问 apache 主页。
然后我为 Jenkins 服务 IP 地址创建了代理规则。基本上我想从 Apache HTTPS 访问 Jenkins --> 以内部 HTTP 流量到 kubernetes 服务。
这里是 Apache 配置：
xxxx@xxxx:/etc/apache2/sites-available$ ls -ltrh
total 28K
-rw-r--r-- 1 root root 1332 Jul 16 18:14 000-default.conf
-rw-r--r-- 1 root root 6338 Jul 16 18:14 default-ssl.conf
drwxr-xr-x 2 root root 4096 Dec 12 17:24 abc
-rw-r--r-- 1 root root 680 Dec 12 13:04 abc.conf
drwxr-xr-x 2 root root 4096 Dec 12 14:29 xyz
-rw-r--r-- 1 root root 1151 Dec 12 13:08 xyz.conf
cat abc/00-redirect-to-https.conf
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^jenkins$ login [L,R=302]
cat abc.conf
<VirtualHost *:80>
ServerAdmin webmaster@localhost
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/abc_error.log
CustomLog ${APACHE_LOG_DIR}/abc_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "http"
RequestHeader set X-Forwarded-Port "80"
</IfModule>
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/abc/*.conf
</VirtualHost>
cat xyz/00-jenkins.conf
ProxyPass /jenkins balancer://jenkins/jenkins
ProxyPassReverse /jenkins balancer://jenkins/jenkins
<Proxy balancer://jenkins>
BalancerMember http://x.x.x.x:8080 loadfactor=1 keepalive=On retry=0
ProxySet lbmethod=bytraffic
</Proxy>
cat xyz.conf
<VirtualHost *:443>
ServerAdmin webmaster@localhost
ServerName FQDN
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/xyz_error.log
CustomLog ${APACHE_LOG_DIR}/xyz_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
</IfModule>
SSLEngine on
SSLProtocol -ALL +TLSv1 +TLSv1.1 +TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
SSLCertificateFile /etc/apache2/certs/ca.cert
SSLCertificateKeyFile /etc/apache2/certs/ca.key
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/xyz/*.conf
</VirtualHost>
如果我从本地 ubuntu VM 执行 curl -k https://localhost/jenkins，则响应显示需要身份验证，如下所示，但重定向 url 变为 window.location.replace('/login?from=%2F')
<html><head><meta http-equiv='refresh' content='1;url=/login?from=%2Fjenkins'/><script>window.location.replace('/login?from=%2Fjenkins');</script></head><body style='background-color:white; color:white;'>
Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body></html>
但在同样的情况下，当我再次从浏览器https://FQDN/jenkins 请求时，URL 变为https://FQDN/login?from=%2F 但是浏览器会抛出 URL Not found 错误
Not Found
The requested URL was not found on this server.
请在此处协助更正配置..
谢谢..
更多日志观察：
当我执行curl -k https://localhost/jenkins 时，apache 访问日志显示 403 没问题，因为我没有通过凭据
127.0.0.1 - - [13/Dec/2019:13:37:40 +0000] "GET /jenkins HTTP/1.1" 403 3297 "-" "curl/7.58.0"
当从 Internet 浏览器 https://FQDN/jenkins 尝试相同时，apache 日志首先显示 403，这是需要的，但在 apache 尝试在同一 VM 中查找更改的 url 而不是重定向之后不久，由于我没有得到 jenkins 页面。
165.225.106.137 - - [13/Dec/2019:13:38:19 +0000] "GET /jenkins HTTP/1.1" 403 3446 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
165.225.106.137 - - [13/Dec/2019:13:38:19 +0000] "GET /jenkins HTTP/1.1" 403 1564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
165.225.106.137 - - [13/Dec/2019:13:38:20 +0000] "GET /login?from=%2F HTTP/1.1" 404 541 "https://DNSname/jenkins" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
【问题讨论】：
-
https://FQDN/jenkins中的 FQDN 是否代表 UbuntuVM 的 FQDN？您是否可以通过外部主机上的浏览器访问托管在 Ubuntu VM 上的任何其他页面？ -
@Nick 是的 FQDN 是 UbuntuVM 的 FQDN。我试过调用FQDN，它工作正常，是的，已经在/var/www/html/test.html 中放置了一个test.html，它通过从外部主机调用FQDN/test.html 来响应
-
不确定
ProxyPass /jenkins balancer://jenkins/jenkins是否应该匹配<Proxy balancer://jenkins>，所以/jenkins 将转到http://x.x.x.x:8080/jenkins/jenkins。需要检查一下。 -
@Nick 感谢您的回复。我已经找到了解决方案，请在下面的答案部分找到。
标签： apache jenkins kubernetes proxy reverse-proxy