Traefik Proxy 后面的 Mailcow 设置导致 https 证书错误

Question

我正在尝试在Traefik 代理后面设置mailcow 安装。显然，Traefik 代理无法识别其网络中的nginx-mailcow 容器，因此不会为https 连接创建证书。因此，当我使用 docker-compose up 启动 mailcow 服务时，我可以访问 mailcow 服务，但连接不安全 (http) 并且浏览器警告连接不安全。

当我从Traefik 检查我的acme.json 文件时：我找不到任何与mailcow 域相关的证书，即mail.tld.com。

我有以下设置：

受影响容器的日志：

Traefik 容器日志：

time="2020-04-18T13:40:35+02:00" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2020-04-18T13:40:35+02:00" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=https
time="2020-04-18T13:40:35+02:00" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2020-04-18T13:40:35+02:00" level=error msg="close tcp [::]:443: use of closed network connection" entryPointName=https
time="2020-04-18T13:40:35+02:00" level=error msg="Cannot connect to docker server context canceled" providerName=docker
time="2020-04-18T13:40:37+02:00" level=info msg="Configuration loaded from file: /traefik.yml"
time="2020-04-19T00:27:31+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" container=nginx-mailcow-mailcowdockerized-5f3a25b43c42fd85df675d2d9682b6053501844c2cfe15b7802cf918df138025 providerName=docker
time="2020-04-19T00:33:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-f4d41ee79e382b413e04b039b5fc91e1c6217c78740245c8666373fe2d6a9b23
2020/04/19 00:39:44 reverseproxy.go:445: httputil: ReverseProxy read error during body copy: unexpected EOF
time="2020-04-19T00:50:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-915f80e492c2c22917d0af81add1dde15577173c82cc928b0b6101c8a260adc5
time="2020-04-19T00:58:43+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" container=nginx-mailcow-mailcowdockerized-852985c4efc48559ca3568b1829e31b46eb9f968fc328a8566e3dc6ab6f1af21 providerName=docker
time="2020-04-19T02:02:39+02:00" level=error msg="Error while Peeking first byte: read tcp 172.21.0.2:80->208.91.109.90:55153: read: connection reset by peer"
time="2020-04-19T08:11:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-840ef4db0ccc9fa84038dc7a52133779926dba4c51554516c17404ede80a2c01

Traefikdocker-compose.yml的内容：

version: '3'

services:
  traefik:
    image: traefik:v2.1
    container_name: traefik
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - proxy
    ports:
      - 80:80
      - 443:443
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data/traefik.yml:/traefik.yml:ro
      - ./data/acme.json:/acme.json
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.traefik.entrypoints=http"
      - "traefik.http.routers.traefik.rule=Host(`traefik.tld.com`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=admin:pass"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=https"
      - "traefik.http.routers.traefik-secure.rule=Host(`traefik.tld.com`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=http"
      - "traefik.http.routers.traefik-secure.service=api@internal"

networks:
  proxy:
    external: true

traefik.yml 的内容（我用 .yml 而不是 .toml）

api:
    dashboard: true

entryPoints:
    http:
        address: ":80"
    https:
        address: ":443"

providers:
    docker:
        endpoint: "unix:///var/run/docker.sock"
        exposedByDefault: false

certificatesResolvers:
    http:
        acme:
            email: myemail@tld.com
            storage: acme.json
            httpChallenge:
                entryPoint: http

需要指出的是，使用Traefik 的这种设置，会自动为gitlab 等其他服务生成证书。为此，我只是正确地标记了gitlab 服务并将Traefik 网络分配给它，Traefik 服务将识别gitlab 服务并在acme.json 中生成证书，但遗憾的是不适用于nginx-mailcow。

我的docker-compose.override.yml邮件内容：

version: '2.1'

services:
  nginx-mailcow:
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.nginx-mailcow.entrypoints=http"
      - "traefik.http.routers.nginx-mailcow.rule=HostRegexp(`{host:(autodiscover|autoconfig|webmail|mail|email).+}`)"
      - "traefik.http.middlewares.nginx-mailcow-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.nginx-mailcow.middlewares=nginx-mailcow-https-redirect"
      - "traefik.http.routers.nginx-mailcow-secure.entrypoints=https"
      - "traefik.http.routers.nginx-mailcow-secure.rule=Host(`mail.tld.com`)" 
      - "traefik.http.routers.nginx-mailcow-secure.tls=true"
      - "traefik.http.routers.nginx-mailcow-secure.service=nginx-mailcow"
      - "traefik.http.services.nginx-mailcow.loadbalancer.server.port=80"
      - "traefik.docker.network=proxy"
    networks:
      proxy:

  certdumper:
      image: humenius/traefik-certs-dumper
      container_name: traefik_certdumper
      network_mode: none
      command: --restart-containers mailcowdockerized_postfix-mailcow_1,mailcowdockerized_dovecot-mailcow_1
      volumes:
        - /opt/containers/traefik/data:/traefik:ro
        - /var/run/docker.sock:/var/run/docker.sock:ro
        - ./data/assets/ssl:/output:rw
      environment:
        - DOMAIN=tld.com 

networks:
  proxy:
    external: true

docker-compose.yml中我的nginx-mailcow服务的内容

version: '2.1'
services:
     ...
    nginx-mailcow:
      depends_on:
        - sogo-mailcow
        - php-fpm-mailcow
        - redis-mailcow
      image: nginx:mainline-alpine
      dns:
        - ${IPV4_NETWORK:-172.22.1}.254
      command: /bin/sh -c "envsubst < /etc/nginx/conf.d/templates/listen_plain.template > /etc/nginx/conf.d/listen_plain.active &&
        envsubst < /etc/nginx/conf.d/templates/listen_ssl.template > /etc/nginx/conf.d/listen_ssl.active &&
        envsubst < /etc/nginx/conf.d/templates/server_name.template > /etc/nginx/conf.d/server_name.active &&
        envsubst < /etc/nginx/conf.d/templates/sogo.template > /etc/nginx/conf.d/sogo.active &&
        envsubst < /etc/nginx/conf.d/templates/sogo_eas.template > /etc/nginx/conf.d/sogo_eas.active &&
        . /etc/nginx/conf.d/templates/sogo.auth_request.template.sh > /etc/nginx/conf.d/sogo_proxy_auth.active &&
        . /etc/nginx/conf.d/templates/sites.template.sh > /etc/nginx/conf.d/sites.active &&
        nginx -qt &&
        until ping phpfpm -c1 > /dev/null; do sleep 1; done &&
        until ping sogo -c1 > /dev/null; do sleep 1; done &&
        until ping redis -c1 > /dev/null; do sleep 1; done &&
        until ping rspamd -c1 > /dev/null; do sleep 1; done &&
        exec nginx -g 'daemon off;'"
      environment:
        - HTTPS_PORT=${HTTPS_PORT:-443}
        - HTTP_PORT=${HTTP_PORT:-80}
        - MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
        - IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
        - TZ=${TZ}
        - ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
      volumes:
        - ./data/web:/web:ro
        - ./data/conf/rspamd/dynmaps:/dynmaps:ro
        - ./data/assets/ssl/:/etc/ssl/mail/:ro
        - ./data/conf/nginx/:/etc/nginx/conf.d/:rw
        - ./data/conf/rspamd/meta_exporter:/meta_exporter:ro
        - sogo-web-vol-1:/usr/lib/GNUstep/SOGo/
      ports:
        - "${HTTPS_BIND:-0.0.0.0}:${HTTPS_PORT:-443}:${HTTPS_PORT:-443}"
        - "${HTTP_BIND:-0.0.0.0}:${HTTP_PORT:-80}:${HTTP_PORT:-80}"
      restart: always
      networks:
        mailcow-network:
          aliases:
            - nginx

....

我也尝试在nginx-mailcow 服务中注释掉端口，但问题仍然存在。我当前的mailcow.conf 更改：

HTTP_BIND=127.0.0.1
HTTP_PORT=8080
HTTPS_BIND=127.0.0.1
HTTPS_PORT=8443

SKIP_LETS_ENCRYPT=y
SKIP_CLAMD=y

上述错误的再现：

我首先设置了 traefik 代理（参见上面的内容）。一旦 Traefik 启动并运行（我还测试了其他服务，它可以正常生成证书）。现在首先我克隆了 mailcow 存储库。然后我运行./generate_config.sh 来生成mailcow.conf 文件。作为 generate_config.sh 的输入，我提供了我的域名，即mail.tld.com

然后我注释掉docker-compose.yml 文件中的端口，因为我不想为nginx-mailcow 使用端口80 和443，因为Traefik 已经在使用这些端口。

然后我创建一个docker-compose.override.yml（参见上面的内容）以向nginx-mailcow 服务（traefik 标签、traefik 网络）添加其他配置。覆盖文件还包含 certdumper 服务，它将 https 证书从 acme.json 复制到 mailcow 服务。

然后，我在mailcow.conf中更改以下两个变量：

SKIP_LETS_ENCRYPT=y
SKIP_CLAMD=y

最后，我使用docker-compose up -d 运行mailcow。在浏览器中，如果检查https://mail.tld.com => 它会警告连接不安全。如果我检查acme.json。我找不到mail.tld.com 的证书。

系统信息：

+-------------------------------------------------+---------------------------------+
|                    Question                     |             Answer              |
+-------------------------------------------------+---------------------------------+
| My operating system                             | linux x86_64 Ubuntu 18.04.1 LTS |
| Is Apparmor, SELinux or similar active?         | No                              |
| Virtualization technlogy                        | KVM                             |
| Server/VM specifications (Memory, CPU Cores)    | 16GB, 6 cores                   |
| Docker Version (docker version)                 | 19.03.8                         |
| Docker-Compose Version (docker-compose version) | 1.25.4, build 8d51620a          |
| Reverse proxy (custom solution)                 | Traefik                         |
+-------------------------------------------------+---------------------------------+

如果您需要更多信息，我很乐意提供。任何帮助都感激不尽。谢谢。

Answer 1

在花费大量时间阅读 Traefik 文档后，我终于解决了这个问题。我在为 nginx-mailcow 服务分配代理标签时犯了一个小错误。解决方法如下。

我忘了提及证书解析器，我不得不公开我现在添加的端口，如下所示：

services:
  nginx-mailcow:
    expose:
      - "8080"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.nginx-mailcow.entrypoints=http"
      - "traefik.http.routers.nginx-mailcow.rule=HostRegexp(`{host:(autodiscover|autoconfig|webmail|mail|email).+}`)"
      - "traefik.http.middlewares.nginx-mailcow-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.nginx-mailcow.middlewares=nginx-mailcow-https-redirect"
      - "traefik.http.routers.nginx-mailcow-secure.entrypoints=https"
      - "traefik.http.routers.nginx-mailcow-secure.rule=Host(`mail.example.com`)" 
      - "traefik.http.routers.nginx-mailcow-secure.tls=true"
      - "traefik.http.routers.nginx-mailcow-secure.certresolver=http"
      - "traefik.http.routers.nginx-mailcow-secure.service=nginx-mailcow"
      - "traefik.http.services.nginx-mailcow.loadbalancer.server.port=8080"
      - "traefik.docker.network=proxy"
    networks:
      proxy:

  certdumper:
    image: humenius/traefik-certs-dumper
    container_name: traefik_certdumper
    network_mode: none
    command: --restart-containers mailcowdockerized_postfix-mailcow_1,mailcowdockerized_dovecot-mailcow_1
    volumes:
      - <path_to_acme.json_file_dir>:/traefik:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./data/assets/ssl:/output:rw
    environment:
      - DOMAIN=example.com 

对于第一次设置的人，我必须事先进行一些额外的更改。

1. 首先，当您运行 generate.sh 文件时，您需要在 mailcow.conf 文件中进行以下更改：

   HTTP_PORT=8080
   HTTP_BIND=127.0.0.1

   HTTPS_PORT=8443
   HTTPS_BIND=127.0.0.1

   SKIP_LETS_ENCRYPT=y
   SKIP_CLAMD=y

   我们进行这些更改是因为我们无法在与 traefik 相同的端口上运行 mailcow nginx。

2. 现在 nginx-mailcow 将在 8080 或 8443 上运行，因此我们需要公开这些端口之一，以便 traefik 可以与 mailcow-nginx 服务通信。我已经在覆盖撰写文件中暴露了端口 8080）

3. 您还需要将负载均衡器端口从 80 调整为 8080。（正如我在上面配置的）
4. 您还需要告诉它应该使用哪个证书解析器。所以你需要在标签中添加这一行（我在上面的覆盖配置中也这样做了）
5. 您必须确保您的 acme.json 文件（证书文件可由 certdumper 服务访问）。所以替换为acme.json目录路径的实际路径

我希望这会有所帮助。