使用 nginx 作为反向代理的 Kibana 只会导致 404 错误

Question

我正在使用docker-compose 运行一个项目的基础架构，其中一部分包括一个 ELK 堆栈。我使用nginx 作为kibana 的反向代理（除其他外，处理使用 LetsEncrypt 作为 CA 的 SSL）。

当我尝试通过 https://my.host.com/kibana 访问 kibana 实例时，我得到一个 JSON 编码的 404 页面。我还在 kibana 的日志中发现了大量错误（见下文）。我尝试修改server.host、server.basePath、server.rewriteBasePath、nginx 的proxy_redirect（将其设置为off）。没有什么真正有帮助的。任何帮助将不胜感激。

这是我的kibana.yml：

elasticsearch.hosts: ["http://elasticsearch:9200"]
logging.dest: stdout
server.host: kibana
server.basePath: /kibana
server.rewriteBasePath: true

这是我nginx.conf 中的相关部分：

  location /kibana {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        Connection "Keep-Alive";
    proxy_set_header        Proxy-Connection "Keep-Alive";

    proxy_pass              http://kibana:5601/;
    proxy_read_timeout      90;
    proxy_redirect          http://kibana:5106 https://my.host.com/kibana;
    proxy_buffering         off;
  }

这是docker-compose.yml的相关部分：

  nginx:
    build: ./nginx
    depends_on:
      - logstash
      - kibana
    networks:
      - backend
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/www:/var/www
      - /etc/letsencrypt:/var/www-secure

  logstash:
    build: ./logstash
    depends_on:
      - elasticsearch
    networks:
      - backend

  elasticsearch:
    image: elasticsearch:7.6.1
    environment:
      discovery.type: single-node
    networks:
      - backend

  kibana:
    build: ./kibana
    networks:
      - backend

我在 kibana 容器的日志中得到的错误是这样的：

{"type":"response","@timestamp":"2020-03-29T22:36:12Z","tags":[],"pid":6,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"my.host.com","x-real-ip":"185.200.118.103","x-forwarded-for":"185.200.118.103","x-forwarded-proto":"https","connection":"Keep-Alive","proxy-connection":"Keep-Alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:74.0) Gecko/20100101 Firefox/74.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-GB,en;q=0.5","accept-encoding":"gzip, deflate, br","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"172.28.0.10","userAgent":"172.28.0.10"},"res":{"statusCode":404,"responseTime":188,"contentLength":9},"message":"GET / 404 188ms - 9.0B"}
{"type":"error","@timestamp":"2020-03-29T22:36:12Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)","name":"UnhandledPromiseRejectionWarning","stack":"UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)\n    at emitWarning (internal/process/promises.js:97:15)\n    at emitPromiseRejectionWarnings (internal/process/promises.js:143:7)\n    at process._tickCallback (internal/process/next_tick.js:69:34)"},"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"error","@timestamp":"2020-03-29T22:36:12Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 7)","name":"UnhandledPromiseRejectionWarning","stack":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 7)"}
{"type":"response","@timestamp":"2020-03-29T22:36:13Z","tags":[],"pid":6,"method":"get","statusCode":404,"req":{"url":"/","method":"get","headers":{"host":"my.host.com","x-real-ip":"185.200.118.103","x-forwarded-for":"185.200.118.103","x-forwarded-proto":"https","connection":"Keep-Alive","proxy-connection":"Keep-Alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:74.0) Gecko/20100101 Firefox/74.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","accept-language":"en-GB,en;q=0.5","accept-encoding":"gzip, deflate, br","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"172.28.0.10","userAgent":"172.28.0.10"},"res":{"statusCode":404,"responseTime":160,"contentLength":9},"message":"GET / 404 160ms - 9.0B"}
{"type":"error","@timestamp":"2020-03-29T22:36:13Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)","name":"UnhandledPromiseRejectionWarning","stack":"UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)\n    at emitWarning (internal/process/promises.js:97:15)\n    at emitPromiseRejectionWarnings (internal/process/promises.js:143:7)\n    at process._tickCallback (internal/process/next_tick.js:69:34)"},"message":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"}
{"type":"error","@timestamp":"2020-03-29T22:36:13Z","tags":["warning","process"],"pid":6,"level":"error","error":{"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 12)","name":"UnhandledPromiseRejectionWarning","stack":"TypeError: Cannot read property 'statusCodes' of undefined\n    at Network._server.events.on (/usr/share/kibana/node_modules/oppsy/lib/network.js:56:38)\n    at module.exports.internals.Podium.internals.Podium.emit (/usr/share/kibana/node_modules/podium/lib/index.js:220:49)\n    at Request._finalize (/usr/share/kibana/node_modules/hapi/lib/request.js:406:27)\n    at Request._reply (/usr/share/kibana/node_modules/hapi/lib/request.js:345:14)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 12)"}

Answer 1

kibana server.rewriteBasePath 选项似乎无法按预期工作。我通过更新 nginx 配置进行显式重写来解决此问题，并将 rewriteBasePath 设置为 false。

新的 nginx.conf：

  location /kibana {
    proxy_set_header        Host $host;
    proxy_set_header        X-Real-IP $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        Connection "Keep-Alive";
    proxy_set_header        Proxy-Connection "Keep-Alive";

    proxy_pass              http://kibana:5601/;
    proxy_read_timeout      90;
    proxy_redirect          http://kibana:5106 https://my.host.com/kibana;
    rewrite                 /kibana/(.*)$ /$1 break;
    proxy_buffering         off;
  }

新的 kibana.yml：

elasticsearch.hosts: ["http://elasticsearch:9200"]
logging.dest: stdout
server.host: kibana
server.basePath: /kibana
server.rewriteBasePath: false

（我已将此报告为a bug in the kibana issue tracker。）

Answer 2

没有kibana.yml，只需在docker-compose.yml 中使用environment：

  kib01:
    image: docker.elastic.co/kibana/kibana:7.10.2
    ports:
      - 5601:5601
    environment:
      - ELASTICSEARCH_URL=http://es01:9200
      - ELASTICSEARCH_HOSTS=http://es01:9200
      - ELASTICSEARCH_USERNAME=elastic
      - "ELASTICSEARCH_PASSWORD={{ elastic_password }}"
      - SERVER_BASEPATH=/kibana

在 nginx 中：

    location /kibana {
        rewrite ^/kibana(.*)$ $1 break;
        proxy_pass http://kib01:5601;
    }