无法从 X-Forwarded-For 获取真实用户的 IP

Question

我在单节点 Kubernetes 集群上运行 Traefik 1.7.3，我试图从 X-Forwarded-For 标头中获取真实用户 IP，但我得到的是 X-Forwarded-For: 10.244.0.1，这是我的 k8s 中的 IP集群。

这是我的 Traefik 部署和服务：

    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: traefik-conf
    data:
      traefik.toml: |
        # traefik.toml
        debug = true
        logLevel = "DEBUG"
        defaultEntryPoints = ["http","https"]
        [entryPoints]
          [entryPoints.http]
          address = ":80"
          compress = true
          [entryPoints.http.forwardedHeaders]
          trustedIPs = [ "0.0.0.0/0" ]
          entryPoint = "https"
          [entryPoints.https]
          address = ":443"
          compress = true
          [entryPoints.https.forwardedHeaders]
          trustedIPs = [ "0.0.0.0/0" ]
          [entryPoints.https.tls]

        [acme]
        email = "xxxx"
        storage = "/acme/acme.json"
        entryPoint = "https"
        onHostRule = true
        #caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
        acmeLogging = true
        [[acme.domains]]
        main = "xxxx"
        [acme.dnsChallenge]
        provider = "route53"
        delayBeforeCheck = 0

        [persistence]
        enabled = true
        existingClaim = "pvc0"
        annotations = {}
        accessMode = "ReadWriteOnce"
        size = "1Gi"

        [kubernetes]
        namespaces = ["default"]
        [accessLog]
        filePath = "/acme/access.log"
        [accessLog.fields]
        defaultMode = "keep"
    ---
    kind: Deployment
    apiVersion: extensions/v1beta1
    metadata:
      name: traefik-ingress-controller
      namespace: default
      labels:
        k8s-app: traefik-ingress-lb
    spec:
      replicas: 1
      selector:
        matchLabels:
          k8s-app: traefik-ingress-lb
      template:
        metadata:
          labels:
            k8s-app: traefik-ingress-lb
            name: traefik-ingress-lb
        spec:
          serviceAccountName: traefik-ingress-controller
          terminationGracePeriodSeconds: 60
          containers:
          - image: traefik
            name: traefik-ingress-lb
            env:
              - name: AWS_ACCESS_KEY_ID
                value: xxxx
              - name: AWS_SECRET_ACCESS_KEY
                value: xxxx
              - name: AWS_REGION
                value: us-west-2
              - name: AWS_HOSTED_ZONE_ID
                value: xxxx
            ports:
            - name: http
              containerPort: 80
            - name: admin
              containerPort: 8080
            args:
            - --api
            - --kubernetes
            - --configfile=/config/traefik.toml
            volumeMounts:
            - mountPath: /config
              name: config
            - mountPath: /acme
              name: acme
          volumes:
          - name: config
            configMap:
              name: traefik-conf
          - name: acme
            persistentVolumeClaim:
              claimName: "pvc0"
    ---
    kind: Service
    apiVersion: v1
    metadata:
      name: traefik-ingress-service
      namespace: default
    spec:
      externalIPs:
      - x.x.x.x
      externalTrafficPolicy: Local
      selector:
        k8s-app: traefik-ingress-lb
      ports:
        - protocol: TCP
          port: 80
          name: web
        - protocol: TCP
          port: 443
          name: https
        - protocol: TCP
          port: 8080
          name: admin
      type: NodePort

这是我的入口：

    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: headers-test
      namespace: default
      annotations:
        ingress.kubernetes.io/proxy-body-size: 500m
        kubernetes.io/ingress.class: traefik
    spec:
      rules:
      - host: xxxx
        http:
          paths:
          - path: /
            backend:
              serviceName: headers-test
              servicePort: 8080

我读到我只需要添加[entryPoints.http.forwardedHeaders] 和trustedIPs 的列表，但这似乎不起作用。我错过了什么吗？

Answer 1

如果您将 NodePort 用于 Traefik Ingress Service，则必须将 service.spec.externalTrafficPolicy 设置为“Local”。否则，当您的连接进入 K8s 集群时，您将拥有 SNAT。如果 Pod 不在同一节点上运行，则此 SNAT 是将传入连接转发到您的 Pod 所必需的。

但请注意，将 service.spec.externalTrafficPolicy 设置为“Local”后，只有执行 Traefik pod 的节点会接受 80、443、8080 上的请求。其他节点不会转发到该 pod了。这可能会导致连接到您的服务时出现奇怪的延迟。为避免您的 Traefik 需要在 HA 设置 (DaemonSet) 中运行。请记住，您需要一个用于分布式 Traefik 设置的 K/V-Store 以使 Letsencrypt 正常工作。

如果 service.spec.externalTrafficPolicy 设置尚未解决您的问题，您可能还需要将 kubernetes 覆盖网络配置为不执行任何 SNAT。

service.spec.externalTrafficPolicy 在这里得到了很好的解释： https://kubernetes.io/docs/tutorials/services/source-ip/