尝试创建命名空间时出现 Terraform/GCP Kubernetes 错误答案

【问题标题】：Terraform/GCP Kubernetes error when trying to create namespace尝试创建命名空间时出现 Terraform/GCP Kubernetes 错误
【发布时间】：2021-01-23 12:23:36
【问题描述】：

我的 terraform 配置出现以下错误。

    Error: Post "https://35.224.178.141/api/v1/namespaces": x509: certificate signed by unknown authority

  on main.tf line 66, in resource "kubernetes_namespace" "example":
  66: resource "kubernetes_namespace" "example" {

这是我的配置，我现在要做的就是用它创建一个集群身份验证，并创建一个命名空间。我已经搜索了每个人，但看不到其他人在哪里遇到了这个问题。这很可能是我正在做的愚蠢的事情。我认为这会相对简单，但结果却很痛苦。我不想在构建脚本中包含 gcloud 命令。

provider "google" {
  project = var.project
  region  = var.region
  zone    = var.zone
  credentials = "google-key.json"
}


terraform {
  backend "gcs" {
    bucket = "tf-state-bucket-devenv"
    prefix = "terraform"
    credentials = "google-key.json"
   }
}

resource "google_container_cluster" "my_cluster" {
  name     = var.kube-clustername
  location = var.zone
  remove_default_node_pool = true
  initial_node_count       = 1

  master_auth {
    username = ""
    password = ""

    client_certificate_config {
      issue_client_certificate = false
    }
  }
}

resource "google_container_node_pool" "primary_preemptible_nodes" {
  name       = var.kube-poolname
  location   = var.zone
  cluster    = google_container_cluster.my_cluster.name
  node_count = var.kube-nodecount

  node_config {
    preemptible  = var.kube-preemptible
    machine_type = "n1-standard-1"
    disk_size_gb = 10
    disk_type = "pd-standard"


    metadata = {
      disable-legacy-endpoints = "true",
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]
  }
}
data "google_client_config" "provider" {}

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
  token = "{data.google_client_config.provider.access_token}"
}


resource "kubernetes_namespace" "example" {
  metadata {
    name = "my-first-namespace"
  }
}

【问题讨论】：

这不是这个问题的答案，但可以为替代解决方案提供一些启示：stackoverflow.com/questions/63782742/…。

标签： google-cloud-platform terraform google-kubernetes-engine terraform-provider-gcp

【解决方案1】：

TL;DR

将提供者定义更改为：

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
  token = data.google_client_config.provider.access_token
}

发生了什么变化？

"{}" 已从 cluster_ca_certificate 和 token 值中删除

我在下面附上了解释。

我使用了您的原始 terraform 文件，并且收到了与您相同的错误。我修改（简化）了您的 terraform 文件并添加了输出定义：

resource "google_container_cluster" "my_cluster" {
  OMMITED 
}

data "google_client_config" "provider" {}

provider "kubernetes" {
  load_config_file = false
  host = "https://${google_container_cluster.my_cluster.endpoint}"
  cluster_ca_certificate = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
  token = "{data.google_client_config.provider.access_token}"
}


output "cert" {
  value = "{base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}"
}

output "token" {
  value = "{data.google_client_config.provider.access_token}"
}

运行上面的文件显示：

$ terraform apply --auto-approve

data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creating...
google_container_cluster.my_cluster: Creation complete after 2m48s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

cert = {base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)}
token = {data.google_client_config.provider.access_token}

如您所见，这些值被解释为来自提供程序的字符串，而不是“处理”以获得所需的值。要解决此问题，您需要将提供程序定义更改为：

  cluster_ca_certificate = base64decode(google_container_cluster.my_cluster.master_auth.0.cluster_ca_certificate)
  token = data.google_client_config.provider.access_token

再次运行$ terraform apply --auto-approve：

data.google_client_config.provider: Refreshing state...
google_container_cluster.my_cluster: Creation complete after 3m18s [id=projects/PROJECT-NAME/locations/europe-west3-c/clusters/gke-terraform]
kubernetes_namespace.example: Creating...
kubernetes_namespace.example: Creation complete after 0s [id=my-first-namespace]

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

cert = -----BEGIN CERTIFICATE-----
MIIDKzCCAhOgAwIBAgIRAO2bnO3FU6HZ0T2u3XBN1jgwDQYJKoZIhvcNAQELBQAw
<--OMMITED-->
a9Ybow5tZGu+fqvFHnuCg/v7tln/C3nVuTbwa4StSzujMsPxFv4ONVl4F4UaGw0=
-----END CERTIFICATE-----

token = ya29.a0AfH6SMBx<--OMMITED-->fUvCeFg

如您所见，命名空间已创建。您可以通过运行来检查它：

$ gcloud container clusters get-credentials CLUSTER-NAME --zone=ZONE
$ kubectl get namespace my-first-namespace

输出：

NAME                 STATUS   AGE
my-first-namespace   Active   3m14s

其他资源：

【讨论】：