通过 Terraform Helm 提供程序和 Azure DevOps 部署 helm 图表，同时从 ACR 获取 helm 图表

Question

我正在尝试使用 Terraform helm 提供程序和 Azure DevOps 容器作业将 helm 图表从 ACR 部署到 AKS 集群，但在从 ACR 获取 helm 图表时失败。请让我知道出了什么问题。

helm provider tf 模块：

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io"
}
# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

提供者：

  version                    = "=1.36.0"
  tenant_id                  = var.ARM_TENANT_ID
  subscription_id            = var.ARM_SUBSCRIPTION_ID
  client_id                  = var.ARM_CLIENT_ID
  client_secret              = var.ARM_CLIENT_SECRET
  skip_provider_registration = true
}

data "azurerm_kubernetes_cluster" "aks_cluster" {
  name                = var.aks_cluster
  resource_group_name = var.resource_group_aks
}

locals {
  kubeconfig_path = "/tmp/kubeconfig"
}

resource "local_file" "kubeconfig" {
  filename = local.kubeconfig_path
  content  = data.azurerm_kubernetes_cluster.aks_cluster.kube_admin_config_raw
}

provider "helm" {
  home = "resources/.helm"
  kubernetes {
    load_config_file = true
    config_path = local.kubeconfig_path
  }
}

module "aks_resources" {
  source = "./modules/helm/aks-resources"
}

错误： 错误：看起来“”不是有效的图表存储库或无法访问：无法获取 /index.yaml : 404 Not Found

Answer 1

到目前为止，Helm 仍然不支持直接从 OCI 注册表安装图表。

推荐的步骤是：

1. helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
2. helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
3. helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
4. cd install & helm install myhelmtest ./hello-world

所以我的解决方案是：

resource "null_resource" "download_chart" {
  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login mycontainerregistry.azurecr.io --username someuser --password somepass
      helm chart remove mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart pull mycontainerregistry.azurecr.io/helm/hello-world:v1
      helm chart export mycontainerregistry.azurecr.io/helm/hello-world:v1 --destination ./install
    EOT
  }
}

resource "helm_release" "chart" {
  name             = "hello_world"
  repository       = "./install"
  chart            = "hello-world"
  version          = "v1"

  depends_on = [null_resource.download_chart]
}

不完美但有效。

Answer 2

问题是您在 Terraform helm_repository 中使用了错误的 url。 ACR 的右 url 如下所示：

https://acrName.azurecr.io/helm/v1/repo

而且 ACR 是一个私有注册表，因此这意味着您需要为其添加用户名和密码。最后，您的 Terraform 代码应该与 2.0+ 版本的 helm provider 一样：

resource "helm_release" "my-chart" {
  name  = "my-chart"
  chart = "my/chart"
  repository  = "https://${var.acr_name}.azurecr.io/helm/v1/repo"
  repository_username = var.acr_user_name
  repository_password = var.acr_user_password
}

或使用 1.x helm 提供程序：

data "helm_repository" "cluster_rbac_helm_chart_repo" {
  name = "mcp-rbac-cluster"
  url  = "https://mcpshareddcr.azurecr.io/helm/v1/repo"

  username = "xxxxx"
  password = "xxxxx"
}

# Deploy Cluster RBAC helm chart onto the cluster
resource "helm_release" "cluster_rbac_helm_chart_release" {
  name  = "mcp-rbac-cluster"
  repository = data.helm_repository.cluster_rbac_helm_chart_repo.metadata[0].name
  chart = "mcp-rbac-cluster"
}

更新

这是运行良好并在 AKS 中部署图表的屏幕截图：

Answer 3

对上述解决方案的小幅增强。包括一个触发器以强制每次下载图表。否则，它希望您在第一次部署后始终维护图表的本地副本

resource "null_resource" "download_chart" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = <<-EOT
      export HELM_EXPERIMENTAL_OCI=1
      helm registry login ${var.registry_fqdn} --username ${var.acr_client_id} --password ${var.acr_client_secret}
      helm chart remove ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart pull ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag}
      helm chart export ${var.registry_fqdn}/helm/${var.chart_name}:${var.chart_tag} --destination ./install
    EOT
  }
}