【问题标题】：Terraform AWS datasource aws_subnet returns no matching subnets foundTerraform AWS 数据源 aws_subnet 返回未找到匹配的子网
【发布时间】：2020-03-24 01:10:09
【问题描述】：

我有一个模块，我试图在其中查找 AWS 子网，然后使用/返回。它是这样称呼的：

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"
   private_subnets   = "${var.private_subnets_uswe2}"
}

模块代码：

variable "azs"              { type = "list" }
variable "private_subnets"  { type = "list" }
variable "vpc_id"           {}

# ========== remove special subnets ==============

locals {
  cnt      = "${length(var.private_subnets) - 3}"
  prv_subs = "${slice(var.private_subnets, 0, local.cnt)}"
}

# ========== get subnet details ==================

data "aws_subnet" "self" {
  count         = "${length(local.prv_subs)}"
  vpc_id        = "${var.vpc_id}"
  cidr_block    = "${local.prv_subs[count.index]}"
}

# ========== get subnets by AZ ===================

locals {
  prv_subs0 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[0]))}"
  prv_subs1 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[1]))}"
  prv_subs2 = "${matchkeys(data.aws_subnet.self.*.id, data.aws_subnet.self.*.availability_zone, list(var.azs[2]))}"
}

# ========== select 1 subnet per AZ ==============

resource "random_shuffle" "prv_sub0" {
    input        = ["${local.prv_subs0}"]
    result_count = 1
}
resource "random_shuffle" "prv_sub1" {
    input        = ["${local.prv_subs1}"]
    result_count = 1
}
resource "random_shuffle" "prv_sub2" {
    input        = ["${local.prv_subs2}"]
    result_count = 1
}

# ========== put selected into 1 list ============

locals {
  prv_sub_az = [
       "${random_shuffle.prv_sub0.result}", 
       "${random_shuffle.prv_sub1.result}", 
       "${random_shuffle.prv_sub2.result}"
  ]
}

output "prv_subnet_grp" {
  value = "${local.prv_sub_az}"
}

抛出这个：

Error: Error refreshing state: 1 error occurred:
    * module.smurf_subnet_grp.data.aws_subnet.self: 6 errors occurred:
    * module.smurf_subnet_grp.data.aws_subnet.self[5]: data.aws_subnet.self.5: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[3]: data.aws_subnet.self.3: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[0]: data.aws_subnet.self.0: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[1]: data.aws_subnet.self.1: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[2]: data.aws_subnet.self.2: no matching subnet found
    * module.smurf_subnet_grp.data.aws_subnet.self[4]: data.aws_subnet.self.4: no matching subnet found

如果我为aws_subnet 数据提供者引入depends_on：

data "aws_subnet" "self" {
  count         = "${length(local.prv_subs)}"
  vpc_id        = "${var.vpc_id}"
  cidr_block    = "${local.prv_subs[count.index]}"
  depends_on    = ["null_resource.module_depends_on"]
}

它会按预期工作，但每次都会重新创建它。

更新 #1

为了解决这个问题，我尝试实施在 Hashicorp 论坛上找到的一个建议，标题为：TIPS: Howto implement Module depends_on emulation。理论上我遇到的问题是顺序/依赖问题。

我用来实现depends_on 的代码如下，用于我的subnet_grp_per_az-test 模块：

/*
    Add the following line to the resource in this module that depends on the completion of external module components:

    depends_on = ["null_resource.module_depends_on"]

    This will force Terraform to wait until the dependant external resources are created before proceeding with the creation of the
    resource that contains the line above.

    This is a hack until Terraform officially support module depends_on.
*/

variable "module_depends_on" {
  default = [""]
}

resource "null_resource" "module_depends_on" {
  triggers = {
    value = "${length(var.module_depends_on)}"
  }
}

【问题讨论】：

请分享“null_resource.module_depends_on”代码？我想它知道 module.smurf_subnet_grp.data.aws_subnet.self 部分需要什么依赖才能在创建子网后执行。我猜是“module.networking_uswe2”创建了这些子网，并且在执行 module.smurf_subnet_grp 时还没有创建它们。
我们有一个误解是 terraform 模块，例如“networking_uswe2”是一个原子单元。一个模块中的所有TF资源都会一次性创建，只有在模块创建完成后，才会执行另一个模块“module.smurf_subnet_grp”，但事实并非如此。当我们编写 TF 脚本时，模块就像一个盒子，用于整理组中的资源。但是，Terraform 运行时不会逐盒处理（模块）。 Terraform 会转储所有要创建的资源，并从所有资源中创建一个有向图，不再有框。
因为 "private_subnets = "${var.private_subnets_uswe2}"" 使用的是变量，而不是对子网资源的实际引用，所以 "data "aws_subnet" "self"" 对子网没有依赖关系，并且 Terraform 在创建子网之前或期间开始执行它，这是我喜欢测试的理论。
有一个针对模块的“depends_on”请求，但未触及。据我了解，这是不可能的，因为模块不是原子单元，terraform 可以在其上在资源图中创建节点边。
@mon - 您的回复，我已将代码添加到我的 Q 中。

标签： amazon-web-services terraform

【解决方案1】：

请注意，这是我的理解。希望 M.Atkins 能够确认。

为什么 Terraform 没有模块的依赖项

Module Cannot be Used in depends_on #18239

作为准备，首先想澄清我们对 Terraform module 可能存在的一个误解（我曾有过）。

目标是确定一个 TF 模块不依赖另一个 TF 模块。仅仅因为模块 A 声明在模块 B 之前，如下面的 根模块 tf 文件中，并不意味着在模块 A 中的资源全部完成之前不会在模块 B 中创建资源。

如果我们有两个模块 A 和 B 并且它们相互依赖会发生什么？

根模块

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
#--------------------------------------------------------------------------------
# Create PRIVATE subnets but create EC2 in PUBLIC subnets (cross module reference)
#--------------------------------------------------------------------------------
module "private_subnet_public_ec2" {
  source               = "../private_subnet_public_ec2"
  vpc_id               = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  public_subnet_ids    = module.public_subnet_private_ec2.public_subnet_ids
  ami_id               = data.aws_ami.this.id
}
#--------------------------------------------------------------------------------
# Create PUBLIC subnets but create EC2 in PRIVATE subnets (cross module reference)
#--------------------------------------------------------------------------------
module "public_subnet_private_ec2" {
  source              = "../public_subnet_private_ec2"
  vpc_id              = aws_vpc.this.id
  public_subnet_cidr_blocks = var.public_subnet_cidr_blocks
  private_subnet_ids  = module.private_subnet_public_ec2.private_subnet_ids
  ami_id              = data.aws_ami.this.id
}

模块 A

在模块 B 中创建的公共子网中创建私有子网和 EC2。

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}
output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

resource "aws_instance" "public_ec2" {
  count = length(var.public_subnet_ids)
  subnet_id = var.public_subnet_ids[count.index]
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags = {
    Name = "PublicEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "Public EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

模块 B

在模块 A 中创建的私有子网中创建公共子网和 EC2。

resource "aws_subnet" "public" {
  count = length(var.public_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.public_subnet_cidr_blocks[count.index]
}
output "public_subnet_ids" {
  value = aws_subnet.public[*].id
}

resource "aws_instance" "private_ec2" {
  count = length(var.private_subnet_ids)
  subnet_id = var.private_subnet_ids[count.index]
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags = {
    Name = "privateEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "private EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

执行结果：

$ terraform apply --auto-approve
Apply complete! Resources: 13 added, 0 changed, 0 destroyed.

所以我们需要记住的是 Terraform 实际看到的内容，这是一个没有模块的扁平世界，只有资源存在。 Terraform 从资源中创建 DAG，并且 DAG 中没有 module 作为 node。

这就是我们不能对模块使用 depends_on 的原因，因为在 Terraform DAG 中，模块不是节点，可以创建顶点来表示依赖关系。

同步监控机制

Terraform 以并发方式执行多个线程来创建资源。我们需要一个同步monitor，线程可以等待，以便这些线程仅在创建依赖资源时启动。

Terraform 中的监控机制（HCL 依赖项除外）使用创建的资源的属性（或通过本地引用）。

在TIPS: Howto implement Module depends_on emulation 中，Martin Atkins 展示了一个示例，该示例使用模块变量“vm_depends_on”引用创建的防火墙资源的属性，module.fw_core01.firewall 以便模块中的那些资源“示例”只能在防火墙创建后才能创建。

问题

在我的理解中，这个问题的问题的原因是缺少monitor，对资源属性的引用应该在线程开始执行时已经创建data "aws_subnet" "self" .

由于我没有原始问题的全部来源，这里有一个重现错误的示例。

根模块

variable "vpc_cidr" {
    default = "10.5.0.0/20"
}
variable "private_subnet_cidr_blocks" {
  default = ["10.5.3.0/24","10.5.4.0/24","10.5.5.0/24"]
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
module "private_subnet" {
  source          = "../private_subnet"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  ami_id          = data.aws_ami.this.id
}

模块子网

variable "vpc_id" {}
variable "private_subnet_cidr_blocks" {
  type = list(string)
}

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}

模块 EC2

variable "vpc_id" {}
variable "private_subnet_cidr_blocks" {
  type = list(string)
}
variable "ami_id" {}

data aws_subnet "private" {
  count      = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}

resource "aws_instance" "private_ec2" {
  count         = length(data.aws_subnet.private[*].id)
  subnet_id     = data.aws_subnet.private[count.index].id
  ami           = var.ami_id
  instance_type = "t2.micro"
  tags          = {
    Name = "privateEC2${count.index}}"
  }
  provisioner "local-exec" {
    command = <<EOF
echo "private EC2 ${count.index} ID is ${self.id}"
EOF
  }
}

执行结果：

$ terraform apply --auto-approve
...
Error: no matching subnet found

  on ../private_ec2/main.tf line 1, in data "aws_subnet" "private":
   1: data aws_subnet "private" {

原因

原因是使用常量变量传递私有子网 CIDR，而不是创建的 AWS 子网的属性。

如果使用了创建的 AWS 子网资源的属性，则它作为监视器工作，执行 data "aws_subnet" "self" 的线程将在该监视器上等待。

根模块

variable "private_subnet_cidr_blocks" {
  default = ["10.5.3.0/24","10.5.4.0/24","10.5.5.0/24"]
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks # <----- Here
  ami_id          = data.aws_ami.this.id
}

由于Module Subnet中的资源与Module EC2中的资源没有依赖关系，因此两个Module中的资源创建是并行运行的。

在原问题中

我相信，原始问题的原因是 private_subnets 传递变量，而不是实际创建的 AWS 子网资源的属性。

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"

   private_subnets   = "${var.private_subnets_uswe2}" <----- Here

}

因此，data "aws_subnet" "self" 在创建或尚未创建 AWS 子网时同时执行。

修复

示例的修复：

根模块

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr
}
module "private_subnet" {
  source          = "../private_subnet"
  vpc_id          = aws_vpc.this.id
  private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
}
module "private_ec2" {
  source          = "../private_ec2"
  vpc_id          = aws_vpc.this.id

  #--------------------------------------------------------------------------------
  # Pass the attributes of created aws_subnet resource attributes
  #--------------------------------------------------------------------------------
  #private_subnet_cidr_blocks = var.private_subnet_cidr_blocks
  private_subnet_cidr_blocks = module.private_subnet.private_subnet_cidr_blocks  # <--- Here
  #--------------------------------------------------------------------------------

  ami_id          = data.aws_ami.this.id
}

模块子网

resource "aws_subnet" "private" {
  count = length(var.private_subnet_cidr_blocks)
  vpc_id     = var.vpc_id
  cidr_block = var.private_subnet_cidr_blocks[count.index]
}
#--------------------------------------------------------------------------------
# Output the cidr_block attributes of the AWS subnet resources created
#--------------------------------------------------------------------------------
output "private_subnet_cidr_blocks" {
  value = aws_subnet.private[*].cidr_block    # <----- Here
}

结果

$ terraform apply --auto-approve
data.aws_availability_zones.all: Refreshing state...
aws_vpc.this: Refreshing state... [id=vpc-0b338898d18a5986e]
data.aws_ami.this: Refreshing state...
data.aws_region.current: Refreshing state...
module.private_subnet.data.aws_ami.ubuntu: Refreshing state...
module.private_subnet.aws_subnet.private[2]: Refreshing state... [id=subnet-0cf916b6b9003f71f]
module.private_subnet.aws_subnet.private[1]: Refreshing state... [id=subnet-0b39beb22b23eef5d]
module.private_subnet.aws_subnet.private[0]: Refreshing state... [id=subnet-0c80c92f4023ba893]
aws_vpc.this: Creating...
aws_vpc.this: Still creating... [10s elapsed]
aws_vpc.this: Still creating... [20s elapsed]
aws_vpc.this: Creation complete after 24s [id=vpc-00069d144b5f76182]
module.private_subnet.aws_subnet.private[1]: Creating...
module.private_subnet.aws_subnet.private[2]: Creating...
module.private_subnet.aws_subnet.private[0]: Creating...
module.private_subnet.aws_subnet.private[2]: Creation complete after 5s [id=subnet-0252c6047cd56abac]
module.private_subnet.aws_subnet.private[1]: Creation complete after 6s [id=subnet-019f8cbd30db10edb]
module.private_subnet.aws_subnet.private[0]: Creation complete after 6s [id=subnet-0a1028bf17d7d81be]
module.private_ec2.data.aws_subnet.private[1]: Refreshing state...
module.private_ec2.data.aws_subnet.private[2]: Refreshing state...
module.private_ec2.data.aws_subnet.private[0]: Refreshing state...
module.private_ec2.aws_instance.private_ec2[2]: Creating...
module.private_ec2.aws_instance.private_ec2[1]: Creating...
module.private_ec2.aws_instance.private_ec2[0]: Creating...
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [10s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [20s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [30s elapsed]
module.private_ec2.aws_instance.private_ec2[2]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[0]: Still creating... [40s elapsed]
module.private_ec2.aws_instance.private_ec2[1]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[1] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 1 ID is i-0ced265565dfec85c\"\n"]
module.private_ec2.aws_instance.private_ec2[1] (local-exec): private EC2 1 ID is i-0ced265565dfec85c
module.private_ec2.aws_instance.private_ec2[1]: Creation complete after 46s [id=i-0ced265565dfec85c]
module.private_ec2.aws_instance.private_ec2[0]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[0] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 0 ID is i-0f6ce62c29376c6fe\"\n"]
module.private_ec2.aws_instance.private_ec2[0] (local-exec): private EC2 0 ID is i-0f6ce62c29376c6fe
module.private_ec2.aws_instance.private_ec2[0]: Creation complete after 47s [id=i-0f6ce62c29376c6fe]
module.private_ec2.aws_instance.private_ec2[2]: Provisioning with 'local-exec'...
module.private_ec2.aws_instance.private_ec2[2] (local-exec): Executing: ["/bin/sh" "-c" "echo \"private EC2 2 ID is i-03be32b7b803eb0cc\"\n"]
module.private_ec2.aws_instance.private_ec2[2] (local-exec): private EC2 2 ID is i-03be32b7b803eb0cc
module.private_ec2.aws_instance.private_ec2[2]: Creation complete after 50s [id=i-03be32b7b803eb0cc]

Apply complete! Resources: 7 added, 0 changed, 0 destroyed.

修复原始问题

我相信下面会解决这个问题。

module "smurf_subnet_grp" {
   source            = "../../modules/networking/subnet_grp_per_az-test"
   vpc_id            = "${module.networking_uswe2.vpc_id}"
   azs               = "${local.az_list_uswe2}"

   #--------------------------------------------------------------------------------
   # Pass the cidr_block attribute of aws_subnet resource created in module.networking_uswe2
   #------------------------------------------------------------------------------   
   #private_subnets   = "${var.private_subnets_uswe2}"
   private_subnets   = module.networking_uswe2.private_subnet_cidr_blocks  # <---- Here
   #------------------------------------------------------------------------------   
}

【讨论】：