【发布时间】：2020-05-05 12:10:59
【问题描述】：
我正在按照教程Implementing Terraform on Microsoft Azure 进行操作。到目前为止一切顺利，我已经达到了使用 terraform 工作区从 Azure DevOps CI/CD 管道创建环境的模块。模块名称为“使用 Azure DevOps”
Azure DevOps 项目是公开的 - https://dev.azure.com/MarkKharitonov0271/_git/Globomantics-testing
运行workspaces 发布管道对我来说始终失败：
2020-01-19T03:23:20.5420275Z Error: authorization.RoleDefinitionsClient#CreateOrUpdate: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client '0e648d2d-a49f-407e-99de-9d6343876a8c' with object id '0e648d2d-a49f-407e-99de-9d6343876a8c' does not have authorization to perform action 'Microsoft.Authorization/roleDefinitions/write' over scope '/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3' or the scope is invalid. If access was recently granted, please refresh your credentials."
2020-01-19T03:23:20.5420512Z
2020-01-19T03:23:20.5420989Z on vnet-peering.tf line 52, in resource "azurerm_role_definition" "vnet-peering":
2020-01-19T03:23:20.5421339Z 52: resource "azurerm_role_definition" "vnet-peering" {
（您可以在项目 repo 中查看 vnet-peering.tf - https://dev.azure.com/MarkKharitonov0271/_git/Globomantics-testing?path=%2Fnetworking%2Fvnet-peering.tf）
无论如何，如果我正确理解错误消息，它声称服务主体0e648d2d-a49f-407e-99de-9d6343876a8c 无权在订阅2b38509c-a310-4c8f-bd78-9e400cc874e3 中创建新角色定义
所以，我检查了服务主体：
PS /home/mark> az ad sp show --id '0e648d2d-a49f-407e-99de-9d6343876a8c'
{
"accountEnabled": "True",
"addIns": [],
"alternativeNames": [],
"appDisplayName": "MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3",
"appId": "0ae4ffc7-149d-45ac-ab15-c9f61e4591f8",
"appOwnerTenantId": "717e5a4d-529c-4ab2-a1c5-6a5f6345d8e4",
"appRoleAssignmentRequired": false,
"appRoles": [],
"applicationTemplateId": null,
"deletionTimestamp": null,
"displayName": "MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3",
"errorUrl": null,
"homepage": "https://VisualStudio/SPN",
"informationalUrls": {
"marketing": null,
"privacy": null,
"support": null,
"termsOfService": null
},
"keyCredentials": [],
"logoutUrl": null,
"notificationEmailAddresses": [],
"oauth2Permissions": [
{
"adminConsentDescription": "Allow the application to access MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3 on behalf of the signed-in user.",
"adminConsentDisplayName": "Access MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3",
"id": "d0f141b9-fc6b-4f3c-9217-018d74712ee1",
"isEnabled": true,
"userConsentDescription": "Allow the application to access MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3 on your behalf.",
"userConsentDisplayName": "Access MarkKharitonov0271-Globomantics-testing-2b38509c-a310-4c8f-bd78-9e400cc874e3",
}
],
"objectId": "0e648d2d-a49f-407e-99de-9d6343876a8c",
"objectType": "ServicePrincipal",
"odata.metadata": "https://graph.windows.net/717e5a4d-529c-4ab2-a1c5-6a5f6345d8e4/$metadata#directoryObjects/@Element",
"odata.type": "Microsoft.DirectoryServices.ServicePrincipal",
"passwordCredentials": [],
"preferredSingleSignOnMode": null,
"preferredTokenSigningKeyEndDateTime": null,
"preferredTokenSigningKeyThumbprint": null,
"publisherName": "Default Directory",
"replyUrls": [
"https://VisualStudio/SPN"
],
"samlMetadataUrl": null,
"samlSingleSignOnSettings": null,
"servicePrincipalNames": [
"https://VisualStudio/SPN136d4f76-7262-4ab0-8fbb-7be74dfc803b",
"0ae4ffc7-149d-45ac-ab15-c9f61e4591f8"
],
"servicePrincipalType": "Application",
"signInAudience": "AzureADMyOrg",
"tags": [],
"tokenEncryptionKeyId": null
}
请注意，服务主体的 appId 等于 0ae4ffc7-149d-45ac-ab15-c9f61e4591f8。
当我将 Terraform 任务添加到管道时，它似乎对应于 Azure DevOps 创建的一个，当它想要授权访问订阅时。确实：
现在，terraform 应用步骤引用了相同的服务主体：
最后，这个 Service Principal 似乎拥有订阅的 Contributor 访问权限：
PS /home/mark> az role assignment list --assignee '0e648d2d-a49f-407e-99de-9d6343876a8c'
[
{
"canDelegate": null,
"id": "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/providers/Microsoft.Authorization/roleAssignments/346f1b92-0621-44c0-b88a-343c52637a0f",
"name": "346f1b92-0621-44c0-b88a-343c52637a0f",
"principalId": "0e648d2d-a49f-407e-99de-9d6343876a8c",
"principalName": "https://VisualStudio/SPN136d4f76-7262-4ab0-8fbb-7be74dfc803b",
"principalType": "ServicePrincipal",
"roleDefinitionId": "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c",
"roleDefinitionName": "Contributor",
"scope": "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3",
"type": "Microsoft.Authorization/roleAssignments"
}
]
PS /home/mark>
所以，我不明白发生了什么。
编辑 1
因此，我能够使用 Azure Cloud Shell 中完全相同的服务主体运行完全相同的 terraform 配置。
首先我重置了凭据，注销并以该服务主体的身份重新登录：
PS /home/mark/terraform> az ad sp credential reset --name https://VisualStudio/SPN136d4f76-7262-4ab0-8fbb-7be74dfc803b
{
"appId": "0ae4ffc7-149d-45ac-ab15-c9f61e4591f8",
"name": "https://VisualStudio/SPN136d4f76-7262-4ab0-8fbb-7be74dfc803b",
"password": "e...3",
"tenant": "717e5a4d-529c-4ab2-a1c5-6a5f6345d8e4"
}
PS /home/mark/terraform> az account clear
Logout successful. Re-login to your initial Cloud Shell identity with 'az login --identity'. Login with a new identity with 'az login'.
PS /home/mark/terraform> az login --service-principal -u https://VisualStudio/SPN136d4f76-7262-4ab0-8fbb-7be74dfc803b -p e...3 --tenant 717e5a4d-529c-4ab2-a1c5-6a5f6345d8e4
Cloud Shell is automatically authenticated under the initial account signed-in with. Run 'az login' only if you need to use a different account
{
"cloudName": "AzureCloud",
"id": "2b38509c-a310-4c8f-bd78-9e400cc874e3",
"isDefault": true,
"name": "Visual Studio Enterprise",
"state": "Enabled",
"type": "servicePrincipal"
}
}
]
然后我克隆了 Git 存储库以获取 Terraform 代码，并复制了从先前模块中保存的后端配置：
PS /home/mark/terraform> git clone https://MarkKharitonov0271@dev.azure.com/MarkKharitonov0271/Globomantics-testing/_git/Globomantics-testing
Cloning into 'Globomantics-testing'...
remote: Azure Repos
remote: We noticed you're using an older version of Git. For the best experience, upgrade to a newer version.
remote: Found 9 objects to send. (2 ms)
Unpacking objects: 100% (9/9), done.
Checking connectivity... done.
PS /home/mark/terraform> cd ./Globomantics-testing/networking/
PS /home/mark/terraform/Globomantics-testing/networking> dir
Directory: /home/mark/terraform/Globomantics-testing/networking
Mode LastWriteTime Length Name
---- ------------- ------ ----
------ 1/19/20 4:41 AM 40 backend.tf
------ 1/19/20 4:41 AM 2256 main.tf
------ 1/19/20 4:41 AM 436 terraform.tfvars
------ 1/19/20 4:41 AM 2423 vnet-peering.tf
------ 1/19/20 4:41 AM 279 workspacetest.sh
PS /home/mark/terraform/Globomantics-testing/networking> copy ../../1-main-vnet/backend-config.txt .
现在我准备初始化 Terraform 并设置开发工作区：
PS /home/mark/terraform/Globomantics-testing/networking> terraform init -backend-config='backend-config.txt'
Initializing modules...
Downloading Azure/vnet/azurerm 1.2.0 for vnet-main...
- vnet-main in .terraform/modules/vnet-main
Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "azurerm" (hashicorp/azurerm) 1.41.0...
- Downloading plugin for provider "template" (hashicorp/template) 2.1.2...
...
PS /home/mark/terraform/Globomantics-testing/networking> terraform workspace select development
Switched to workspace "development".
现在的地形计划：
PS /home/mark/terraform/Globomantics-testing/networking> terraform plan -var sec_client_secret='F...$' -out main.tfplan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
data.template_file.subnet_prefixes[0]: Refreshing state...
data.template_file.subnet_prefixes[1]: Refreshing state...
data.template_file.subnet_prefixes[2]: Refreshing state...
azurerm_resource_group.main: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet]
data.azurerm_subscription.current: Refreshing state...
module.vnet-main.azurerm_resource_group.vnet: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet]
module.vnet-main.azurerm_virtual_network.vnet: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet]
module.vnet-main.azurerm_subnet.subnet[0]: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet/subnets/web]
module.vnet-main.azurerm_subnet.subnet[2]: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet/subnets/app]
module.vnet-main.azurerm_subnet.subnet[1]: Refreshing state... [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet/subnets/database]
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# azurerm_role_assignment.vnet will be created
+ resource "azurerm_role_assignment" "vnet" {
+ id = (known after apply)
+ name = (known after apply)
+ principal_id = "63309b8d-908a-4dcf-b95b-25eae33aaceb"
+ principal_type = (known after apply)
+ role_definition_id = (known after apply)
+ role_definition_name = (known after apply)
+ scope = "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet"
+ skip_service_principal_aad_check = (known after apply)
}
# azurerm_role_definition.vnet-peering will be created
+ resource "azurerm_role_definition" "vnet-peering" {
+ assignable_scopes = [
+ "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3",
]
+ id = (known after apply)
+ name = "allow-vnet-peer-action-development"
+ role_definition_id = (known after apply)
+ scope = "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3"
+ permissions {
+ actions = [
+ "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write",
+ "Microsoft.Network/virtualNetworks/peer/action",
+ "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/read",
+ "Microsoft.Network/virtualNetworks/virtualNetworkPeerings/delete",
]
+ not_actions = []
}
}
# azurerm_virtual_network_peering.main will be created
+ resource "azurerm_virtual_network_peering" "main" {
+ allow_forwarded_traffic = (known after apply)
+ allow_gateway_transit = (known after apply)
+ allow_virtual_network_access = (known after apply)
+ id = (known after apply)
+ name = "development_2_sec"
+ remote_virtual_network_id = "/subscriptions/2b1285d1-f4a7-4cc3-a7b1-0d0fc31d6192/resourceGroups/security/providers/Microsoft.Network/virtualNetworks/security"
+ resource_group_name = "development-vnet"
+ use_remote_gateways = (known after apply)
+ virtual_network_name = "development-vnet"
}
# azurerm_virtual_network_peering.sec will be created
+ resource "azurerm_virtual_network_peering" "sec" {
+ allow_forwarded_traffic = (known after apply)
+ allow_gateway_transit = (known after apply)
+ allow_virtual_network_access = (known after apply)
+ id = (known after apply)
+ name = "sec_2_development"
+ remote_virtual_network_id = "/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet"
+ resource_group_name = "security"
+ use_remote_gateways = (known after apply)
+ virtual_network_name = "security"
}
Plan: 4 to add, 0 to change, 0 to destroy.
...
------------------------------------------------------------------------
This plan was saved to: main.tfplan
To perform exactly these actions, run the following command to apply:
terraform apply "main.tfplan"
最后 terraform 应用：
PS /home/mark/terraform/Globomantics-testing/networking> terraform apply main.tfplan
azurerm_role_definition.vnet-peering: Creating...
azurerm_role_definition.vnet-peering: Creation complete after 1s [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/providers/Microsoft.Authorization/roleDefinitions/b9614597-de13-c2c6-a275-9186847642ed]
azurerm_role_assignment.vnet: Creating...
azurerm_role_assignment.vnet: Creation complete after 1s [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet/providers/Microsoft.Authorization/roleAssignments/7f6a642d-d43e-05b5-cf34-3d20d02cd255]
azurerm_virtual_network_peering.main: Creating...
azurerm_virtual_network_peering.sec: Creating...
azurerm_virtual_network_peering.main: Still creating... [10s elapsed]
azurerm_virtual_network_peering.sec: Still creating... [10s elapsed]
azurerm_virtual_network_peering.main: Creation complete after 11s [id=/subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet/virtualNetworkPeerings/development_2_sec]
azurerm_virtual_network_peering.sec: Creation complete after 11s [id=/subscriptions/2b1285d1-f4a7-4cc3-a7b1-0d0fc31d6192/resourceGroups/security/providers/Microsoft.Network/virtualNetworks/security/virtualNetworkPeerings/sec_2_development]
...
Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
Outputs:
resource_group_name = development-vnet
vnet_id = /subscriptions/2b38509c-a310-4c8f-bd78-9e400cc874e3/resourceGroups/development-vnet/providers/Microsoft.Network/virtualNetworks/development-vnet
vnet_name = development-vnet
使用与 Azure DevOps 创建的完全相同的服务主体，一切正常。
接下来我将尝试删除 Azure DevOps 项目上的服务连接，手动创建一个新的并重试发布。
编辑 2
新的服务连接出现同样的错误。我不明白。我觉得“好奇”的是main.tf 中的所有资源都配置得很好，但vnet-peering.tf 中没有。
这是怎么回事？
编辑 3
将 Microsoft DevLabs 的 Terraform 任务替换为 Charles Zipp 的任务，因为 Pluralsight 课程作者使用了它们。结果一样。
【问题讨论】：
-
我已将最后一次编辑作为答案重新发布 - 请始终将答案作为答案发布。让我知道是否想以您自己的名义发布 - 如果是，我将删除我的 CW 副本。
标签： azure azure-devops terraform