【问题标题】:jq - extract nested JSON into multiple arraysjq - 将嵌套的 JSON 提取到多个数组中
【发布时间】:2016-08-05 10:27:30
【问题描述】:

我在这篇文章的底部有一些(编辑过的)JSON,这是在 AWS 中运行 ec2-describe-instances 的结果。示例 JSON 有 2 个数组,但是这可能更大。我以前使用过 jq,但我无法提取深度嵌套在 JSON 中的值。我特别感兴趣的是:

  • InstanceId ["Reservations": ["Instances": ["InstanceId":
  • DeviceName [“Reservations”:[“Instances”:[“BlockDeviceMappings”:[“DeviceName”:
  • VolumeId [“Reservations”:[“Instances”:[“BlockDeviceMappings”:[“Ebs”:{“VolumeId”:
  • TAG1 值/键
  • TAG2 值/键

总之,我想要一个可以迭代的 JSON 数组,它将给我一个 InstanceID、该实例上的标签和一个 single 卷。您会注意到每个实例都附加了 3 个卷(可能更多,也可能更少)。我希望能够单独迭代每一个。我希望的最终结果是这样的:

[
{
"InstanceId": "i-11111111",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-1111111a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdf",
"VolumeId": "vol-1111111b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-11111111",
"DeviceName": "xvdg",
"VolumeId": "vol-1111111c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sda1",
"VolumeId": "vol-2222222a",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdb",
"VolumeId": "vol-2222222b",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
},
{
"InstanceId": "i-22222222",
"DeviceName": "/dev/sdc",
"VolumeId": "vol-2222222c",
"TAG1": "VALUE1",
"TAG2": "VALUE2"
}
]

这类似于我发布的另一个问题here on stackoverflow。我尝试了很多不同的组合,但无法让它发挥作用。非常感谢任何帮助。

这是 JSON:

{
  "Reservations": [
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-11111111",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdf",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-1111111b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "xvdg",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-11111111c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    },
    {
      "OwnerId": "xx",
      "ReservationId": "xx",
      "Groups": [],
      "Instances": [
        {
          "Monitoring": {
            "State": "xx"
          },
          "PublicDnsName": "",
          "Platform": "xx",
          "State": {
            "Code": xx,
            "Name": "xx"
          },
          "EbsOptimized": xx,
          "LaunchTime": "xx",
          "PrivateIpAddress": "xx",
          "ProductCodes": [],
          "VpcId": "xx",
          "StateTransitionReason": "",
          "InstanceId": "i-22222222",
          "ImageId": "xx",
          "PrivateDnsName": "xx",
          "KeyName": "xx",
          "SecurityGroups": [
            {
              "GroupName": "xx",
              "GroupId": "xx"
            },
            {
              "GroupName": "xx",
              "GroupId": "xx"
            }
          ],
          "ClientToken": "xx",
          "SubnetId": "xx",
          "InstanceType": "xx",
          "NetworkInterfaces": [
            {
              "Status": "xx",
              "MacAddress": "xx",
              "SourceDestCheck": xx,
              "VpcId": "xx",
              "Description": "xx",
              "NetworkInterfaceId": "xx",
              "PrivateIpAddresses": [
                {
                  "PrivateDnsName": "xx",
                  "Primary": xx,
                  "PrivateIpAddress": "xx"
                }
              ],
              "PrivateDnsName": "xx",
              "Attachment": {
                "Status": "xx",
                "DeviceIndex": xx,
                "DeleteOnTermination": xx,
                "AttachmentId": "xx",
                "AttachTime": "xx"
              },
              "Groups": [
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                },
                {
                  "GroupName": "xx",
                  "GroupId": "xx"
                }
              ],
              "SubnetId": "xx",
              "OwnerId": "xx",
              "PrivateIpAddress": "xx"
            }
          ],
          "SourceDestCheck": xx,
          "Placement": {
            "Tenancy": "xx",
            "GroupName": "xx",
            "AvailabilityZone": "xx"
          },
          "Hypervisor": "xx",
          "BlockDeviceMappings": [
            {
              "DeviceName": "/dev/sda1",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222a",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdb",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222b",
                "AttachTime": "xx"
              }
            },
            {
              "DeviceName": "/dev/sdc",
              "Ebs": {
                "Status": "attached",
                "DeleteOnTermination": xx,
                "VolumeId": "vol-2222222c",
                "AttachTime": "xx"
              }
            }
          ],
          "Architecture": "xx",
          "RootDeviceType": "xx",
          "RootDeviceName": "xx",
          "VirtualizationType": "hvm",
          "Tags": [
            {
              "Value": "TAG1",
              "Key": "VALUE1"
            },
            {
              "Value": "TAG2",
              "Key": "VALUE2"
            },
            {
              "Value": "TAG3",
              "Key": "VALUE3"
            },
            {
              "Value": "TAG4",
              "Key": "VALUE4"
            },
            {
              "Value": "TAG5",
              "Key": "VALUE5"
            },
            {
              "Value": "TAG6",
              "Key": "Value6"
            }
          ],
          "AmiLaunchIndex": xx
        }
      ]
    }
]

编辑:我尝试过的一个例子(这个不是为了简单起见而试图获取标签:

jq -r '.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })' <(echo "$json_array_windows")

jq: error: Add/0 is not defined at <top-level>, line 1:
.Reservations.Instances | map(({ InstanceId } + (.BlockDeviceMappings | add) + (.BlockDeviceMappings.Ebs | Add))| { InstanceId, DeviceName, VolumeId })                                      
jq: 1 compile error

【问题讨论】:

    标签: json jq


    【解决方案1】:

    以下 jq 过滤器根据请求生成 JSON 对象流。 如果您确实需要 JSON 数组,只需将整个表达式括在方括号中即可。

    .Reservations[] | .Instances[]
    | { InstanceId }
      + (.BlockDeviceMappings[]
         | { DeviceName,
             "VolumeId": .Ebs.VolumeId } )
      + (.Tags
         | { "TAG1": ( map( select(.Value == "TAG1"))[] | .Key),
             "TAG2": ( map( select(.Value == "TAG2"))[] | .Key) } )
    

    这是输出(一旦输入 JSON 被更正):

    {
      "InstanceId": "i-11111111",
      "DeviceName": "/dev/sda1",
      "VolumeId": "vol-1111111a",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    {
      "InstanceId": "i-11111111",
      "DeviceName": "xvdf",
      "VolumeId": "vol-1111111b",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    {
      "InstanceId": "i-11111111",
      "DeviceName": "xvdg",
      "VolumeId": "vol-11111111c",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    {
      "InstanceId": "i-22222222",
      "DeviceName": "/dev/sda1",
      "VolumeId": "vol-2222222a",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    {
      "InstanceId": "i-22222222",
      "DeviceName": "/dev/sdb",
      "VolumeId": "vol-2222222b",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    {
      "InstanceId": "i-22222222",
      "DeviceName": "/dev/sdc",
      "VolumeId": "vol-2222222c",
      "TAG1": "VALUE1",
      "TAG2": "VALUE2"
    }
    

    【讨论】:

    • 非常感谢!我注意到所有设备名称都是“/dev/sda1”,这似乎是错误的。
    • 您说“我希望能够单独迭代每个”。正如我所指出的,我只显示了前三个 JSON 对象。
    • 很抱歉没有正确措辞。我确实需要将“DeviceName”正确映射到“VolumeId”。你觉得这可能吗?
    • 我想你会发现修改后的答案符合你的期望。
    • 有什么方法可以获取未找到的 TAG 的值,只是说“null”,而不是根本不显示任何内容?很高兴打开另一个线程
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多