使用 jq 解析 json 树中的多个键/值

Question

使用 jq，我想从以下 json 中挑选键/值对：

{
  "project": "Project X",
  "description": "This is a description of Project X",
  "nodes": [
    {
      "name": "server001",
      "detail001": "foo",
      "detail002": "bar",
      "networks": [
        {
          "net_tier": "network_tier_001",
          "ip_address": "10.1.1.10",
          "gateway": "10.1.1.1",
          "subnet_mask": "255.255.255.0",
          "mac_address": "00:11:22:aa:bb:cc"
        }
      ],
      "hardware": {
        "vcpu": 1,
        "mem": 1024,
        "disks": [
          {
            "disk001": 40,
            "detail001": "foo"
          },
          {
            "disk002": 20,
            "detail001": "bar"
          }
        ]
      },
      "os": "debian8",
      "geo": {
        "region": "001",
        "country": "Sweden",
        "datacentre": "Malmo"
      },
      "detail003": "baz"
    }
  ],
  "detail001": "foo"
}

为了举例，我想解析以下键及其值：“Project”、“name”、“net_tier”、“vcpu”、“mem”、“disk001”、“disk002” .

我能够毫无问题地解析单个元素，但由于完整解析的分层性质，我在解析不同的分支（即网络和硬件 > 磁盘）时运气不佳。

任何帮助表示赞赏。

编辑：

为清楚起见，我要输出的是逗号分隔的 CSV。在解析所有组合方面，现在覆盖示例中的示例数据即可。我希望能够扩展任何建议。

Answer 1

这是一个不同的过滤器，它计算唯一的网络层和磁盘名称集，然后生成一个包含适合数据的列的结果。

  {
    tiers: [ .nodes[].networks[].net_tier ] | unique
  , disks: [ .nodes[].hardware.disks[] | keys[] | select(startswith("disk")) ] | unique
  } as $n

| def column_names($n): [ "project", "name" ] + $n.tiers + ["vcpu", "mem"] + $n.disks ;
  def tiers($n):        [ $n.tiers[] as $t | .networks[] | if .net_tier==$t then $t else null end ] ;
  def disks($n):        [ $n.disks[] as $d | map(select(.[$d]!=null)|.[$d])[0] ] ;
  def rows($n):
      .project as $project
    | .nodes[]
    | .name as $name
    | tiers($n) as $tier_values
    | .hardware
    | .vcpu as $vcpu
    | .mem as $mem
    | .disks
    | disks($n) as $disk_values
    | [$project, $name] + $tier_values + [$vcpu, $mem] + $disk_values
  ;
  column_names($n), rows($n)

| @csv

如果我们在样本数据中添加另一个节点，这种方法的好处就显而易见了：

{
  "name": "server002",
  "networks": [
    {
      "net_tier": "network_tier_002"
    }
  ],
  "hardware": {
    "vcpu": 1,
    "mem": 1024,
    "disks": [
      {
        "disk002": 40,
        "detail001": "foo"
      }
    ]
  }
}

样本运行（假设filter.jq 中的过滤器和data.json 中的修正数据）

$ jq -Mr -f filter.jq data.json
"project","name","network_tier_001","network_tier_002","vcpu","mem","disk001","disk002"
"Project X","server001","network_tier_001","",1,1024,40,20
"Project X","server002",,"network_tier_002",1,1024,,40

Try it online!

Answer 2

这是实现所需输出的一种方法。

program.jq：

["project","name","net_tier","vcpu","mem","disk001","disk002"],
  [.project]
+ (.nodes[] | .networks[] as $n |
    [
      .name,
      $n.net_tier,
      (.hardware |
        .vcpu,
        .mem,
        (.disks | add["disk001","disk002"])
      )
    ]
  )
| @csv

$ jq -r -f program.jq input.json
"project","name","net_tier","vcpu","mem","disk001","disk002"
"Project X","server001","network_tier_001",1,1024,40,20

基本上，您需要将所需的字段投影到数组中，以便将这些数组转换为 csv 行。您的输入使给定节点看起来可能有多个网络。因此，如果您想输出所有组合，则必须将其展平。

Answer 3

这是另一种方法，它足够简短，足以说明问题：

def s(f): first(.. | f? // empty) // null;

[s(.project), s(.name), s(.net_tier), s(.vcpu), s(.mem), s(.disk001), s(.disk002)]
| @csv

调用：

$ jq -r -f value-pairs.jq input.json

结果：

"Project X","server001","network_tier_001",1,1024,40,20

带标题

使用与上面相同的s/1：

. as $d
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"]
| (., map( . as $v | $d | s(.[$v])))
| @csv

有多个节点

再次使用s/1，如上：

.project as $p
| ["project", "name", "net_tier", "vcpu", "mem", "disk001","disk002"] as $h
| ($h,
   (.nodes[] as $d
   | $h
   | map( . as $v | $d | s(.[$v]) )
   | .[0] = $p)
   ) | @csv

带有说明性多节点数据的输出：

"project","name","net_tier","vcpu","mem","disk001","disk002"
"Project X","server001","network_tier_001",1,1024,40,20
"Project X","server002","network_tier_002",1,1024,,40