Powershell 嵌套 JSON 到 csv 转换

Question

我有一个相当奇特的嵌套 JSON，在某些情况下，键值对正常出现，但在其他情况下，键的类型出现在进一步的嵌套中。

{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}

在示例中您可以看到metadata 的字段是简单的键值对，但在recordContent 下方，我们有positionDate，这是一个简单的键值，但"account":{"string":"G32"} 和"strike":{"double":4.4} 不是.

我想放弃类型信息并得到如下的 CSV 结构：

systemId, legalEntity, positionDate,         account,seg,strike
da1895,   A0,          2019-04-08 00:00:00.0,G32,    S,  4.4
4536d,    5G,          2019-04-08 00:00:00.0,G81,    S,  5.0

关于如何使用 Powershell 将这种结构转换为 CSV 的任何想法？

这是我尝试过的：

$TemplateParametersFile = "c:\data\output.json"
$JsonParameters = Get-Content $TemplateParametersFile | ConvertFrom-Json

$metadatafields = $JsonParameters.metadata[0].PSObject.Properties.Name
$recordcontentfields = $JsonParameters.recordContent[0].PsObject.Properties.Name

$oData = New-Object PSObject

$metadatafields |
ForEach {
    Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.metadata.($_)
}

$recordcontentfields |
ForEach {
    Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.recordContent.($_)
}

这给了我：

$oData

systemId     : {da1895, 45364d}
legalEntity  : {A0, 5G}
positionDate : {2019-04-08 00:00:00.0, 2019-04-08 00:00:00.0}
account      : {@{string=G32}, @{string=G81}}
seg          : {@{string=S}, @{string=S}}
strike       : {@{double=4.4}, @{double=5.0}}

我现在有点卡住了，上面没有转换为 csv。

请注意，除了元数据和记录内容之外，我没有硬编码任何字段名，并且我希望在 JSON 结构发生变化时保持这种灵活性。

谢谢

Answer 1

我建议在有序的hashtable ([ordered] @{}) 中迭代地收集属性-名称-值对，然后可以将其转换为[pscustomobject] 以将其转换为自定义对象。

没有属性 names 在以下解决方案中被硬编码，但 object-graph structure 假定遵循示例 JSON 中的模式，即仅限于一层嵌套 - 如果您需要处理任意嵌套的对象，this answer 可能是一个起点。

反射（属性名称和值的发现）是通过 PowerShell 为所有对象提供的intrinsic .psobject property 执行的。

# Parse sample JSON into an array of [pscustomobject] graphs.
$fromJson = ConvertFrom-Json @'
[
  {"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
  ,
  {"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
'@

# Initialize an aux. ordered hashtable to collect the property-name-value
# pairs in.
$oht = [ordered] @{}

$fromJson | ForEach-Object {
    $oht.Clear()
    # Loop over top-level properties.
    foreach ($topLevelProp in $_.psobject.Properties) {
      # Loop over second-level properties.
      foreach ($prop in $topLevelProp.Value.psobject.Properties) {
        if ($prop.Value -is [System.Management.Automation.PSCustomObject]) {
          # A nested value: Use the value of the (presumed to be one-and-only)
          #                 property of the object stored in the value.
          $oht[$prop.Name] = $prop.Value.psobject.Properties.Value
        }
        else {
          # A non-nested value: use as-is.
          $oht[$prop.Name] = $prop.Value
        }
      }
    }
    # Construct and output a [pscustomobject] from the aux. ordered hashtble.
    [pscustomobject] $oht
  } |
      ConvertTo-Csv  # Replace this with Export-Csv to export to a file.

以上产出：

"systemId","legalEntity","positionDate","account","seg","strike"
"da1895","A0","2019-04-08 00:00:00.0","G32","S","4.4"
"45364d","5G","2019-04-08 00:00:00.0","G81","S","5"

Answer 2

几年前，我为此编写了一个可重用的Flatten-Object 函数。
唯一的区别是它将（子）属性名称与父属性名称组合在一起，因为它们可能不是唯一的：

$JsonParameters |Flatten-Object |Format-Table

metadata.systemId metadata.legalEntity recordContent.positionDate recordContent.account.string recordContent.seg.string recordContent.strike.double
----------------- -------------------- -------------------------- ---------------------------- ------------------------ ---------------------------
da1895            A0                   2019-04-08 00:00:00.0      G32                          S                                                4.4
45364d            5G                   2019-04-08 00:00:00.0      G81                          S                                                  5

Answer 3

试试这个：

$data = ConvertFrom-Json @"
[
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}},
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
"@

$data | Select-Object -Property @{l="systemId"; e={$_.metadata.systemId}}, @{l="legalEntity"; e={$_.metadata.legalEntity}},
@{l="positionDate"; e={$_.recordContent.positionDate}}, @{l="account"; e={$_.recordContent.account.string}},
@{l="seg"; e={$_.recordContent.seg.string}}, @{l="strike"; e={$_.recordContent.strike.double}} | Export-Csv

Answer 4

这应该适用于任何嵌套的 psobject。

$json = @'
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}

'@
$obj = ConvertFrom-Json $json


$obj.recordContent | gm -MemberType NoteProperty | % {
    $prop = $_.name
    if ($obj.recordContent.$prop.GetType().name -eq 'pscustomobject') {
        $obj.recordContent.$prop = $obj.recordContent.$prop.psobject.Members | where membertype -eq noteproperty | select -ExpandProperty value
    }
    $obj.metadata | add-member -MemberType NoteProperty -Name $prop -Value $obj.recordContent.$prop

}
$newobj = $obj.metadata
$newobj