嵌套数组中的 JOLT 连接值 (Apache NiFi)

Question

我有一个 JSON：

{
  "reports": [
    {
      "columnHeader": {
        "metricHeader": {
          "metricHeaderEntries": [
            {
              "name": "ga:sessions",
              "type": "INTEGER"
            },
            {
              "name": "ga:bounces",
              "type": "INTEGER"
            },
            {
              "name": "ga:sessionDuration",
              "type": "TIME"
            },
            {
              "name": "ga:pageviews",
              "type": "INTEGER"
            }
          ]
        }
      },
      "data": {
        "rows": [
          {
            "metrics": [
              {
                "values": [
                  "25",
                  "18",
                  "1269.0",
                  "27"
                ]
              }
            ]
          }
        ],
        "totals": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "rowCount": 1,
        "minimums": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "maximums": [
          {
            "values": [
              "25",
              "18",
              "1269.0",
              "27"
            ]
          }
        ],
        "isDataGolden": true
      }
    }
  ]
}

metricHeaderEntries 和 values 是分开的。值在data.totals 数组中（顺序保存正确）。 我想修改 JSON 并获得以下结构（或类似于此，我只需要对 metric.name = metric.value）：

{
  "metrics": [
            {
              "name": "ga:sessions",
              "value": "25"
            },
            {
              "name": "ga:bounces",
              "type": "18"
            },
            {
              "name": "ga:sessionDuration",
              "type": "1269.0"
            },
            {
              "name": "ga:pageviews",
              "type": "27"
            }
          ],
    "isDataGolden": true      
}

JOLT 可以吗？ 之前我只使用shift 规范来完成一些非常简单的任务。以下规格：

[
  {
    "operation": "shift",
    "spec": {
      "reports": {
        "*": {
          "columnHeader": {
            "metricHeader": {
              "metricHeaderEntries": {
                "*": {
                  "name": "@(1,name)"
                }
              }
            }
          },
          "isDataGolden": "isDataGolden"
        }
      }
    }
  }
]

返回：

{
  "ga:sessions" : "ga:sessions",
  "ga:bounces" : "ga:bounces",
  "ga:sessionDuration" : "ga:sessionDuration",
  "ga:pageviews" : "ga:pageviews"
}

“几乎”。当然不是我想要的。我需要一个数组metrics，其中包含name 和value 字段，如上所述。但我不知道如何从data.totals 获取这些值并将它们放入指标中。 isDataGolden 也消失了。我读了一些关于modify-overwrite-beta 的文章，我可以将它用于我的案例吗？

Answer 1

你可以使用executegroovyscript

import groovy.json.*

def ff=session.get()
if(!ff)return

//read flow file content and parse it
def body = ff.read().withReader("UTF-8"){reader-> 
    new JsonSlurper().parse(reader) 
}

def rep0=body.reports[0]

def result = [ 
    metrics : rep0.columnHeader.metricHeader.metricHeaderEntries.indexed().collect{i,m->
            [ 
                name : m.name,
                value: rep0.data.totals[0].values[i]
            ]
        }, 
    isDataGolden : rep0.data.isDataGolden 
]

//write new flow file content
ff.write("UTF-8"){writer-> 
    new JsonBuilder(result).writeTo(writer) 
}
//transfer
REL_SUCCESS << ff