展平双嵌套 JSON

Question

我正在尝试展平如下所示的 JSON 文件：

{
"teams": [
  {
    "teamname": "1",
    "members": [
      {
        "firstname": "John", 
        "lastname": "Doe",
        "orgname": "Anon",
        "phone": "916-555-1234",
        "mobile": "",
        "email": "john.doe@wildlife.net"
      },
      {
        "firstname": "Jane",
        "lastname": "Doe",
        "orgname": "Anon",
        "phone": "916-555-4321",
        "mobile": "916-555-7890",
        "email": "jane.doe@wildlife.net"
      }
    ]
  },
  {
    "teamname": "2",
    "members": [
      {
        "firstname": "Mickey",
        "lastname": "Moose",
        "orgname": "Moosers",
        "phone": "916-555-0000",
        "mobile": "916-555-1111",
        "email": "mickey.moose@wildlife.net"
      },
      {
        "firstname": "Minny",
        "lastname": "Moose",
        "orgname": "Moosers",
        "phone": "916-555-2222",
        "mobile": "",
        "email": "minny.moose@wildlife.net"
      }
    ]
  }       
]

}

我希望将其导出到 Excel 表。 我目前的代码是这样的：

from pandas.io.json import json_normalize
import json
import pandas as pd

inputFile = 'E:\\teams.json'
outputFile = 'E:\\teams.xlsx'

f = open(inputFile)
data = json.load(f)
f.close()

df = pd.DataFrame(data)

result1 = json_normalize(data, 'teams' )
print result1

结果如下：

members                                              teamname
0  [{u'firstname': u'John', u'phone': u'916-555-...        1
1  [{u'firstname': u'Mickey', u'phone': u'916-555-...      2

每行嵌套了 2 个成员的数据。我想要一个输出表，显示所有 4 名成员的数据以及他们关联的团队名称。

Answer 1

使用pandas.io.json.json_normalize

json_normalize(data,record_path=['teams','members'],meta=[['teams','teamname']])

output:
         email                firstname lastname mobile      orgname    phone       teams.teamname
0   john.doe@wildlife.net       John    Doe                   Anon      916-555-1234    1
1   jane.doe@wildlife.net       Jane    Doe     916-555-7890  Anon      916-555-4321    1
2   mickey.moose@wildlife.net   Mickey  Moose   916-555-1111  Moosers   916-555-0000    2
3   minny.moose@wildlife.net    Minny   Moose                 Moosers   916-555-2222    2

说明

from pandas.io.json import json_normalize
import pandas as pd

我最近才学会如何使用 json_normalize 函数，所以我的解释可能不对。

从我所说的“第 0 层”开始

json_normalize(data)

output:
     teams
0   [{'teamname': '1', 'members': [{'firstname': '...

有 1 列和 1 行。一切都在“团队”列中。

使用 record_path=查看我所说的“第 1 层”

json_normalize(data,record_path='teams')

output:
     members                                          teamname
0   [{'firstname': 'John', 'lastname': 'Doe', 'org...    1
1   [{'firstname': 'Mickey', 'lastname': 'Moose', ...    2

在第 1 层中，我们已经扁平化了“团队名称”，但内部“成员”更多。

使用 record_path= 查看第 2 层。这个符号起初是不直观的。我现在通过 ['layer','deeperlayer'] 记住它，结果是 layer.deeperlayer。

json_normalize(data,record_path=['teams','members'])

output:
           email              firstname lastname   mobile     orgname   phone
0   john.doe@wildlife.net      John        Doe                  Anon    916-555-1234
1   jane.doe@wildlife.net       Jane        Doe   916-555-7890  Anon    916-555-4321
2   mickey.moose@wildlife.net   Mickey     Moose   916-555-1111 Moosers 916-555-0000
3   minny.moose@wildlife.net    Minny       Moose               Moosers 916-555-2222

请原谅我的输出，我不知道如何在响应中制作表格。

最后我们使用 meta= 添加第 1 层列

json_normalize(data,record_path=['teams','members'],meta=[['teams','teamname']])

output:
         email                firstname lastname mobile      orgname    phone       teams.teamname
0   john.doe@wildlife.net       John    Doe                   Anon      916-555-1234    1
1   jane.doe@wildlife.net       Jane    Doe     916-555-7890  Anon      916-555-4321    1
2   mickey.moose@wildlife.net   Mickey  Moose   916-555-1111  Moosers   916-555-0000    2
3   minny.moose@wildlife.net    Minny   Moose                 Moosers   916-555-2222    2

请注意，我们需要一个 meta=[[]] 的列表列表来引用第 1 层。 如果我们想要从第 0 层和第 1 层得到一列，我们可以这样做：

json_normalize(data,record_path=['layer1','layer2'],meta=['layer0',['layer0','layer1']])

json_normalize 的结果是一个 pandas 数据帧。

Answer 2

这是一种方法。应该给你一些想法。

df = pd.concat(
    [
        pd.concat([pd.Series(m) for m in t['members']], axis=1) for t in data['teams']
    ], keys=[t['teamname'] for t in data['teams']]
)

                                     0                         1
1 email          john.doe@wildlife.net     jane.doe@wildlife.net
  firstname                       John                      Jane
  lastname                         Doe                       Doe
  mobile                                            916-555-7890
  orgname                         Anon                      Anon
  phone                   916-555-1234              916-555-4321
2 email      mickey.moose@wildlife.net  minny.moose@wildlife.net
  firstname                     Mickey                     Minny
  lastname                       Moose                     Moose
  mobile                  916-555-1111                          
  orgname                      Moosers                   Moosers
  phone                   916-555-0000              916-555-2222

要获得一个漂亮的表格，其中团队名称和成员作为行，所有属性都在列中：

df.index.levels[0].name = 'teamname'
df.columns.name = 'member'

df.T.stack(0).swaplevel(0, 1).sort_index()

要将团队名称和成员作为实际列，只需重置索引即可。

df.index.levels[0].name = 'teamname'
df.columns.name = 'member'

df.T.stack(0).swaplevel(0, 1).sort_index().reset_index()

整件事

import json
import pandas as pd

json_text = """{
"teams": [
  {
    "teamname": "1",
    "members": [
      {
        "firstname": "John", 
        "lastname": "Doe",
        "orgname": "Anon",
        "phone": "916-555-1234",
        "mobile": "",
        "email": "john.doe@wildlife.net"
      },
      {
        "firstname": "Jane",
        "lastname": "Doe",
        "orgname": "Anon",
        "phone": "916-555-4321",
        "mobile": "916-555-7890",
        "email": "jane.doe@wildlife.net"
      }
    ]
  },
  {
    "teamname": "2",
    "members": [
      {
        "firstname": "Mickey",
        "lastname": "Moose",
        "orgname": "Moosers",
        "phone": "916-555-0000",
        "mobile": "916-555-1111",
        "email": "mickey.moose@wildlife.net"
      },
      {
        "firstname": "Minny",
        "lastname": "Moose",
        "orgname": "Moosers",
        "phone": "916-555-2222",
        "mobile": "",
        "email": "minny.moose@wildlife.net"
      }
    ]
  }       
]
}"""


data = json.loads(json_text)

df = pd.concat(
    [
        pd.concat([pd.Series(m) for m in t['members']], axis=1) for t in data['teams']
    ], keys=[t['teamname'] for t in data['teams']]
)

df.index.levels[0].name = 'teamname'
df.columns.name = 'member'

df.T.stack(0).swaplevel(0, 1).sort_index().reset_index()