如何使用 json_normalize 展平嵌套 Json 数据

Question

我正在尝试通过 json_normalize 将 JSON 数据导入 Dataframe，但无法正常工作。

我的数据：

a键与c1键

相同

[
  {
    "a": "A1",
    "b": "B1",
    "c": [
      {
        "c1": "C111",
        "c2": "C121",
        "c3": ["C1131","C1132"]
      }
    ]
  },
  {
    "a": "A2",
    "b": "B2",
    "c": [
      {
        "c1": "C211",
        "c2": "C212",
        "c3": ["C2131","C2132"]
      },
      {
        "c1": "C221",
        "c2": "C222",
        "c3": ["C2231"]
      }
    ]
  }
]

我想做一个类似的DataFrame

           a     c1(a)      c2                    c3
0         A1      C111    C121     ["C1131","C1132"]
1         A2      C211    C212     ["C2131","C2132"]
2         A2      C221    C222             ["C2231"]

当我使用 json_normalize 时，它​​显示 ValueError:

entity_df = json_normalize(data, 'c', 'a')

ValueError: Conflicting metadata name a, need distinguishing prefix 

我应该如何更改 json_normalize 参数？ 任何帮助将不胜感激。

Answer 1

如果您已经经历过隐藏真实数据的痛苦，那么让模拟数据也具有与真实数据相同的特征。

假设你有这个 JSON：

json_data =\
[{'a': 'A1',
  'b': 'B1',
  'c': [{'a': 'C111', 'c2': 'C121', 'c3': ['C1131', 'C1132']}]},
 {'a': 'A2',
  'b': 'B2',
  'c': [{'a': 'C211', 'c2': 'C212', 'c3': ['C2131', 'C2132']},
   {'a': 'C221', 'c2': 'C222', 'c3': ['C2231']}]}]

只需要一行代码即可提取：

pd.json_normalize(json_data, 'c', ['a', 'b'], record_prefix='data.')

结果：

  data.a data.c2         data.c3   a   b
0   C111    C121  [C1131, C1132]  A1  B1
1   C211    C212  [C2131, C2132]  A2  B2
2   C221    C222         [C2231]  A2  B2

record_prefix='data.' 是 ValueError: Conflicting metadata name a, need distinguishing prefix 错误消息的意思。

Answer 2

你可以试试：

from collections import defaultdict

norm_data = defaultdict(list)
for item in data:
    for element in item['c']:
        norm_data['a'].append(item['a'])
        for k, v in element.items():
            if k in {'a', 'c1'}:
                norm_data['c1(a)'].append(v)
            else:
                norm_data[k].append(v)

pd.DataFrame(norm_data)

Answer 3

data = [
  {
    "a": "A1",
    "b": "B1",
    "c": [
      {
        "c1": "C111",
        "c2": "C121",
        "c3": ["C1131","C1132"]
      }
    ]
  },
  {
    "a": "A2",
    "b": "B2",
    "c": [
      {
        "c1": "C211",
        "c2": "C212",
        "c3": ["C2131","C2132"]
      },
      {
        "c1": "C221",
        "c2": "C222",
        "c3": ["C2231"]
      }
    ]
  }
]
pd.io.json.json_normalize(data,"c", ['a', 'b',])

输出：

    c1      c2      c3              a   b
0   C111    C121    [C1131, C1132]  A1  B1
1   C211    C212    [C2131, C2132]  A2  B2
2   C221    C222    [C2231]         A2  B2