使用 Pandas 将嵌套的 CSV 转换为嵌套的 JSON

【问题标题】：Convert nested CSV to nested JSON using Pandas使用 Pandas 将嵌套的 CSV 转换为嵌套的 JSON
【发布时间】：2021-08-26 14:49:59
【问题描述】：

我有一个这样的数据框

org.iden.account,org.iden.id,adress.city,adress.country,person.name.fullname,person.gender,person.birthYear,subs.id,subs.subs1.birthday,subs.subs1.org.address.country,subs.subs1.org.address.strret1,subs.org.buyer.email.address,subs.org.buyer.phone.number
account123,id123,riga,latvia,laura,female,1990,subs123,1990-12-14T00:00:00Z,latvia,street 1,email1@myorg.com|email2@sanoma.com,+371401234567
account123,id000,riga,latvia,laura,female,1990,subs456,1990-12-14T00:00:00Z,latvia,street 1,email1@myorg.com,+371401234567
account123,id456,riga,latvia,laura,female,1990,subs789,1990-12-14T00:00:00Z,latvia,street 1,email1@myorg.com,+371401234567

并且我需要将其转换为基于由点 (.) 分隔的列的嵌套 JSON。所以对于第一行，预期的结果应该是

{
    "org": {
        "iden": {
            "account":  "account123",
            "id": "id123"
        }
    },
    "address": {
        "city": "riga",
        "country": "country"
    },
    "person": {
        "name": {
            "fullname": laura,
        },
        "gender": "female",
        "birthYear": 1990
    },
    "subs": {
        "id": "subs123",
        "subs1": {
            "birthday": "1990-12-14T00:00:00Z",
            "org": {
                "address": {
                    "country": "latvia",
                    "street1": "street 1"
                }
            }
        },
        "org": {
            "buyer": {
                "email": {
                    "address": "email1@myorg.com|email2@sanoma.com"
                },
            "phone": {
                "number": "+371401234567"
                }
            }
        }
    }

}

然后当然是所有记录作为一个列表。我曾尝试使用简单的 pandas .to_json() 但没有帮助，我得到以下没有我需要的嵌套结构的内容。

[{"org.iden.account":"account123","org.iden.id":"id123","adress.city":"riga","adress.country":"latvia","person.name.fullname":"laura","person.gender":"female","person.birthYear":1990,"subs.id":"subs123","subs.subs1.birthday":"1990-12-14T00:00:00Z","subs.subs1.org.address.country":"latvia","subs.subs1.org.address.strret1":"street 1","subs.org.buyer.email.address":"email1@myorg.com|email2@sanoma.com","subs.org.buyer.phone.number":371401234567},{"org.iden.account":"account123","org.iden.id":"id000","adress.city":"riga","adress.country":"latvia","person.name.fullname":"laura","person.gender":"female","person.birthYear":1990,"subs.id":"subs456","subs.subs1.birthday":"1990-12-14T00:00:00Z","subs.subs1.org.address.country":"latvia","subs.subs1.org.address.strret1":"street 1","subs.org.buyer.email.address":"email1@myorg.com","subs.org.buyer.phone.number":371407654321},{"org.iden.account":"account123","org.iden.id":"id456","adress.city":"riga","adress.country":"latvia","person.name.fullname":"laura","person.gender":"female","person.birthYear":1990,"subs.id":"subs789","subs.subs1.birthday":"1990-12-14T00:00:00Z","subs.subs1.org.address.country":"latvia","subs.subs1.org.address.strret1":"street 1","subs.org.buyer.email.address":"email1@myorg.com","subs.org.buyer.phone.number":371407654321}]

对此的任何帮助将不胜感激！

【问题讨论】：

  • 您是否可以使用直接处理您的json 数据而不是通过pandas 的解决方案？
  • @Axe319 当然可以。 Pandas 实际上在数据帧上提供了很大的灵活性，但绝对欢迎使用其他解决方案

标签： json python-3.x pandas dataframe csv


【解决方案1】：

假设您的 json 结构看起来像这样

json_data = [
    {
        "org.iden.account": "account123",
        "org.iden.id": "id123",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs123",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com|email2@sanoma.com",
        "subs.org.buyer.phone.number": 371401234567
    },
    {
        "org.iden.account": "account123",
        "org.iden.id": "id000",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs456",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com",
        "subs.org.buyer.phone.number": 371407654321
    },
    {
        "org.iden.account": "account123",
        "org.iden.id": "id456",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs789",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com",
        "subs.org.buyer.phone.number": 371407654321
    }
]

您可以在 dict 和 dict 的基础上嵌套它。

def nestify(unnested):
    nested = dict()
    for k, v in unnested.items():
        current_dict = nested
        parts = k.split('.')
        for i in parts[:-1]:
            if i not in current_dict:
                current_dict[i] = dict()
            current_dict = current_dict[i]
        current_dict[parts[-1]] = v
    return nested

此函数采用未嵌套的dicts 之一，遍历键并将值分配给最终深度。

评论版本

def nestify(unnested):
    # this will be our return value
    nested = dict()
    for k, v in unnested.items():
        # current_dict is the current dict were operating on
        # gets reset to the base dict on each unnested key
        current_dict = nested
        parts = k.split('.')
        # only create dicts up to the final period
        # for example, current_dict is the base
        # and creates an empty dict under the org key
        # then current_dict is under the org key
        # and creates an empty dict under the iden key
        # then current_dict is under the iden key
        for i in parts[:-1]:
            # no reason to create an empty dict if it was
            # already created for a prior key
            if i not in current_dict:
                current_dict[i] = dict()
            current_dict = current_dict[i]
        # assign the value of the unnested dict
        # to each final current_dict
        # for example, the final part of the first key is "account"
        # so rather than assign an empty dict, assign it "account123" 
        current_dict[parts[-1]] = v
    return nested

然后您可以在理解中对json_data list 的每个元素调用它。

nested = [nestify(i) for i in json_data]

完整代码：

json_data = [
    {
        "org.iden.account": "account123",
        "org.iden.id": "id123",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs123",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com|email2@sanoma.com",
        "subs.org.buyer.phone.number": 371401234567
    },
    {
        "org.iden.account": "account123",
        "org.iden.id": "id000",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs456",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com",
        "subs.org.buyer.phone.number": 371407654321
    },
    {
        "org.iden.account": "account123",
        "org.iden.id": "id456",
        "adress.city": "riga",
        "adress.country": "latvia",
        "person.name.fullname": "laura",
        "person.gender": "female",
        "person.birthYear": 1990,
        "subs.id": "subs789",
        "subs.subs1.birthday": "1990-12-14T00:00:00Z",
        "subs.subs1.org.address.country": "latvia",
        "subs.subs1.org.address.strret1": "street 1",
        "subs.org.buyer.email.address": "email1@myorg.com",
        "subs.org.buyer.phone.number": 371407654321
    }
]


def nestify(unnested):
    nested = dict()
    for k, v in unnested.items():
        current_dict = nested
        parts = k.split('.')
        for i in parts[:-1]:
            if i not in current_dict:
                current_dict[i] = dict()
            current_dict = current_dict[i]
        current_dict[parts[-1]] = v
    return nested

nested = [nestify(i) for i in json_data]
print(nested)

输出：

[
    {
        'adress': {
            'city': 'riga', 
            'country': 'latvia'
        },
        'org': {
            'iden': {
                'account': 'account123', 
                'id': 'id123'
            }
        },
        'person': {
            'birthYear': 1990,
            'gender': 'female',
            'name': {
                'fullname': 'laura'
            }
        },
        'subs': {
            'id': 'subs123',
            'org': {
                'buyer': {
                    'email': {
                        'address': 'email1@myorg.com|email2@sanoma.com'
                    },
                    'phone': {
                        'number': 371401234567
                    }
                }
            },
            'subs1': {
                'birthday': '1990-12-14T00:00:00Z',
                'org': {
                    'address': {
                        'country': 'latvia',
                        'strret1': 'street 1'
                    }
                }
            }
        }
    },
    {
        'adress': {
            'city': 'riga', 
            'country': 'latvia'
        },
        'org': {
            'iden': {
                'account': 'account123', 
                'id': 'id000'
            }
        },
        'person': {
            'birthYear': 1990,
            'gender': 'female',
            'name': {
                'fullname': 'laura'
            }
        },
        'subs': {
            'id': 'subs456',
            'org': {
                'buyer': {
                    'email': {
                        'address': 'email1@myorg.com'
                    },
                    'phone': {
                        'number': 371407654321
                    }
                }
            },
            'subs1': {
                'birthday': '1990-12-14T00:00:00Z',
                'org': {
                    'address': {
                        'country': 'latvia',
                        'strret1': 'street 1'
                    }
                }
            }
        }
    },
    {
        'adress': {
            'city': 'riga', 
            'country': 'latvia'
        },
        'org': {
            'iden': {
                'account': 'account123', 
                'id': 'id456'
            }
        },
        'person': {
            'birthYear': 1990,
            'gender': 'female',
            'name': {
                'fullname': 'laura'
            }
        },
        'subs': {
            'id': 'subs789',
            'org': {
                'buyer': {
                    'email': {
                        'address': 'email1@myorg.com'
                    },
                    'phone': {
                        'number': 371407654321
                    }
                }
            },
            'subs1': {
                'birthday': '1990-12-14T00:00:00Z',
                'org': {
                    'address': {
                        'country': 'latvia',
                        'strret1': 'street 1'
                    }
                }
            }
        }
    }
]

【讨论】：

    【解决方案2】： 
    def df_to_json(row):
    tree = {}
    for item in row.index:
        t = tree
        for part in item.split('.'):
            prev, t = t, t.setdefault(part, {})
        prev[part] = row[item]
    return tree

    >>> df.apply(df_to_json, axis='columns').tolist()

[{'org': {'iden': {'account': 'account123', 'id': 'id123'}},
  'adress': {'city': 'riga', 'country': 'latvia'},
  'person': {'name': {'fullname': 'laura'},
   'gender': 'female',
   'birthYear': 1990},
  'subs': {'id': 'subs123',
   'subs1': {'birthday': '1990-12-14T00:00:00Z',
    'org': {'address': {'country': 'latvia', 'strret1': 'street 1'}}},
   'org': {'buyer': {'email': {'address': 'email1@myorg.com|email2@sanoma.com'},
     'phone': {'number': 371401234567}}}}},
 {'org': {'iden': {'account': 'account123', 'id': 'id000'}},
  'adress': {'city': 'riga', 'country': 'latvia'},
  'person': {'name': {'fullname': 'laura'},
   'gender': 'female',
   'birthYear': 1990},
  'subs': {'id': 'subs456',
   'subs1': {'birthday': '1990-12-14T00:00:00Z',
    'org': {'address': {'country': 'latvia', 'strret1': 'street 1'}}},
   'org': {'buyer': {'email': {'address': 'email1@myorg.com'},
     'phone': {'number': 371401234567}}}}},
 {'org': {'iden': {'account': 'account123', 'id': 'id456'}},
  'adress': {'city': 'riga', 'country': 'latvia'},
  'person': {'name': {'fullname': 'laura'},
   'gender': 'female',
   'birthYear': 1990},
  'subs': {'id': 'subs789',
   'subs1': {'birthday': '1990-12-14T00:00:00Z',
    'org': {'address': {'country': 'latvia', 'strret1': 'street 1'}}},
   'org': {'buyer': {'email': {'address': 'email1@myorg.com'},
     'phone': {'number': 371401234567}}}}}]

    【讨论】：

      猜你喜欢
      • 2020-07-01
      • 2017-06-10
      • 2019-11-14
      • 2018-01-07
      • 2020-10-28
      • 1970-01-01
      • 2019-12-20
      • 1970-01-01
      相关资源
      最近更新 更多
      热门标签
      Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode