使用 pandas 将 CSV 转换为嵌套 JSON - 输出略有偏差答案

【问题标题】：CSV to nested JSON using pandas - Output slightly off使用 pandas 将 CSV 转换为嵌套 JSON - 输出略有偏差
【发布时间】：2017-11-19 06:31:50
【问题描述】：

我无法定制 this code 以满足我的需求。我觉得我很接近，但我还不够。目标是从 csv 文件创建嵌套的 JSON。我在下面有所需的输出、CSV 数据和我当前的代码。任何帮助表示赞赏。

当前代码：

import json
import pandas as pd

df = pd.read_csv('txn_data.csv')

def get_nested_rec(key, grp):
    rec = {}
    rec['date'] = key[0]
    rec['name'] = key[1]
    rec['value'] = key[2]

    for field in ['name','value']:
        rec[field] = list(grp[field].unique())

    return rec

records = []
for key, grp in df.groupby(['date']):
    rec = get_nested_rec(key, grp)
    records.append(rec)

records = dict(data = records)

print(json.dumps(records, indent=4))

CSV 数据：

date,name,value
1/1/13,Quick Serve,304127
1/1/13,Restaurant,1843286
1/1/13,Retail,239675
1/2/13,Quick Serve,422847
1/2/13,Restaurant,1582848
1/2/13,Retail,394358

所需的 JSON 输出：

desired_output = [  
   {  
      "date":"2017-01-01",
      "details":[  
         {  
            "name":"Retail",
            "value":9192
         },
         {  
            "name":"Restaurant",
            "value":6753
         },
         {  
            "name":"Quickserve",
            "value":1219
         }
      ]
   },
   {  
      "date":"2017-02-01",
      "details":[  
         {  
            "name":"Retail",
            "value":9192
         },
         {  
            "name":"Restaurant",
            "value":6753
         },
         {  
            "name":"Quickserve",
            "value":1219
         }
      ]
   }
]

我目前得到的：

{
    "data": [
        {
            "date": "1", 
            "name": [
                "Automotive", 
                "Durable Goods", 
                "Entertainment", 
                "Food", 
                "Lodging", 
                "Petroleum", 
                "Quick Serve", 
                "Restaurant", 
                "Retail", 
                "Service", 
                "Transportation & Utilities", 
                "Unknown"
            ], 
            "value": [
                91406, 
                9889, 
                172676, 
                358922, 
                63502, 
                1982048, 
                304127, 
                1843286, 
                239675, 
                106462, 
                25924, 
                909
            ]
        }, 
        {
            "date": "1", 
            "name": [
                "Automotive", 
                "Durable Goods", 
                "Entertainment", 
                "Food", 
                "Lodging", 
                "Petroleum", 
                "Quick Serve", 
                "Restaurant", 
                "Retail", 
                "Service", 
                "Transportation & Utilities", 
                "Unknown"
            ], 
            "value": [
                146041, 
                33090, 
                103159, 
                336956, 
                66726, 
                2191346, 
                422847, 
                1582848, 
                394358, 
                339989, 
                49477, 
                494
            ]
        }
    ]
}

【问题讨论】：

标签： python json csv

【解决方案1】：

我会尝试用一种更简单的方法来解决这个任务，如下所示：

import json
import pandas as pd

df = pd.read_csv('test.csv')
l_data = []
data = {}

for key,grp in df.groupby('date'):
    data['date'] = key
    data['details'] = df.loc[df['date'] == key][['name','value']].to_json(orient='records')
    l_data.append(data)

In [32]:
print(json.dumps(l_data))

Out[32]:
[  
   {  
      "date":"1/2/13",
      "details":[  
         {  
            "name":"Quick Serve",
            "value":422847
         },
         {  
            "name":"Restaurant",
            "value":1582848
         },
         {  
            "name":"Retail",
            "value":394358
         }
      ]
   },
   {  
      "date":"1/2/13",
      "details":[  
         {  
            "name":"Quick Serve",
            "value":422847
         },
         {  
            "name":"Restaurant",
            "value":1582848
         },
         {  
            "name":"Retail",
            "value":394358
         }
      ]
   }
]

【讨论】：

【解决方案2】：

我已将您的代码调整为以您要求的格式输出。

import json
import pandas as pd

df = pd.read_csv('txn_data.csv')

def get_nested_rec(key, grp):
    rec = {}
    rec['date'] = key
    rec['details'] = []

    for index, row in grp.iterrows():
        rec['details'].append({
            'name': row['name'],
            'value': row['value']
        })

    return rec

records = []
for key, grp in df.groupby(['date']):
    rec = get_nested_rec(key, grp)
    records.append(rec)

records = dict(data = records)

print(json.dumps(records, indent=4))

这是结果输出：

{
    "data": [
        {
            "date": "1/1/13", 
            "details": [
                {
                    "name": "Quick Serve", 
                    "value": 304127
                }, 
                {
                    "name": "Restaurant", 
                    "value": 1843286
                }, 
                {
                    "name": "Retail", 
                    "value": 239675
                }
            ]
        }, 
        {
            "date": "1/2/13", 
            "details": [
                {
                    "name": "Quick Serve", 
                    "value": 422847
                }, 
                {
                    "name": "Restaurant", 
                    "value": 1582848
                }, 
                {
                    "name": "Retail", 
                    "value": 394358
                }
            ]
        }
    ]
}

【讨论】：