将 csv 转换为 Json 以使用 Python 发送到 watson-personality Insight API

Question

我正在尝试转换一个 csv，其中包含我提取的一些评论。我需要转换为 Json 作为 IBM watson 个性洞察的输入。

带有评论的 csv (WordFinal.csv) 是这样的：

ID,Review,advice,con,date,employeeType,overallStar,position,pro,reviewLink,reviewNo,summary
3,ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ,"Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ," Nov 9, 2017",Former Employee - Operations Manager,3,I worked at Amazon full-time (More than 3 years),ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm,empReview_17818923,Take it with a grain of salt
19, company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ,None, work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ," Oct 29, 2016",Current Employee - Product Manager,4,I have been working at Amazon full-time (More than 3 years), company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ,https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm,empReview_12494284,Incredible growth opportunity with downsides

IBM-Watson 接受的格式如下：

    {
   "contentItems": [
      {
         "content": "Wow, I liked @TheRock before, now I really SEE how special he is. The daughter story was IT for me. So great! #MasterClass",
         "contenttype": "text/plain",
         "created": 1447639154000,
         "id": "666073008692314113",
         "language": "en"
      },
      {
         "content": ".@TheRock how did you Know to listen to your gut and Not go back to football? #Masterclass",
         "contenttype": "text/plain",
         "created": 1447638226000,
         "id": "666069114889179136",
         "language": "en"
      },
      {
         "content": ".@TheRock moving back in with your parents so humbling. \" on the other side of your pain is something good if you can hold on\" #masterclass",
         "contenttype": "text/plain",
         "created": 1447638067000,
         "id": "666068446325665792",
         "language": "en"
      }   
   ]
}

我正在尝试生成 IBM-watson 接受的输出，但我不知道如何才能完成这样的任务。代码如下：

import json, csv

csvfile = 'C:\\WordFinal.csv'
jsonfile = 'C:\\OutputJ.json'

fieldnames=['ID','summary']
data= {}
with open(csvfile) as cs:
    #reader = csv.DictReader(cs)
    reader = csv.DictReader(cs, fieldnames)
    for row in reader:
    csvid = row["ID"]
    data[csvid] = row

with open(jsonfile, "w",encoding='utf8') as js:
    js.write(json.dumps(data,indent=2))

以及输出（我只需要我的 csv 中的 ID 和汇总列 - 第一列和最后一列）：

   {
  "ID": {
    "ID": "ID",
    "summary": "Review",
    "null": [
      "advice",
      "con",
      "date",
      "employeeType",
      "overallStar",
      "position",
      "pro",
      "reviewLink",
      "reviewNo",
      "summary"
    ]
  },
  "3": {
    "ID": "3",
    "summary": "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company . my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
    "null": [
      "Cali - keep it up, you have culture nailedJersey - This is amazon, you can't raise the bar at amazon without actually being innovating and taking big steps to change things up.",
      "my only con was the work life balance i was put on nights my entire time with amazon in order to continue to help the lower performing shifts and this would lead to having a daily schedule of wake up go straight to work usually forget to eat leave after 14 hours go to the gym eat maybe sleep wake up after 4 5 hours do it again based on location your leadership team s culture may vary drastically in california i had an extremely supportive senior and general manager team once i came to new jersey the senior team was pretty blind to their own operation super focused on creating reports continuous innovation projects in excel sheets versus just letting us go out there and do the actual improvements there was an extremely slow uptake on just do it recommendations and it was highly evident that this mentality was built into the department as well so again take it with a grain of salt ",
      " Nov 9, 2017",
      "Former Employee - Operations Manager",
      "3",
      "I worked at Amazon full-time\u00c2\u00a0(More than 3 years)",
      "ive been with amazon for 3 years before i finally resigned i started directly out of college as an area manager and promoted once a year to operation manager and was starting to get tapped for senior operations manager the progression at this company is unbelievable and growth potential is nearly limitless you get to work with a highly dedicated team that strives to raise the bar dont get me wrong you may come across individuals who also do the minimum and cut corners to get results but that is the case anywhere i was able to lead teams of up to 700 associates and indirectly oversee 1500 associates during the holiday season all at the age of 25 i was able to influence shift wide department wide building wide and even region wide procedures processes decisions for the company ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW17818923.htm",
      "empReview_17818923",
      "Take it with a grain of salt"
    ]
  },
  "19": {
    "ID": "19",
    "summary": " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry .  work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
    "null": [
      "None",
      " work life balance can be a challenge work demands are high and teams are often too lean you have to set your own boundaries even with kind managers overachievers will feel under water frugality as a core value goes overboard if amazon doesn t have to give it you it won t no perks no free food or drinks bad coffee unsubsidized cafeterias mediocre hardware for non technical people there doesn t seem to be a morale budget and you will have few official team outings compensation policies are not employee friendly 401k matching is subpar once your signing cash bonus is fully vested your entire compensation will be base salary and stock base salary is capped at 160k across the company stock vests twice a year if you re below a director so your compensation is very lumpy stock price appreciation is taken into consideration in your total compensation targets ie if the value of previously offered shares increases the company will count that as a raise and might not grant you additional stock bonuses despite strong performance ",
      " Oct 29, 2016",
      "Current Employee - Product Manager",
      "4",
      "I have been working at Amazon full-time\u00c2\u00a0(More than 3 years)",
      " company is on an unstoppable growth trajectory amazons business model is incredible is riding a number of secular trends ecommerce cloud ai and the stock is a winner employees are making more money than expected leadership principles drive a high performance culture that focuses on customers it feels great to work on products that customers love you get to work on very difficult problems with smart people once you establish yourself as a high performer you have a high level of job security and internal mobility teams are constantly hiring and building really innovative things and you are encouraged to move around and explore teams tend to be lean and you will be asked to learn a lot quickly ownership is highly valued office environment is really desirable located in a great downtown seattle neighborhood many people walk to work bring dogs to the office and restaurants and bars are very accessible amazon veterans tend to be incredibly talented individuals and other companies realize it being successful at amazon is well respected in the industry ",
      "https://www.glassdoor.com/Reviews/Amazon-Reviews-E6036.htm/Reviews/Employee-Review-Amazon-RVW12494284.htm",
      "empReview_12494284",
      "Incredible growth opportunity with downsides"
    ]
  }
}

知道如何获得正确的格式吗？

Answer 1

不要指定字段名称。从完整记录中提取您想要的：

with open(csvfile, "r", encoding="utf-8") as infile:
    reader = csv.DictReader(infile)

    with open(jsonfile, "w", encoding="utf-8") as outfile:
        outfile.write(json.dumps([{"ID": row["ID"], "summary": row["summary"]} for row in reader], indent=2))

给予：

[
  {
    "ID": "3",
    "summary": "Take it with a grain of salt"
  },
  {
    "ID": "19",
    "summary": "Incredible growth opportunity with downsides"
  }
]

Answer 2

如果您不需要 summary 以外的任何内容，只需将 summary 存储在 data 中即可

with open(csvfile) as cs:
    reader = csv.DictReader(cs, fieldnames)
    first_row = next(reader)             # skip reading the header line
    for row in reader[:
        csvid = row["ID"]
        data[csvid] = row['summary']     # just store the summary

Answer 3

所以我发现 pandas 更容易使用。

首先创建一个包含您的 csv 信息的数据框（假设您的文件格式正确）。

import pandas as pd
df = pd.read_csv('WordFinal.csv')

然后您可以迭代数据框。

records = []
for index, row in df.iterrows(): 
    rec = { 
        row['ID']: { 
            'ID': row['ID'],
            'summary': row['summary']
            # ... etc
        }
    }

    records.append(rec)

您的记录对象将包含您可以保存的 JSON 代码。