将 json 转换为 csv 保持非 ascii 字符不变

Question

我有一个这种格式的 .json 文件

{"contributors": null, "truncated": false, "text": "“cool jeans,” i tell a cute boy\nlittle did he know that im talking about his genes bc those chromosomes have combined beautifully ay papi", "is_quote_status": false, "in_reply_to_status_id": null, "id": 786650297116532736, "favorite_count": 631, "source": "<a href=\"http://bufferapp.com\" rel=\"nofollow\">Buffer</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [], "hashtags": [], "urls": []}, "in_reply_to_screen_name": null, "in_reply_to_user_id": null, "retweet_count": 233, "id_str": "786650297116532736", "favorited": false, "user": {"follow_request_sent": false, "has_extended_profile": false, "profile_use_background_image": true, "default_profile_image": false, "id": 321445166, "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/378800000024702436/c56435230dd53432a3aa7685fdd98cf7.jpeg", "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/782124224198631424/zuUAjl5o_normal.jpg", "profile_sidebar_fill_color": "EFEFEF", "entities": {"url": {"urls": [{"indices": [0, 23], "expanded_url": "https://youtu.be/1qjR-p_o3BE", "display_url": "youtu.be/1qjR-p_o3BE"}]}, "description": {"urls": []}}, "followers_count": 4992012, "profile_sidebar_border_color": "FFFFFF", "id_str": "321445166", "profile_background_color": "182D66", "listed_count": 5284, "is_translation_enabled": false, "utc_offset": 7200, "statuses_count": 28488, "description": "There's a fine line between being sassy and being an asshole and I cross it everyday. Youtuber and student. Link in bio thatssarcasmposts@gmail.com", "friends_count": 55420, "location": "Cape Town", "profile_link_color": "536BA7", "profile_image_url": "http://pbs.twimg.com/profile_images/782124224198631424/zuUAjl5o_normal.jpg", "following": false, "geo_enabled": false, "profile_banner_url": "https://pbs.twimg.com/profile_banners/321445166/1475082140", "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/378800000024702436/c56435230dd53432a3aa7685fdd98cf7.jpeg", "screen_name": "ThatsSarcasm", "lang": "en", "profile_background_tile": false, "favourites_count": 3931, "name": "joke", "notifications": false, "created_at": "Tue Jun 21 15:52:41 +0000 2011", "contributors_enabled": false, "time_zone": "Pretoria", "protected": false, "default_profile": false, "is_translator": false}, "geo": null, "in_reply_to_user_id_str": null, "lang": "en", "created_at": "Thu Oct 13 19:30:20 +0000 2016", "in_reply_to_status_id_str": null, "place": null}
{"contributors": null, "truncated": false, "text": "If you're not following @relatabIe for the most relatable tweets ever, then what are you doing?I love their posts????????", "is_quote_status": false, "in_reply_to_status_id": null, "id": 786649227002781696, "favorite_count": 29, "source": "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>", "retweeted": false, "coordinates": null, "entities": {"symbols": [], "user_mentions": [{"id": 485854360, "indices": [24, 34], "id_str": "485854360", "screen_name": "relatabIe", "name": "Relatable Tweets!"}], "hashtags": [], "urls": []}, "in_reply_to_screen_name": null, "in_reply_to_user_id": null, "retweet_count": 6, "id_str": "786649227002781696", "favorited": false, "user": {"follow_request_sent": false, "has_extended_profile": false, "profile_use_background_image": true, "default_profile_image": false, "id": 321445166, "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/378800000024702436/c56435230dd53432a3aa7685fdd98cf7.jpeg", "verified": false, "profile_text_color": "333333", "profile_image_url_https": "https://pbs.twimg.com/profile_images/782124224198631424/zuUAjl5o_normal.jpg", "profile_sidebar_fill_color": "EFEFEF", "entities": {"url": {"urls": [{"indices": [0, 23], "expanded_url": "https://youtu.be/1qjR-p_o3BE", "display_url": "youtu.be/1qjR-p_o3BE"}]}, "description": {"urls": []}}, "followers_count": 4992012, "profile_sidebar_border_color": "FFFFFF", "id_str": "321445166", "profile_background_color": "182D66", "listed_count": 5284, "is_translation_enabled": false, "utc_offset": 7200, "statuses_count": 28488, "description": "There's a fine line between being sassy and being an asshole and I cross it everyday. Youtuber and student. Link in bio thatssarcasmposts@gmail.com", "friends_count": 55420, "location": "Cape Town", "profile_link_color": "536BA7", "profile_image_url": "http://pbs.twimg.com/profile_images/782124224198631424/zuUAjl5o_normal.jpg", "following": false, "geo_enabled": false, "profile_banner_url": "https://pbs.twimg.com/profile_banners/321445166/1475082140", "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/378800000024702436/c56435230dd53432a3aa7685fdd98cf7.jpeg", "screen_name": "ThatsSarcasm", "lang": "en", "profile_background_tile": false, "favourites_count": 3931, "name": "joke", "notifications": false, "created_at": "Tue Jun 21 15:52:41 +0000 2011", "contributors_enabled": false, "time_zone": "Pretoria", "protected": false, "default_profile": false, "is_translator": false}, "geo": null, "in_reply_to_user_id_str": null, "lang": "en", "created_at": "Thu Oct 13 19:26:05 +0000 2016", "in_reply_to_status_id_str": null, "place": null}

我用来转换的python代码是

import json
import csv

x=open('MyFile.json')
data=json.load(x)
x.close()

f=csv.writer(open('data.csv','wb+')) 
for item in data:
    f.writerow(item["text"])

f.close()

我的目标是只在 csv 文件中写入文本属性（第三个），但它显示以下错误

File "to_csv.py", line 10, in <module>
  f.writerow(item["text"])
UnicodeEncodeError: 'ascii' codec can't encode character u'\U0001f60f' in position 0: ordinal not in range(128)

我已经搜索并找到了一个提示。我将第 10 行替换为：

f.writerow(unicode(item["text"]).encode("utf-8"))

但无济于事。找不到保持非 ASCII 字符完整的方法。

编辑：感谢 mx0 的解决方案，代码运行良好。我注意到的一件事是有换行符，因此单个文本属性会打印在两行中，例如检查 json 文件第一行中的文本属性。 “男孩”和“小”之间有一个换行符。我想避免打印那些所以我写了这段代码

f = csv.writer(open('data.csv', 'wb+'))
for item in data:
    if item["text"].encode('utf-8')=='\n':
        f.writerow(" ")
    else:
        f.writerow([item["text"].encode('utf-8')])

但它似乎不起作用。有什么问题吗？

Answer 1

更新 问题没有指定python版本，第一个答案是python 3。

您的代码中存在一些问题。

您的 json 文件无效（或者您在此处粘贴错误）。应该是这样的

[{"first":"object"}, {"second":"object"}] 

读取json时，必须指定编码

x = open('MyFile.json', encoding='utf-8')

写入时不要使用二进制模式，并指定编码。还要添加newline='' 参数，否则文件将在输出中包含双换行符。

f = csv.writer(open('data.csv', 'w+', encoding='utf-8', newline=''))

函数writerow 获取一个列表并将其写入行。如果您不将其包含在另一个列表中，您的文本将以I,f, ,y,o,u,',r,e, ,n,o,t 结尾

f.writerow([item["text"]])

完整的工作示例

Python 3.5

import json
import csv

x = open('MyFile.json', encoding='utf-8')
data = json.load(x)
x.close()

f = csv.writer(open('data.csv', 'w+', encoding='utf-8', newline=''))
for item in data:
    f.writerow([item["text"]])

Python 2.7

对于 python 2，您非常接近获得工作程序。在 py2 中，csv.writerow 以 bytes 模式写入文件，但您的 item["text"] 是 unicode 字符串，因此您必须先对其进行编码。只要您的 json 文件是真正的 unicode 编码，这将起作用。

import json
import csv

x = open('MyFile.json')
data = json.load(x)
x.close()


f = csv.writer(open('data.csv', 'wb+'))
for item in data:
    f.writerow([item["text"].encode('utf-8')])

将换行符改为空格

f.writerow([item["text"].replace('\n', ' ').encode('utf-8')])

将换行符写成\n

f.writerow([item["text"].replace('\n', '\\n').encode('utf-8')])