如何将我抓取的文本格式的 json 数据集保存到本地机器中，以及如何将文件读入 Pandas DataFrame？

【问题标题】：How can i save my scraped json dataset that is a text format into my local machine and also how can i read the file into Pandas DataFrame?如何将我抓取的文本格式的 json 数据集保存到本地机器中，以及如何将文件读入 Pandas DataFrame？
【发布时间】：2021-12-04 14:47:33
【问题描述】：

代码如下所示：
我想用它来分析

response = requests.request("GET", url, headers=headers, params=querystring)

print(response.text)

{"@type":"imdb.api.title.ratings","id":"/title/tt0944947/","title":"Game of Thrones","titleType":"tvSeries","year":2011,"canRate":true,"otherRanks":[{"id":"/chart/ratings/toptv","label":"Top 250 TV","rank":12,"rankType":"topTv"}],"rating":9.2,"ratingCount":1885115,"ratingsHistograms":{"Males Aged 18-29":{"aggregateRating":9.3,"demographic":"Males Aged 18-29","histogram":{"1":11186,"2":693,"3":801,"4":962,"5":2103,"6":3583,"7":9377,"8":22859,"9":52030,"10":174464},"totalRatings":278058},"IMDb Staff":{"aggregateRating":8.7,"demographic":"IMDb Staff","histogram":{"1":0,"2":0,"3":0,"4":0,"5":1,"6":3,"7":6,"8":19,"9":27,"10":17},"totalRatings":73}

【问题讨论】：

如果您将其作为单个字符串，则只需使用open()、write()、close()

标签： python json pandas api web

【解决方案1】：

坦率地说，您应该在任何 Python 教程或requests 的许多示例中找到它

fh = open("output.json")
fh.write(response.text)
fh.close()

或

with open("output.json") as fh:
    fh.write(response.text)

至于pandas你可以试试看

df = pd.read_json("output.json")

或者您可以使用模块io 读取它而不保存在磁盘上

import io

fh = io.StringIO(response.text)
df = pd.read_json(fh)

但是pandas 将数据保存为包含行和列的表格，但您有嵌套的列表/字典，因此可能需要一些工作才能将其保存在DataFrame 中。

如果您只想从json 获取一些数据，那么您可以使用response.json()

【讨论】：