如何将 requests.models.Response(obj) 转换为有用的东西，如列表，然后转换为 DataFrame？

Question

我使用 Postman 在网站上进行了 scrape。之后，在同一个程序上，我使用该工具生成了 sn-p 代码，所以我可以在 jupyterNote 上使用它。

代码：

import requests

url = "https://www.tennet.eu/electricity-market/transparency-pages/transparency-germany/reporting-market-relevance/feed-in-management-according-to-par-14/information-on-feed-in-management-levels/?tx_tennetcw_transparency%5Baction%5D=csvDownload&tx_tennetcw_transparency%5Bcontroller%5D=Transparency&cHash=d3d32d8a6d86f1121d287aa727ff5110"

payload='tx_tennetcw_transparency%5B__referrer%5D%5B%40extension%5D=TennetCw&tx_tennetcw_transparency%5B__referrer%5D%5B%40vendor%5D=Arvato&tx_tennetcw_transparency%5B__referrer%5D%5B%40controller%5D=Transparency&tx_tennetcw_transparency%5B__referrer%5D%5B%40action%5D=list&tx_tennetcw_transparency%5B__referrer%5D%5Barguments%5D=YTowOnt96b882eafe83827345698c053f8021fe553f44326&tx_tennetcw_transparency%5B__referrer%5D%5B%40request%5D=a%3A4%3A%7Bs%3A10%3A%22%40extension%22%3Bs%3A8%3A%22TennetCw%22%3Bs%3A11%3A%22%40controller%22%3Bs%3A12%3A%22Transparency%22%3Bs%3A7%3A%22%40action%22%3Bs%3A4%3A%22list%22%3Bs%3A7%3A%22%40vendor%22%3Bs%3A6%3A%22Arvato%22%3B%7D8f91dd11199b4d706eaa7939dfa04d22d7122893&tx_tennetcw_transparency%5B__trustedProperties%5D=a%3A5%3A%7Bs%3A7%3A%22current%22%3Bi%3A1%3Bs%3A3%3A%22uid%22%3Bi%3A1%3Bs%3A5%3A%22sDate%22%3Bi%3A1%3Bs%3A5%3A%22eDate%22%3Bi%3A1%3Bs%3A5%3A%22email%22%3Bi%3A1%3B%7D172537dd2ccfc75d9cd1f075558d525c95ff58e2&tx_tennetcw_transparency%5Bcurrent%5D=FeedInManagementActivities&tx_tennetcw_transparency%5Buid%5D=55594&tx_tennetcw_transparency%5BsDate%5D=2021-08-01%2000%3A00&tx_tennetcw_transparency%5BeDate%5D=2021-08-24%2023%3A59&type=7788'

headers = {
  'Content-Type': 'application/x-www-form-urlencoded'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

输出为：

"Action ID";"feed-in source";"level [%]";Start;"Time of publication"

TOG2021-0126;"Veja Mate AC105";100,0;"2021-08-13 20:11";"2021-08-13 20:20"

TOG2021-0126;"Veja Mate AC105";85,2;"2021-08-13 19:52";"2021-08-13 19:54"

TOG2021-0126;"Veja Mate AC105";75,7;"2021-08-13 16:14";"2021-08-13 16:22"

TOG2021-0126;"Veja Mate AC104";100,0;"2021-08-13 20:11";"2021-08-13 20:20"

TOG2021-0126;"Veja Mate AC104";79,8;"2021-08-13 19:52";"2021-08-13 19:54"

TOG2021-0126;"Veja Mate AC104";71;"2021-08-13 16:14";"2021-08-13 16:22"

我尝试过的：

1. 将响应解析为 JSON 对象。
2. 转换成字符串或字符串列表。

对我来说，这看起来像是 word 中的 excel 表格。

我想要的是 5 列标题，以及每个 Action-ID 的连续行。然后，我终于可以把它放到一个DataFrame中了。

在此先感谢您，对任何菜鸟表示抱歉

Answer 1

您可以使用io.StringIO + pd.read_csv 将响应读入 panda DataFrame：

import requests
import pandas as pd
from io import StringIO


url = "https://www.tennet.eu/electricity-market/transparency-pages/transparency-germany/reporting-market-relevance/feed-in-management-according-to-par-14/information-on-feed-in-management-levels/?tx_tennetcw_transparency%5Baction%5D=csvDownload&tx_tennetcw_transparency%5Bcontroller%5D=Transparency&cHash=d3d32d8a6d86f1121d287aa727ff5110"
payload = "tx_tennetcw_transparency%5B__referrer%5D%5B%40extension%5D=TennetCw&tx_tennetcw_transparency%5B__referrer%5D%5B%40vendor%5D=Arvato&tx_tennetcw_transparency%5B__referrer%5D%5B%40controller%5D=Transparency&tx_tennetcw_transparency%5B__referrer%5D%5B%40action%5D=list&tx_tennetcw_transparency%5B__referrer%5D%5Barguments%5D=YTowOnt96b882eafe83827345698c053f8021fe553f44326&tx_tennetcw_transparency%5B__referrer%5D%5B%40request%5D=a%3A4%3A%7Bs%3A10%3A%22%40extension%22%3Bs%3A8%3A%22TennetCw%22%3Bs%3A11%3A%22%40controller%22%3Bs%3A12%3A%22Transparency%22%3Bs%3A7%3A%22%40action%22%3Bs%3A4%3A%22list%22%3Bs%3A7%3A%22%40vendor%22%3Bs%3A6%3A%22Arvato%22%3B%7D8f91dd11199b4d706eaa7939dfa04d22d7122893&tx_tennetcw_transparency%5B__trustedProperties%5D=a%3A5%3A%7Bs%3A7%3A%22current%22%3Bi%3A1%3Bs%3A3%3A%22uid%22%3Bi%3A1%3Bs%3A5%3A%22sDate%22%3Bi%3A1%3Bs%3A5%3A%22eDate%22%3Bi%3A1%3Bs%3A5%3A%22email%22%3Bi%3A1%3B%7D172537dd2ccfc75d9cd1f075558d525c95ff58e2&tx_tennetcw_transparency%5Bcurrent%5D=FeedInManagementActivities&tx_tennetcw_transparency%5Buid%5D=55594&tx_tennetcw_transparency%5BsDate%5D=2021-08-01%2000%3A00&tx_tennetcw_transparency%5BeDate%5D=2021-08-24%2023%3A59&type=7788"
headers = {"Content-Type": "application/x-www-form-urlencoded"}
response = requests.request("POST", url, headers=headers, data=payload)

df = pd.read_csv(StringIO(response.text), sep=";")
print(df)

打印：

        Action ID            feed-in source level [%]             Start Time of publication
0    TOG2021-0126           Veja Mate AC105     100,0  2021-08-13 20:11    2021-08-13 20:20
1    TOG2021-0126           Veja Mate AC105      85,2  2021-08-13 19:52    2021-08-13 19:54
2    TOG2021-0126           Veja Mate AC105      75,7  2021-08-13 16:14    2021-08-13 16:22
3    TOG2021-0126           Veja Mate AC104     100,0  2021-08-13 20:11    2021-08-13 20:20
4    TOG2021-0126           Veja Mate AC104      79,8  2021-08-13 19:52    2021-08-13 19:54

...and so on.