python：将类型requests.models.Response导入数据框时出错答案

【问题标题】：python: Getting error when importing type requests.models.Response into dataframepython：将类型requests.models.Response导入数据框时出错
【发布时间】：2021-02-22 16:10:27
【问题描述】：

我是 python 新手，我正在尝试使用人口普查地理编码服务 API 对地址进行地理编码，然后将输出转换为数据框。我已经能够读取我的地址文件并且可以看到输出，但我似乎无法弄清楚如何将其导入数据框。我提供了我在下面使用的代码以及地址文件的内容。

输出似乎不是 JSON 格式，而是 CSV。我尝试像导入 CSV 文件一样导入输出，但我不知道如何像导入 CSV 文件一样导入变量，也不知道如何将输出导出到我可以的 CSV 文件导入。

描述 API 的 URL 是https://geocoding.geo.census.gov/geocode...es_API.pdf

import requests
import pandas as pd
import json
url = 'https://geocoding.geo.census.gov/geocoder/geographies/addressbatch'
payload = {'benchmark':'Public_AR_Current','vintage':'Current_Current'}
files = {'addressFile': ('C:\PYTHON_CLASS\CSV\ADDRESS_SAMPLE.csv', open('C:\PYTHON_CLASS\CSV\ADDRESS_SAMPLE.csv', 'rb'), 'text/csv')}
response = requests.post(url, files=files, data = payload)
type(response)
print(response.text)

-我尝试了下面的代码（在许多其他版本中），这是我通常导入 CSV 文件的方式，但它会生成错误消息“无效的文件路径或缓冲区对象类型："

df = pd.read_csv(response)

我用来生成地理编码的地址文件的内容是：

id、地址、城市、州、邮政编码 1,1600 宾夕法尼亚大道 NW，华盛顿特区，20500 2,4 S Market St,波士顿,MA,02109 3,1200 Getty Center Drive,洛杉矶,CA,90049 4,1800 Congress Ave,奥斯汀,TX,78701 5,One Caesars Palace Drive,拉斯维加斯,NV,89109 6,1060 West Addison,芝加哥,IL,60613 7,One East 161st Street,Bronx,NY,10451 8,201 E Jefferson St,Phoenix,AZ,85004 9,600 N 1st Ave,Minneapolis,MN,55403 10,400 W Church St,Orlando,FL,32801

输出如下图：

print(response.text)

"1","1600 Pennsylvania Avenue NW, Washington, DC, 20500","Match","Non_Exact","1600 PENNSYLVANIA AVE NW, WASHINGTON, DC, 20006","-77.03535,38.898754","76225813 ","L","11","001","006202","1031" "2","4 S Market St, Boston, MA, 02109","Match","Exact","4 S MARKET ST, BOSTON, MA, 02109","-71.05566,42.359936","85723841"," R","25","025","030300","2017" "3","1200 Getty Center Drive, Los Angeles, CA, 90049","Match","Exact","1200 GETTY CENTER DR, LOS ANGELES, CA, 90049","-118.47564,34.08857","142816014" ,"L","06","037","262302","1005" "4","1800 Congress Ave, Austin, TX, 78701","Match","Exact","1800 CONGRESS AVE, AUSTIN, TX, 78701","-97.73847,30.279745","63946318","L" ,"48","453","000700","1007" "5","One Caesars Palace Drive, Las Vegas, NV, 89109","No_Match" "6","1060 West Addison, Chicago, IL, 60613","Match","Non_Exact","1060 W ADDISON ST, CHICAGO, IL, 60613","-87.65581,41.947227","111863716","R ","17","031","061100","1014" "7","One East 161st Street, Bronx, NY, 10451","No_Match" "8","201 E Jefferson St, Phoenix, AZ, 85004","Match","Exact","201 E Jefferson St, PHOENIX, AZ, 85004","-112.07113,33.44675","128300920"," L","04","013","114100","1058" "9","600 N 1st Ave, Minneapolis, MN, 55403","No_Match" "id","地址、城市、州、邮政编码","No_Match" "10","400 W Church St, Orlando, FL, 32801","Match","Exact","400 W CHURCH ST, ORLANDO, FL, 32801","-81.38436,28.540176","94416807"," L","12","095","010500","1002"

response.text 的输出是：

'"1","1600 Pennsylvania Avenue NW, Washington, DC, 20500","Match","Non_Exact","1600 PENNSYLVANIA AVE NW, WASHINGTON, DC, 20006","-77.03535,38.898754"," 76225813","L","11","001","006202","1031"\n"2","4 S Market St, Boston, MA, 02109","Match","Exact"," 4 S MARKET ST, BOSTON, MA, 02109","-71.05566,42.359936","85723841","R","25","025","030300","2017"\n"3","1200 Getty Center Drive, Los Angeles, CA, 90049","Match","Exact","1200 GETTY CENTER DR, LOS ANGELES, CA, 90049","-118.47564,34.08857","142816014","L"," 06","037","262302","1005"\n"4","1800 Congress Ave, Austin, TX, 78701","Match","Exact","1800 CONGRESS AVE, AUSTIN, TX, 78701 ","-97.73847,30.279745","63946318","L","48","453","000700","1007"\n"5","One Caesars Palace Drive, Las Vegas, NV, 89109 ","No_Match"\n"6","1060 West Addison, Chicago, IL, 60613","Match","Non_Exact","1060 W ADDISON ST, CHICAGO, IL, 60613","-87.65581,41.947227" ,"111863716","R","17","031","061100","1014"\n"7","One East 161st Street, Bronx, NY, 10451","No_Match"\n"8 ","201 E Jefferson St, Phoenix, AZ, 85004","Match","Exact","201 E JEFFERSON ST, PHOENIX, AZ, 85004","-112.07113,33.44675","128300920","L","04" ,"013","114100","1058"\n"9","600 N 1st Ave, Minneapolis, MN, 55403","No_Match"\n"id","地址、城市、州、邮政编码", "No_Match"\n"10","400 W Church St, Orlando, FL, 32801","Match","Exact","400 W CHURCH ST, ORLANDO, FL, 32801","-81.38436,28.540176", "94416807","L","12","095","010500","1002"\n'

当我尝试时

df = pd.read_csv(io.StringIO(response), sep=',', header=None, quoting=csv.QUOTE_ALL)

我收到了错误消息

TypeError                                 Traceback (most recent call last)
<ipython-input-60-55e6c5ac54af> in <module>
----> 1 df = pd.read_csv(io.StringIO(response), sep=',', header=None, quoting=csv.QUOTE_ALL)

TypeError: initial_value must be str or None, not Response

当我尝试时

df = pd.read_csv(io.StringIO(response.replace('" "', '"\n"')), sep=',', header=None, quoting=csv.QUOTE_ALL)

我明白了

AttributeError                            Traceback (most recent call last)
<ipython-input-61-a92a7ffcf170> in <module>
----> 1 df = pd.read_csv(io.StringIO(response.replace('" "', '"\n"')), sep=',', header=None, quoting=csv.QUOTE_ALL)

AttributeError: 'Response' object has no attribute 'replace'

【问题讨论】：

标签： python pandas api dataframe python-requests

【解决方案1】：

如果response.text 看起来像下面的s 字符串，即用换行符分隔行，您可以尝试：

>>> import csv
>>> import io
>>> import pandas as pd 
>>>
>>> # let's pretend s is response.text
>>> s = '''"1","1600 Pennsylvania Avenue NW, Washington, DC, 20500","Match","Non_Exact","1600 PENNSYLVANIA AVE NW, WASHINGTON, DC, 20006","-77.03535,38.898754","76225813","L","11","001","006202","1031"
... "2","4 S Market St, Boston, MA, 02109","Match","Exact","4 S MARKET ST, BOSTON, MA, 02109","-71.05566,42.359936","85723841","R","25","025","030300","2017"
... "3","1200 Getty Center Drive, Los Angeles, CA, 90049","Match","Exact","1200 GETTY CENTER DR, LOS ANGELES, CA, 90049","-118.47564,34.08857","142816014","L","06","037","262302","1005"
... "4","1800 Congress Ave, Austin, TX, 78701","Match","Exact","1800 CONGRESS AVE, AUSTIN, TX, 78701","-97.73847,30.279745","63946318","L","48","453","000700","1007"'''
>>>
>>> df = pd.read_csv(io.StringIO(s), sep=',', header=None, quoting=csv.QUOTE_ALL)
>>> 
>>> with pd.option_context(
...       'display.width', None, 
...       'display.max_columns', None,
...       'display.max_colwidth', -1,
...       'display.colheader_justify', 'left'):
...     print(df)
... 
   0 1                                                   2      3          4                                                 \
0  1  1600 Pennsylvania Avenue NW, Washington, DC, 20500  Match  Non_Exact  1600 PENNSYLVANIA AVE NW, WASHINGTON, DC, 20006   
1  2  4 S Market St, Boston, MA, 02109                    Match  Exact      4 S MARKET ST, BOSTON, MA, 02109                  
2  3  1200 Getty Center Drive, Los Angeles, CA, 90049     Match  Exact      1200 GETTY CENTER DR, LOS ANGELES, CA, 90049      
3  4  1800 Congress Ave, Austin, TX, 78701                Match  Exact      1800 CONGRESS AVE, AUSTIN, TX, 78701              

  5                     6         7   8   9    10      11    
0  -77.03535,38.898754  76225813   L  11  1    6202    1031  
1  -71.05566,42.359936  85723841   R  25  25   30300   2017  
2  -118.47564,34.08857  142816014  L  6   37   262302  1005  
3  -97.73847,30.279745  63946318   L  48  453  700     1007

如果response.text 没有分隔行的换行符（但在您的描述中有空格），您需要将io.StringIO(s) 替换为io.StringIO(s.replace('" "', '"\n"'))。

【讨论】：

非常感谢您的明确指示。我更新了我的原始帖子以包含 response.text 的输出以及当我使用您的建议更新我的代码时产生的错误消息。我想我很接近了。我似乎无法一路走到那里。
您是否检查过我上面提到的response.text（不是response）（即io.StringIO(response.text) 或io.StringIO(response.text.replace('" "', '"\n"'))）？
@SAS2PYTHON 要清楚，您的错误消息表明您尝试使用 io.StringIO(response) 和 io.StringIO(response.replace('" "', '"\n"') 而不是 io.StringIO(response.text) 和 io.StringIO(response.text.replace('" "', '"\n"')
是的。我将变量名更改为 S 以避免与响应对象混淆。我的内衬读取 'df = pd.read_csv(io.StringIO(s), sep=',', header=None, quoting=csv.QUOTE_ALL)' 和 'df = pd.read_csv(io.StringIO(s.replace( '" "', '"\n"')), sep=',', header=None, quoting=csv.QUOTE_ALL)' 但两者都导致上述相同的错误。当我使用您的代码将 s.text 值粘贴到 S 变量中时，它会完美导入，但是您的 S 变量是文本对象，而我的变量是响应对象。如果我能弄清楚如何将其转换为文本对象，那么我认为它会起作用。
@SAS2PYTHON 不是 response.text 字符串吗？如果没有，type(response.text) 会给你什么？

【解决方案2】：

尼古拉斯提供了答案。我的最终代码是

import requests
import pandas as pd
import io
import csv
url = 'https://geocoding.geo.census.gov/geocoder/geographies/addressbatch'
payload = {'benchmark':'Public_AR_Current','vintage':'Current_Current'}
files = {'addressFile': ('C:\PYTHON_CLASS\CSV\ADDRESS_SAMPLE.csv', open('C:\PYTHON_CLASS\CSV\ADDRESS_SAMPLE.csv', 'rb'), 'text/csv')}
s = requests.post(url, files=files, data = payload)
df = pd.read_csv(io.StringIO(s.text), sep=',', header=None, quoting=csv.QUOTE_ALL)
with pd.option_context(
    'display.width', None, 
    'display.max_columns', None,
    'display.max_colwidth', -1,
    'display.colheader_justify', 'left'):
    print(df)

谢谢尼古拉斯！

【讨论】：