使用 Python 抓取多个 wikitable

Question

我是 Python 的初学者。我的任务是从维基百科页面上抓取信息表。我想使用下面的代码进行抓取：

from pandas.io.html import read_html
page = requests.get('https://de.wikipedia.org/wiki/Köln')
wikitables = read_html(page, attrs={"class":"hintergrundfarbe5 float-right toptextcells infobox"})
print("Extracted {num} wikitables".format(num=len(wikitables)))

wikitables[0]

但由于 URL 中的特殊字符为 Köln，我收到以下错误：请帮助我在程序中的何处进行修改以抓取信息。

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-168-d9bd1e1d7548> in <module>
      2 page = requests.get('https://de.wikipedia.org/wiki/Köln')
      3 Soup = BeautifulSoup(page.content)
----> 4 wikitables = read_html(page, attrs={"class":"hintergrundfarbe5 float-right toptextcells infobox"})
      5 print("Extracted {num} wikitables".format(num=len(wikitables)))
      6 

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\io\html.py in read_html(io, match, flavor, header, index_col, skiprows, attrs, parse_dates, tupleize_cols, thousands, encoding, decimal, converters, na_values, keep_default_na, displayed_only)
   1092                   decimal=decimal, converters=converters, na_values=na_values,
   1093                   keep_default_na=keep_default_na,
-> 1094                   displayed_only=displayed_only)

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\io\html.py in _parse(flavor, io, match, attrs, encoding, displayed_only, **kwargs)
    914             break
    915     else:
--> 916         raise_with_traceback(retained)
    917 
    918     ret = []

~\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\compat\__init__.py in raise_with_traceback(exc, traceback)
    418         if traceback == Ellipsis:
    419             _, _, traceback = sys.exc_info()
--> 420         raise exc.with_traceback(traceback)
    421 else:
    422     # this version of raise is a syntax error in Python 3

TypeError: Cannot read object of type 'Response'

Answer 1

这与美丽的科隆无关......

你需要改变

wikitables = read_html(page, attrs={"..."})

到

wikitables = read_html(page.text, attrs={"..."})

它应该可以工作。