使用 BeautifulSoup 只考虑网页的某一部分内容

Question

如何让 BeautifulSoup 只考虑网页内容的特定部分？

例如，我想在页面http://www.dailypress.com/ 上的“当前查看最多”之后仅提取所有div 标签。

是这样的：

from bs4 import BeautifulSoup
import urllib2

url = ' http://www.dailypress.com/ '
page = urllib2.urlopen(url)
soup = BeautifulSoup(page.read())

我可以使用：

str(soup).find(' Most viewed right now')

定位句子，但对确定我想要的内容部分没有帮助。

Answer 1

找到包含最多浏览文章的div，并找到其中的所有链接：

>>> from bs4 import BeautifulSoup
>>> import urllib2
>>> import re
>>> url = "http://www.dailypress.com"
>>> soup = BeautifulSoup(urllib2.urlopen(url))
>>> most_viewed = soup.find('div', class_=re.compile('mostViewed'))
>>> for item in most_viewed.find_all('a'):
...     print item.text.strip()
... 
Body of driver recovered from Chesapeake Bay Bridge-Tunnel wreck
Hampton police looking for man linked to Friday's fatal apartment shooting
Police identify suspect in Saturday's fatal shooting in Hampton
Teen spice user: 'It's the new crack'
When spice came to Gloucester

这里的诀窍是我们首先找到Most Viewed 链接的容器——它是一个具有mostViewed 类的div。您可以在浏览器开发工具的帮助下检查它。