【发布时间】:2015-12-14 19:09:57
【问题描述】:
url = 'http://www.millercenter.org/president/speeches'
conn = urllib2.urlopen(url)
html = conn.read()
miller_center_soup = BeautifulSoup(html)
links = miller_center_soup.find_all('a')
for tag in links:
link = tag.get('href',None)
if link is not None:
print link
这是我的一些输出:
/president/washington/speeches/speech-3939
/president/washington/speeches/speech-3939
/president/washington/speeches/speech-3461
https://www.facebook.com/millercenter
https://twitter.com/miller_center
https://www.flickr.com/photos/miller_center
https://www.youtube.com/user/MCamericanpresident
http://forms.hoosonline.virginia.edu/s/1535/16-uva/index.aspx?sid=1535&gid=16&pgid=9982&cid=17637
mailto:mcpa-webmaster@virginia.edu
我正在尝试对网站millercenter.org/president/speeches 上的所有总统演讲进行网络抓取,但我无法保存我将从中抓取语音数据的适当语音链接。更明确地说,我需要乔治华盛顿的演讲,可通过http://www.millercenter.org/president/washington/speeches/speech-3461 访问 - 我只需要能够访问该网址。我正在考虑将所有演讲的所有 url 存储在一个列表中,然后编写一个 for 循环来抓取和清理所有数据。
【问题讨论】:
标签: python list python-2.7 beautifulsoup scrape