【问题标题】:Beautiful soup looping through Urls and displaying data美丽的汤循环通过网址并显示数据
【发布时间】:2016-04-29 02:04:50
【问题描述】:

我正在使用 Beautiful soup 来抓取此网址 http://www.gbgb.org.uk/resultsRace.aspx?id=1839041 它可以很好地根据需要显示所有字段。但是它只在夹具结果卡上显示一场比赛,我想提取整个比赛会议,卡上的比赛在 9 到 14 场之间变化 这是整个会议的 URL http://www.gbgb.org.uk/resultsMeeting.aspx?id=135488。 有什么方法可以让我循环遍历完整的比赛卡片并在卡片上显示所有比赛的内容。下面是一场比赛的代码。

 from urllib import urlopen

from bs4 import BeautifulSoup
html = urlopen("http://www.gbgb.org.uk/resultsRace.aspx?id=1839041")

bsObj = BeautifulSoup(html)
nameList = bsObj. findAll("div", {"class": "track"})
for name in nameList:
 print(name. get_text())

nameList = bsObj. findAll("div", {"class": "date"})
for name in nameList:
 print(name. get_text())

nameList = bsObj. findAll("div", {"class": "datetime"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("div", {"class": "grade"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("div", {"class": "distance"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("div", {"class": "prizes"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "first essential fin"}) 
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "essential greyhound"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "trap"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "sp"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "timeSec"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "timeDistance"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "essential trainer"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("li", {"class": "first essential comment"})
for name in nameList:
 print(name. get_text())
nameList = bsObj. findAll("div", {"class": "resultsBlockFooter"})
for name in nameList:
 print(name. get_text())

【问题讨论】:

  • 请您将我的答案标记为正确,或评论如何改进它。

标签: python web web-scraping beautifulsoup


【解决方案1】:

您只需要遍历结果块。标签略有不同,但本质上是一样的。我在 Chrome 中使用检查元素功能,使 HTML 抓取变得容易。

from urllib import urlopen

from bs4 import BeautifulSoup
baseURL = 'http://www.gbgb.org.uk/resultsMeeting.aspx?id=135488'
html = urlopen(baseURL)
bsObj = BeautifulSoup(html, 'lxml')
nameList = bsObj.findAll("div", {"class": "resultsBlock"})
for i in nameList:
    # just the trap info, the rest is similar
    nameList2 = i.findAll("li", {"class": "trap"})
    for j in nameList2:
        print(j.get_text())

【讨论】:

  • 您好,ncfirth,非常感谢您的回复。不过我遇到了一些问题。按照您的指示设法将所有字段都拿出来,但循环很少超出第 7 场或第 8 场比赛会议(大多数会议 12 或 14 场比赛),偶尔在最后一场比赛中,而不是 6 只狗,实际上只有 4 或 5 只狗实际显示。此外,在一些会议上,我不断收到“进程完成,退出代码 0”。有时如果我刷新它会消失,但在某些情况下,我没有尝试在第 7 或第 8 场比赛的源代码中寻找特殊性,但一切似乎都一样..
  • 如果您还有关于使用 bs4 的问题,那么我建议您打开另一个问题,似乎错误可能比这个问题更复杂。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2019-05-05
  • 2020-09-28
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多