【发布时间】：2018-02-09 05:36:33
【问题描述】：
所以，假设我在<body> 标签中有这样的页面
<!-- Tag <a> with <img> inside of it -->
<div class="album_item">
<a href="http://www.foo.com/img/1"><img src="http://thumbnail.foo.com/img/1.jpg" /></a>
<a href="http://www.foo.com/img/2"><img src="http://thumbnail.foo.com/img/2.jpg" /></a>
<a href="http://www.foo.com/img/3"><img src="http://thumbnail.foo.com/img/3.jpg" /></a>
<a href="http://www.foo.com/img/4"><img src="http://thumbnail.foo.com/img/4.jpg" /></a>
</div>
<!-- Only tag <img> -->
<div class="album_item">
<img src="http://large.foo.com/img/5.jpg" />
<img src="http://large.foo.com/img/6.jpg" />
</div>
<!-- Combination Of Both Above -->
<div class="album_item">
<a href="http://www.foo.com/img/7"><img src="http://thumbnail.foo.com/img/7.jpg" /></a>
<a href="http://www.foo.com/img/8"><img src="http://thumbnail.foo.com/img/8.jpg" /></a>
<a href="http://www.foo.com/img/9"><img src="http://thumbnail.foo.com/img/9.jpg" /></a>
<a href="http://www.foo.com/img/10"><img src="http://thumbnail.foo.com/img/10.jpg" /></a>
<img src="http://large.foo.com/img/11.jpg" />
<img src="http://large.foo.com/img/12.jpg" />
</div>
我想用下面的代码报废：
import requests
from bs4 import BeautifulSoup as soup
my_url = 'http://www.foo-url.com'
uClient = requests.get(my_url)
page_html = uClient.text
uClient.close()
page_soup = soup(page_html, "html.parser")
#Identify Each Post Group
containers = page_soup.findAll("div",{"class": "album-item"})
data = []
for container in containers:
#Store Each Pictures To An Object
items = container.findAll("a")
for item in items:
#Set The Link Location
link_location = item.attrs['href']
image_item = item.find("img")
#Set The Image Location
img_location = image_item.attrs['src']
data.append((link_location, img_location))
#Just Incase Only Image
imgs = container.findAll("img")
for img in imgs:
link_location = "NoLink"
img_location = img.attrs['src']
data.append((link_location, img_location))
for link_location, img_location in data:
print(link_location + " | " + img_location)
结果，有很多这样的重复：
http://www.foo.com/img/1 | http://thumbnail.foo.com/img/1.jpg
http://www.foo.com/img/2 | http://thumbnail.foo.com/img/2.jpg
http://www.foo.com/img/3 | http://thumbnail.foo.com/img/3.jpg
http://www.foo.com/img/4 | http://thumbnail.foo.com/img/4.jpg
NoLink | http://thumbnail.foo.com/img/1.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/2.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/3.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/4.jpg #duplicate
NoLink | http://large.foo.com/img/5.jpg
NoLink | http://large.foo.com/img/6.jpg
http://www.foo.com/img/7 | http://thumbnail.foo.com/img/7.jpg
http://www.foo.com/img/8 | http://thumbnail.foo.com/img/8.jpg
http://www.foo.com/img/9 | http://thumbnail.foo.com/img/9.jpg
http://www.foo.com/img/10 | http://thumbnail.foo.com/img/10.jpg
NoLink | http://thumbnail.foo.com/img/7.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/8.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/9.jpg #duplicate
NoLink | http://thumbnail.foo.com/img/10.jpg #duplicate
NoLink | http://large.foo.com/img/11.jpg
NoLink | http://large.foo.com/img/12.jpg
我的想法是，检查<div class="album_item">的内部
如果所有孩子都标记 <a> ，则执行 for item in items:
否则，如果所有孩子都标记 <img> ，则执行 for img in imgs:
但是如果两个标签都有呢？
我也不确定如何检查该标签
在第一个<div>
我尝试使用if(container.select("img"))，这应该是错误的，
但该值为 true，因为它检测到标签 <img> 内部的标签 <a>
那么，我应该如何处理呢？
【问题讨论】：
标签： python python-3.x web-scraping beautifulsoup