【问题标题】:BeautifulSoup: get elements that have a certain attribute, independent of its valueBeautifulSoup:获取具有某个属性的元素,与它的值无关
【发布时间】:2014-06-24 16:50:17
【问题描述】:
假设我有以下 html:
<div id='0'>
stuff here
</div>
<div id='1'>
stuff here
</div>
<div id='2'>
stuff here
</div>
<div id='3'>
stuff here
</div>
是否有一种简单的方法可以提取所有具有id 属性的div,而与使用BeautifulSoup 的值无关?我意识到使用 xpath 执行此操作很简单,但似乎无法在 BeautifulSoup 中执行 xpath 搜索。
【问题讨论】:
标签:
python
parsing
xpath
html-parsing
beautifulsoup
【解决方案1】:
使用id=True 仅匹配具有属性集的元素:
soup.find_all('div', id=True)
反之亦然;您可以使用id 属性排除标签:
soup.find_all('div', id=False):
要查找具有给定属性的标签,您还可以使用CSS selectors:
soup.select('div[id]'):
但不幸的是,这不支持搜索逆运算所需的运算符。
演示:
>>> from bs4 import BeautifulSoup
>>> sample = '''\
... <div id="id1">This has an id</div>
... <div>This has none</div>
... <div id="id2">This one has an id too</div>
... <div>But this one has no clue (or id)</div>
... '''
>>> soup = BeautifulSoup(sample)
>>> soup.find_all('div', id=True)
[<div id="id1">This has an id</div>, <div id="id2">This one has an id too</div>]
>>> soup.find_all('div', id=False)
[<div>This has none</div>, <div>But this one has no clue (or id)</div>]
>>> soup.select('div[id]')
[<div id="id1">This has an id</div>, <div id="id2">This one has an id too</div>]
【解决方案2】:
BeautifulSoup4 支持commonly-used css selectors。
>>> import bs4
>>>
>>> soup = bs4.BeautifulSoup('''
... <div id="0"> this </div>
... <div> not this </div>
... <div id="2"> this too </div>
... ''')
>>> soup.select('div[id]')
[<div id="0"> this </div>, <div id="2"> this too </div>]