如何使用beautifulsoup获取没有任何类或id的特定ul元素li的文本和href答案

【问题标题】：How to get specific ul element li's text and href having no any class or id using beautifulsoup如何使用beautifulsoup获取没有任何类或id的特定ul元素li的文本和href
【发布时间】：2018-04-27 12:18:43
【问题描述】：

我有一个从邮件列表页面报废的 html，如下所示：

<ul>
<li> <b>Messages sorted by:</b>
<a href="thread.html#start">[ thread ]</a>
<a href="author.html#start">[ author ]</a>
<a href="date.html#start">[ date ]</a>
<li><b><a href="https://mail.kde.org/mailman/listinfo/okular-devel">More info on this list...
                    </a></b></li>
</li></ul>, 


<ul>
<li><a href="000006.html">[Okular-devel] "why okular is cool and what's our focus" text
</a><a name="6"> </a>
<i>Albert Astals Cid
</i>
<li><a href="000000.html">[Okular-devel] playground/graphics/okular
</a><a name="0"> </a>
<i>Tobias Koenig
</i>
<li><a href="000001.html">[Okular-devel] playground/graphics/okular
</a><a name="1"> </a>
<i>Tobias Koenig
</i>
<li><a href="000004.html">[Okular-devel] Rotation &amp; object rects
</a><a name="4"> </a>
<i>Pino Toscano
</i>
<li><a href="000005.html">[Okular-devel] Rotation &amp; object rects
</a><a name="5"> </a>
<i>Albert Astals Cid
</i>
<li><a href="000002.html">[Okular-devel] Slow painting on QImage
</a><a name="2"> </a>
<i>Tobias Koenig
</i>
<li><a href="000003.html">[Okular-devel] Slow painting on QImage
</a><a name="3"> </a>
<i>Albert Astals Cid
</i>
</li></li></li></li></li></li></li></ul>, 


<ul>
<li> <b>Messages sorted by:</b>
<a href="thread.html#start">[ thread ]</a>
<a href="author.html#start">[ author ]</a>
<a href="date.html#start">[ date ]</a>
<li><b><a href="https://mail.kde.org/mailman/listinfo/okular-devel">More info on this list...
                    </a></b></li>
</li></ul>

您可以看到三个 <ul> 元素中包含 li 元素，我只想获取第二个 <ul> 元素的 li 元素，其中 <LI> 为大写，输出应如下所示：

[Okular-devel] "why okular is cool and what's our focus" text - 000006.html
[Okular-devel] playground/graphics/okular - 000000.html
[Okular-devel] playground/graphics/okular - 000001.html
[Okular-devel] Rotation & object rects - 000004.html
and so on...

格式是<LI> 元素的文本和关联的<href> 链接。我的代码给出了所有 <ul> 元素的 li 并且输出重复了 2-3 次，我无法连同它们一起提取 href -
我的代码：

for ele in soup.find_all('ul'):
    for litag in ele.find_all('li'):
        for link in litag.find_all('href'):
            print(litag.text + '-' + link)

它没有给我想要的输出。我该怎么办？

【问题讨论】：

标签： python html web-scraping beautifulsoup

【解决方案1】：

从您提供的 html 中解析。

from bs4 import BeautifulSoup
soup = BeautifulSoup(s, "html.parser")

for el in soup.find_all('ul'):
    for i in el.find_all("li"):
        if i.find("li"):
            print(i.li.a.text.strip(), "---", i.li.a['href'].strip())

输出：

More info on this list... --- https://mail.kde.org/mailman/listinfo/okular-devel
[Okular-devel] playground/graphics/okular --- 000000.html
[Okular-devel] playground/graphics/okular --- 000001.html
[Okular-devel] Rotation & object rects --- 000004.html
[Okular-devel] Rotation & object rects --- 000005.html
[Okular-devel] Slow painting on QImage --- 000002.html
[Okular-devel] Slow painting on QImage --- 000003.html
More info on this list... --- https://mail.kde.org/mailman/listinfo/okular-devel

【讨论】：

【解决方案2】：

你需要对锚标签进行查找：

soup = BeautifulSoup(html, "html.parser")
ele = soup.find_all('ul')[1]     # use only the 2nd one

for litag in ele.find_all('li'):
    for link in litag.find_all('a', href=True):
        print('{} - {}'.format(link.get_text(strip=True), link['href']))

给你：

[Okular-devel] "why okular is cool and what's our focus" text - 000006.html
[Okular-devel] playground/graphics/okular - 000000.html
[Okular-devel] playground/graphics/okular - 000001.html
[Okular-devel] Rotation & object rects - 000004.html
[Okular-devel] Rotation & object rects - 000005.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] playground/graphics/okular - 000000.html
[Okular-devel] playground/graphics/okular - 000001.html
[Okular-devel] Rotation & object rects - 000004.html
[Okular-devel] Rotation & object rects - 000005.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] playground/graphics/okular - 000001.html
[Okular-devel] Rotation & object rects - 000004.html
[Okular-devel] Rotation & object rects - 000005.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] Rotation & object rects - 000004.html
[Okular-devel] Rotation & object rects - 000005.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] Rotation & object rects - 000005.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] Slow painting on QImage - 000002.html
[Okular-devel] Slow painting on QImage - 000003.html
[Okular-devel] Slow painting on QImage - 000003.html

添加href=True 确保只返回包含href 的标签。

【讨论】：