无法提取某个“b”标签之后的所有内容，直到下一个“b”标签答案

【问题标题】：Unable to extract all the content after some "b" tag up to the next "b" tag无法提取某个“b”标签之后的所有内容，直到下一个“b”标签
【发布时间】：2021-09-27 12:24:18
【问题描述】：

我正在尝试从网页中抓取任何b 标记之后的所有可用内容，直到下一个b 标记。为了让大家形象化，我附上相关的html elements。

以下三个是b标签Job Description,This is an internship和Qualifications的内容。所以，当我选择任何b 标签时，我会想在那个特定的b 标签和下一个b 标签之间抓取任何东西。

我试过这样：

import requests
from bs4 import BeautifulSoup
from itertools import takewhile

link = 'https://filebin.varnish-software.com/tgmdhg8dp37tycmk/doc.html'

res = requests.get(link)
soup = BeautifulSoup(res.text,"lxml")
desc = [i.get_text(strip=True) for i in takewhile(lambda tag: tag.name!='b', soup.select("div > b:contains('Job Description') ~ *"))]
print(desc)

我得到的输出：

['This job entails researching developing testing and deploying mechanical solutions.', '', 'Typical activities include:', '', 'Designing and developing thermal or mechanical tooling systems.', '', 'The ideal candidate should exhibit the following behavioral traits:', '', 'Work in a technically diverse environment-Adapt to changing requirements.Verbal and written communication.Project management.', '', 'This is an internship.', 'Qualifications']

我希望得到的输出（踢掉最后两个b标签的内容）：

['This job entails researching developing testing and deploying mechanical solutions.', '', 'Typical activities include:', '', 'Designing and developing thermal or mechanical tooling systems.', '', 'The ideal candidate should exhibit the following behavioral traits:', '', 'Work in a technically diverse environment-Adapt to changing requirements.Verbal and written communication.Project management.']

编辑：

这是另一个 link 供您测试。

【问题讨论】：

标签： python python-3.x web-scraping beautifulsoup

【解决方案1】：

试试这个：

import requests
from bs4 import BeautifulSoup

link = 'https://filebin.varnish-software.com/tgmdhg8dp37tycmk/doc.html'
soup = BeautifulSoup(requests.get(link).text, "lxml")
desc = [
    i.strip() for i in soup.find_all(text=True)
    if i.strip() and i.parent.name != "b"
]
print("\n".join(desc))

输出：

This job entails researching developing testing and deploying mechanical solutions.
Typical activities include:
Designing and developing thermal or mechanical tooling systems.
The ideal candidate should exhibit the following behavioral traits:
Work in a technically diverse environment-Adapt to changing requirements.
Verbal and written communication.
Project management.

【讨论】：

对不起@baduker，你的解决方案很遗憾这次没有成功。请参考我在上面添加的另一个链接。谢谢。

【解决方案2】：

这应该可以工作（最接近的解决方案）：

import requests
from bs4 import BeautifulSoup

link = 'https://filebin.varnish-software.com/vu2hq4qctqw4xd48/document.html'

desc = []
res = requests.get(link)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select_one("div > b:contains('Job Description')").find_all_next():
    if item.name=="b": break
    if not item.find(text=True,recursive=False): continue
    desc.append(item.get_text(strip=True))

print(' '.join(desc))

【讨论】：