Beautiful Soup 将多个 <div> 和连续的 <p> 标签解析为字典

Question

我有多个内联 div（它们是“标题”）和下面的段落标签（不是在 div 中），理论上是“孩子”...我想将其转换为字典。我无法完全弄清楚最好的方法。网站大致如下：

<div><span>This should be dict key1</span></div>
<p>This should be the value of key1</p>
<p>This should be the value of key1</p>
<div><span>This should be dict key2</span></div>
<p>This should be the value of key2</p>

我使用的 Python 代码如下所示：

soup = bs.BeautifulSoup(source,'lxml')

full_discussion = soup.find(attrs={'class' : 'field field-type-text field-field-discussion'})

ava_discussion = full_discussion.find(attrs = {'class': 'field-item odd'})

for div in ava_discussion.find_all("div"):
    discussion = []

    if div.findNextSibling('p'):
        discussion.append(div.findNextSibling('p').get_text())

    location = div.get_text()

    ava_dict.update({location: {"discussion": discussion}}

但是，问题是这段代码只添加了 FIRST <p> 标记，然后移动到下一个 div。最终，我想我想将每个<p> 添加到discussion 的列表中。救命！

更新：

添加while 循环会产生第一个循环的副本

标签表示存在的数量。代码如下：

for div in ava_discussion.find_all("div"):
    ns = div.nextSibling

    discussion = []

    while ns is not None and ns.name != "div":
        if ns.name == "p":
            discussion.append(div.findNextSibling('p').get_text())
        ns = ns.nextSibling

    location = div.get_text()

    ava_dict.update({location : {"discussion": discussion}})

print(json.dumps(ava_dict, indent=2))

Answer 1

我没有添加正确的文本。此代码有效：

for div in ava_discussion.find_all("div"):
    ns = div.nextSibling

    discussion = []

    while ns is not None and ns.name != "div":
        if ns.name == "p":
            discussion.append(ns.get_text())
        ns = ns.nextSibling

    location = div.get_text()

    ava_dict.update({location : {"discussion": discussion}})

print(json.dumps(ava_dict, indent=2))

Answer 2

这个怎么样？

    paragraphs = div.findNextSiblings('p') 
    for sibling in div.findNextSiblings():
        if sibling in paragraphs:
            discussion.append(sibling.get_text())
        else:
            break

现在，谁能告诉我如何让它更优雅:)