【问题标题】:BeautifulSoup soup.select cutting off child tagsBeautifulSoup soup.select 切断子标签
【发布时间】:2019-08-07 16:38:44
【问题描述】:

在运行脚本以检索所有带有“FlatParagraph”类的块引用标签时,我似乎切断了块引用标签中的一些子标签。是否有包含所有子标签的查询?问题似乎与<blockquote><i><a>text<a/><i/> 一组标签有关。所以不是所有孩子的问题。

我正在使用以下代码

import urllib


from urllib.request import urlopen
from bs4 import BeautifulSoup

fhand = urllib.request.urlopen('https://www.legislation.qld.gov.au/view/whole/html/2018-07-01/sl-2006-0200').read()

soup = BeautifulSoup(fhand, 'html.parser')
fp = soup.select('blockquote[class="FlatParagraph"]')
for i in fp: 
    print(i.text)
    print('---------')

然后我使用 for 循环从每一行检索文本

changedfplist = list()
for i in fp:
    changedfplist.append(i.text.replace(u'\xa0', ' ').encode('utf-8'))

这是我正在解析的示例 -

<blockquote class="FlatParagraph"><blockquote class="Paragraph"><span class="ListNumber">(1)</span>This section applies if—<blockquote class="Paragraph List"><span class="ListNumber">(a)</span>before the commencement—<blockquote class="Paragraph List"><span class="ListNumber">(i)</span>a person applied under <a href="#sec.28">section&nbsp;28</a>(1) of the repealed regulation for approval of a proposed fire engineering design brief for stated building work; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(ii)</span>an authorised representative of the service attended a former fire engineering brief meeting relating to the approval of the proposed fire engineering design brief; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(iii)</span>the service had not decided whether or not to approve the proposed fire engineering design brief; and</blockquote>
</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(b)</span>the person has not paid the former fire engineering design brief meeting fee for the attendance of the representative of the service at the former fire engineering brief meeting.</blockquote>
</blockquote><blockquote class="Paragraph"><span class="ListNumber">(2)</span>For assessing the fire engineering design brief for the stated building work—<blockquote class="Paragraph List"><span class="ListNumber">(a)</span><a href="#sec.61">section&nbsp;61</a> applies as if the reference to a fire engineering brief were a reference to the proposed fire engineering design brief; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(b)</span><a href="#sec.62">section&nbsp;62</a>(1)(d) applies as if the reference to each fire engineering brief meeting included a reference to each former fire engineering brief meeting; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(c)</span><a href="#sch.2">schedule&nbsp;2</a>, <a href="#sch.2-pt.3">part&nbsp;3</a>, item 3 applies as if a reference to a meeting included a reference to a former fire engineering brief meeting.</blockquote>
</blockquote><blockquote class="Paragraph"><span class="ListNumber">(3)</span>In this section—<blockquote class="Paragraph-No-Number"><b><i><a name="sec.90-ssec.3-def.formerfireengineeringbriefmeeting"></a>former fire engineering brief meeting</i></b> means a fire engineering brief meeting under <a href="#sec.28">section&nbsp;28</a>(2)(d) of the repealed regulation.</blockquote><blockquote class="Paragraph-No-Number"><b><i><a name="sec.90-ssec.3-def.formerfireengineeringdesignbriefmeetingfee"></a>former fire engineering design brief meeting fee</i></b> means the fire engineering design brief meeting fee stated in <a href="#sch.3">schedule&nbsp;3</a> of the repealed regulation.</blockquote></blockquote></blockquote>

当我解析这个时,我得到了

(1)本条适用于以下情况 - (a) 开始之前 - (i) 一个人 根据已废除规例第 28(1) 条申请批准 拟定的建筑工程消防工程设计简介;和

(ii)该服务的授权代表参加了一次前火灾 与批准提议有关的工程简报会 消防工程设计简介;和

(iii) 服务尚未决定是否批准 拟议的消防工程设计简介;和

(b)该人未支付以前的消防工程设计简报 服务代表出席会议费用 前消防工程简短会议。

(2)用于评估规定的消防工程设计简介 建筑工程 - (a) 第 61 条适用,如同提及火灾一样 工程简介是对拟议消防工程的参考 设计简介;和

(b)第 62(1)(d) 条适用,如同对每场火灾的提及 工程简要会议包括对每一次火灾的参考 工程简会;和

(c)附表 2,第 3 部分,第 3 项适用,如同提及会议一样 包括对前一次消防工程简短会议的参考。

(3)本节-前消防工程简会

最后一行的末尾缺少文本。它已在

处被切断
<blockquote class="Paragraph-No-Number"><b><i><a name="sec.90-ssec.3-def.formerfireengineeringbriefmeeting"></a>former fire engineering brief meeting</i></b> 

更新 - 我试图避免使用一个类,因此使用 .FlatParagraph 没有奏效。我试图避免 class=FlatParagraph view-history-note。 FlatParagraph view-history-note 是 Fl​​atParagraph 类标签的子标签的一个类。

我已经用 lxml 和 html.parser 尝试了上面的代码,我得到了 lxml 的所有文本,以及 html.parser 的截断文本。如果有人知道为什么,我很想听听!

【问题讨论】:

  • 完成,抱歉,发帖太早了
  • 您使用的是什么解析器?我试过soup1 = soup(html, 'lxml'),听起来不错
  • 截断文本的类名与其他类名不同。您正在搜索 FlatParagraph,而您的缺失测试中的类是 Paragraph-No-Number。看看stackoverflow.com/questions/18725760/…
  • @PRMoureu,我正在使用 'html-parser' fhand1 = urllib.urlopen(link).read() soup1 = BeautifulSoup(fhand1, 'html.parser')
  • 我也可以使用print(BeautifulSoup(html, 'html.parser').text) 获取所有文本。也许你应该完成你正在使用的代码

标签: python python-3.x beautifulsoup


【解决方案1】:

您可以使用select()find() 查看下面的代码,我正在获取全文!

html = '''
<blockquote class="FlatParagraph"><blockquote class="Paragraph"><span class="ListNumber">(1)</span>This section applies if—<blockquote class="Paragraph List"><span class="ListNumber">(a)</span>before the commencement—<blockquote class="Paragraph List"><span class="ListNumber">(i)</span>a person applied under <a href="#sec.28">section&nbsp;28</a>(1) of the repealed regulation for approval of a proposed fire engineering design brief for stated building work; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(ii)</span>an authorised representative of the service attended a former fire engineering brief meeting relating to the approval of the proposed fire engineering design brief; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(iii)</span>the service had not decided whether or not to approve the proposed fire engineering design brief; and</blockquote>
</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(b)</span>the person has not paid the former fire engineering design brief meeting fee for the attendance of the representative of the service at the former fire engineering brief meeting.</blockquote>
</blockquote><blockquote class="Paragraph"><span class="ListNumber">(2)</span>For assessing the fire engineering design brief for the stated building work—<blockquote class="Paragraph List"><span class="ListNumber">(a)</span><a href="#sec.61">section&nbsp;61</a> applies as if the reference to a fire engineering brief were a reference to the proposed fire engineering design brief; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(b)</span><a href="#sec.62">section&nbsp;62</a>(1)(d) applies as if the reference to each fire engineering brief meeting included a reference to each former fire engineering brief meeting; and</blockquote>
<blockquote class="Paragraph List"><span class="ListNumber">(c)</span><a href="#sch.2">schedule&nbsp;2</a>, <a href="#sch.2-pt.3">part&nbsp;3</a>, item 3 applies as if a reference to a meeting included a reference to a former fire engineering brief meeting.</blockquote>
</blockquote><blockquote class="Paragraph"><span class="ListNumber">(3)</span>In this section—<blockquote class="Paragraph-No-Number"><b><i><a name="sec.90-ssec.3-def.formerfireengineeringbriefmeeting"></a>former fire engineering brief meeting</i></b> means a fire engineering brief meeting under <a href="#sec.28">section&nbsp;28</a>(2)(d) of the repealed regulation.</blockquote><blockquote class="Paragraph-No-Number"><b><i><a name="sec.90-ssec.3-def.formerfireengineeringdesignbriefmeetingfee"></a>former fire engineering design brief meeting fee</i></b> means the fire engineering design brief meeting fee stated in <a href="#sch.3">schedule&nbsp;3</a> of the repealed regulation.</blockquote></blockquote></blockquote>
'''
soup = BeautifulSoup(html,'lxml')
fp = soup.select('.FlatParagraph')
for i in fp:
    print(i.text)

fp = soup.find('blockquote',attrs={'class':'FlatParagraph'})
print(fp.text)

输出:

(1)This section applies if—(a)before the commencement—(i)a person applied under section 28(1) of the repealed regulation for approval of a proposed fire engineering design brief for stated building work; and
(ii)an authorised representative of the service attended a former fire engineering brief meeting relating to the approval of the proposed fire engineering design brief; and
(iii)the service had not decided whether or not to approve the proposed fire engineering design brief; and

(b)the person has not paid the former fire engineering design brief meeting fee for the attendance of the representative of the service at the former fire engineering brief meeting.
(2)For assessing the fire engineering design brief for the stated building work—(a)section 61 applies as if the reference to a fire engineering brief were a reference to the proposed fire engineering design brief; and
(b)section 62(1)(d) applies as if the reference to each fire engineering brief meeting included a reference to each former fire engineering brief meeting; and
(c)schedule 2, part 3, item 3 applies as if a reference to a meeting included a reference to a former fire engineering brief meeting.
(3)In this section—former fire engineering brief meeting means a fire engineering brief meeting under section 28(2)(d) of the repealed regulation.former fire engineering design brief meeting fee means the fire engineering design brief meeting fee stated in schedule 3 of the repealed regulation.

【讨论】:

  • 我目前正在使用 select() 并且它不起作用,如上所述,并且 find() 只返回第一项,而不是所有 Flat Paragraph 的列表。我还想避免另一个标签(FlatParagraph view-history-note)。我已经尝试使用 find_all 并且因为我想避免使用上面的标签,所以我在列表中附加了太多的项目。我将在上面添加更多的 cmets。
  • 您的代码有问题,请出示您的完整代码!
  • 我认为是解析器,我用'lxml'尝试了上面的方法,它可以工作
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2018-11-07
  • 2020-01-23
  • 2011-08-27
  • 1970-01-01
  • 1970-01-01
  • 2020-02-16
  • 2012-12-05
相关资源
最近更新 更多