通过python的XPath答案

【问题标题】：XPath through python通过python的XPath
【发布时间】：2015-09-17 22:35:35
【问题描述】：

我有以下我一直在使用的 html：

<html>
<body>
<div class="directions" itemprop="instructions">
<h6>Instructions</h6>
<p>Sharpen your <a href="pencil.html" class="crosslink">pencil</a> (or, alternatively, use your pen)</p>
<p>In a large paper sheet, write your name. When the ink thickens slightly, gently open the <a href="envelop.html" class="crosslink">envelop</a> and insert the <a href="letter.html" class="crosslink" >letter</a> inside folded into 3. Set aside.</p>
<p>Use the pen again to <a href="write.html" class="crosslink">write</a> your name and address into the evelope. Include the destination <a href="address.html" class="crosslink">address</a>.</p>
  <p>Seal the envelop and stamp it</p>
<p class="copyright">Instruction courtesy of John Doe</p>
</div>
</body>
  </html>

我期望的结果是得到一个有序的文本元素数组，而不考虑 html 标签。

result=[
'Sharpen your pencil (or, alternatively, use your pen)',
'In a large paper sheet, write your name. When the ink thickens slightly, gently open the envelop and insert the letter inside folded into 3. Set aside',
'Use the pen again to write your name and address into the envelop. Include the destination address',
'Seal the envelop and stamp it'
]

我正在使用 python 来解析 html 并获取我需要的信息。使用 tree.xpath(''//@[itemprop="instructions"]') 我得到了我需要的元素。但我似乎无法以我想要的方式获取信息。

我最近的尝试（仍然失败）如下：

for a in tree.xpath('//*[@itemprop="instructions"]'):
    for i in a.xpath('./p'):
        temptext = ""
        for c in i.xpath('text()'):
            temptext += c
        for c in i.xpath('./a'):
            temptext += c.text
        tempIteration.append(temptext)

为清晰而编辑：

这让我得到了这个不正确的结果（'a' 节点文本的顺序错误）。查看铅笔在元素 1 末尾的方式，而不是在“锐化你的”之后。同样的事情发生在其余的行中。

result=[
'Sharpen your (or, alternatively, use your pen)pencil',
'In a large paper sheet, write your name. When the ink thickens slightly, gently open the and insert the inside folded into 3. Set asideenvelopletter',
'Use the pen again to your name and address into the envelop. Include the destination writeaddress',
'Seal the envelop and stamp it',
'Instruction Courtesy of John Doe'
]

我无法让这个工作，所以任何帮助将不胜感激。

【问题讨论】：

根据我对 python 的有限了解，看起来您正在遍历每个
标签。它怎么会没有得到最后一个元素？您可以根据标签的类别排除标签吗？
如果我做 a.xpath('./*[not(@class="copyright")]'): 我可以避免得到最后一个完美的元素！
哇哦！很高兴你自己解决了。
是的，只是缺少主要问题，即如何更正将“”节点文本添加到结果字符串中的顺序。

标签： python html xpath

【解决方案1】：

您可以使用getchildren() 方法和元素的text 和tail 属性。我从未使用过lxml，但从文档here 中我可以在下面的示例中使用它。

from lxml import etree

html='''<html>
<body>
<div class="directions" itemprop="instructions">
<h6>Instructions</h6>
<p>Sharpen your <a href="pencil.html" class="crosslink">pencil</a> (or, alternatively, use your pen)</p>
<p>In a large paper sheet, write your name. When the ink thickens slightly, gently open the <a href="envelop.html" class="crosslink">envelop</a> and insert the <a href="letter.html" class="crosslink" >letter</a> inside folded into 3. Set aside.</p>
<p>Use the pen again to <a href="write.html" class="crosslink">write</a> your name and address into the evelope. Include the destination <a href="address.html" class="crosslink">address</a>.</p>
  <p>Seal the envelop and stamp it</p>
<p class="copyright">Instruction courtesy of John Doe</p>
</div>
</body>
  </html>'''

tree=etree.HTML(html)
result=[]
for a in tree.xpath('//*[@itemprop="instructions"]'):
    for i in a.xpath('./p'):
        temptext = ""
        temptext += i.text
        for j in i.getchildren():
            temptext += j.text
            temptext += j.tail
        result.append(temptext)

print result

产生

[
'Sharpen your pencil (or, alternatively, use your pen)', 
'In a large paper sheet, write your name. When the ink thickens slightly, gently open the envelop and insert the letter inside folded into 3. Set aside.', 
'Use the pen again to write your name and address into the evelope. Include the destination address.', 
'Seal the envelop and stamp it', 
'Instruction courtesy of John Doe'
]

然后你可以通过result[:-1] 扔掉最后一个

【讨论】：

太棒了！我没有使用 eTree 功能，您的解决方案非常完美！谢谢米格尔！

【解决方案2】：

不确定这是否有帮助，我对 XPATH 的了解有限，但可能是因为您没有关闭 <div class="directions" itemprop="instructions"> 元素吗？

你不应该没有这个吗：

<html>
<body>
    <div class="directions" itemprop="instructions">
        <h6>Instructions</h6>
        <p>Sharpen your <a href="pencil.html" class="crosslink">pencil</a> (or, alternatively, use your pen)</p>
        <p>In a large paper sheet, write your name. When the ink thickens slightly, gently open the <a href="envelop.html" class="crosslink">envelop</a> and insert the <a href="letter.html" class="crosslink" >letter</a> inside folded into 3. Set aside.</p>
        <p>Use the pen again to <a href="write.html" class="crosslink">write</a> your name and address into the evelope. Include the destination <a href="address.html" class="crosslink">address</a>.</p>
        <p>Seal the envelop and stamp it</p>
    </div>
    <p class="copyright">Instruction courtesy of John Doe</p>
</body>
</html>

注意我添加了</div>

希望这会有所帮助:)

【讨论】：

你对
的看法是对的，但它在最后一个
之后。我更新了源 html 以确保正确性。它实际上并没有改变我得到的结果。

看来你得到了（就我得到 XPATH 而言，我不是 100% 确定）具有 html 属性“itemprop”和值“指令”的元素的内部元素，然后您正在遍历带有标签“p”的所有元素，并且您正在提取所有子文本和子“a”标签元素的所有文本。因此，如果您在 <p class="copyright">Instruction courtesy of John Doe</p> 之后关闭 div，那么我认为您应该得到 'Instruction Courtesy of John Doe'，所以以我的拙见，您将获得预期的行为。