Scrapy：从返回到 items.py 字典的数组中删除 html

Question

首先，感谢您的所有帮助！

...stackoverflow (& python) 的新手 si 我为使用错误的术语道歉:)

我正在使用 Scrapy 从 html 源中提取数据，它使用 Scrapy 的选择器在其 items.py 中创建 dict 字段：

def parse_item(self, response):

    item = SiennaautoItem() #instatiating dict
    item['attributes'] = response.css('p.attrgroup').extract()

    yield item

这会返回一个包含多个值的数组/列表的字典：

> ['<p class="attrgroup">\n\n\n\n            <span><b>2014 honda odyssey
> touring elite</b></span>\n            <br>\n\n    </p>', '<p
> class="attrgroup">\n\n\n\n            <span>VIN:
> <b>5FNRL5H66EB107700</b></span>\n            <br>\n\n\n\n\n           
> <span>condition: <b>like new</b></span>\n            <br>\n\n\n\n\n   
> <span>cylinders: <b>6 cylinders</b></span>\n            <br>\n\n\n\n\n
> <span>drive: <b>fwd</b></span>\n            <br>\n\n\n\n\n           
> <span>fuel: <b>gas</b></span>\n            <br>\n\n\n\n\n           
> <span>odometer: <b>99000</b></span>\n            <br>\n\n\n\n\n       
> <span>paint color: <b>white</b></span>\n            <br>\n\n\n\n\n    
> <span>size: <b>full-size</b></span>\n            <br>\n\n\n\n\n       
> <span>title status: <b>clean</b></span>\n            <br>\n\n\n\n\n   
> <span>transmission: <b>automatic</b></span>\n           
> <br>\n\n\n\n\n            <span>type: <b>mini-van</b></span>\n        
> <br>\n\n    </p>']

这是渲染的 html：

['\n\n\n\n 2014本田奥德赛 旅游精英\n
\n\n

', '\n\n\n\n VIN: 5FNRL5H66EB107700\n
\n\n\n\n\n
条件：像新的\n
\n\n\n\n\n
气缸：6 个气缸\n
\n\n\n\n\n 驱动器：fwd\n
\n\n\n\n\n
燃料：gas\n
\n\n\n\n\n
里程表：99000\n
\n\n\n\n\n
油漆颜色：白色\n
\n\n\n\n\n
尺寸：全尺寸\n
\n\n\n\n\n
标题状态：干净\n
\n\n\n\n\n
传输：自动\n

\n\n\n\n\n 类型：小型货车\n

\n \n']

我的问题是，如何删除 html 标签以及如何从 span 标签创建键，它们是：

状况、驾驶、里程表等

我希望从 item[attributes] 返回的值创建自己的 dict 值，例如：

项目[里程表] 项目情况] 等等

非常感谢您的帮助，因为我已经坚持了一段时间！

Answer 1

我的 xpath 有点生疏了，但这里有一种不使用 xpath 的方法，只需使用 w3lib 库

from w3lib.html import remove_tags,replace_escape_chars

html_array=['<p class="attrgroup">\n\n\n\n            <span><b>2014 honda odyssey > touring elite</b></span>\n            <br>\n\n    </p>', '<p > class="attrgroup">\n\n\n\n            <span>VIN: > <b>5FNRL5H66EB107700</b></span>\n            <br>\n\n\n\n\n            > <span>condition: <b>like new</b></span>\n            <br>\n\n\n\n\n    > <span>cylinders: <b>6 cylinders</b></span>\n            <br>\n\n\n\n\n > <span>drive: <b>fwd</b></span>\n            <br>\n\n\n\n\n            > <span>fuel: <b>gas</b></span>\n            <br>\n\n\n\n\n            > <span>odometer: <b>99000</b></span>\n            <br>\n\n\n\n\n        > <span>paint color: <b>white</b></span>\n            <br>\n\n\n\n\n     > <span>size: <b>full-size</b></span>\n            <br>\n\n\n\n\n        > <span>title status: <b>clean</b></span>\n            <br>\n\n\n\n\n    > <span>transmission: <b>automatic</b></span>\n            > <br>\n\n\n\n\n            <span>type: <b>mini-van</b></span>\n         > <br>\n\n    </p>']
html=replace_escape_chars(' '.join(list(map(lambda x:remove_tags(x),html_array))))


data={}
for i in html.split('>'):
    splitted_content = list(map(lambda x:x.strip(),i.split(":")))
    if splitted_content[0].replace(':','').strip() in ['condition','cylinders','drive','fuel']: #put in this array the elements you need
        data[splitted_content[0]]=splitted_content[1]

print(data)

输出：

   
{'condition': 'like new', 'cylinders': '6 cylinders', 'drive': 'fwd', 'fuel': 'gas'}