【问题标题】:HTML parsing div.p.ol returns blank in PythonHTML 解析 div.p.ol 在 Python 中返回空白
【发布时间】:2020-08-03 22:09:22
【问题描述】:

根据下面的屏幕截图,我试图将“ol”标签作为标题标签,这样我就可以运行一个带有所有“li”标签的 for 循环,以获取“aria”的内容/地址-标签。

但是,我的代码返回空白。任何人都知道我怎样才能让这个“ol”充当标题?非常感谢!

import requests
from bs4 import BeautifulSoup

# website
sitemap = 'https://www.walmart.com/store/finder?location=87321&distance=100'
# content of website
sitemap_content = requests.get(sitemap).content
# parsing website
soup = BeautifulSoup(sitemap_content, 'html.parser')
#print(soup)
header_div = soup.div.ol.li
print(header_div)

screenshot of inspect element

【问题讨论】:

    标签: python html python-3.x beautifulsoup html-lists


    【解决方案1】:

    您在页面上看到的数据以 Json 形式存储在 <script> 元素中。

    import json
    import requests
    from bs4 import BeautifulSoup
    
    
    url = 'https://www.walmart.com/store/finder?location=02468&distance=100'
    soup = BeautifulSoup(requests.get(url).content, 'html.parser')
    data = json.loads(soup.select_one('#storeFinder').string)
    
    # uncomment this to print all data:
    # print(json.dumps(data, indent=4))
    
    # print some data to screen:
    for store in data['storeFinder']['storeFinderCarousel']['stores']:
        print(store['displayName'])
        print(store['address']['address'])
        print(store['address']['postalCode'], store['address']['city'])
        print('-' * 80)
    

    打印:

    Framingham Store
    121 Worcester Rd
    01701 Framingham
    --------------------------------------------------------------------------------
    Walpole Supercenter
    550 Providence Hwy
    02081 Walpole
    --------------------------------------------------------------------------------
    Quincy Store
    301 Falls Blvd
    02169 Quincy
    --------------------------------------------------------------------------------
    
    ...and so on.
    

    【讨论】:

    • 太棒了,谢谢!出于好奇,当您的代码输出所有商店的位置时,为什么检查元素中的
    • @Regression1234 可能该网站通过Javascript动态操作页面内容。总是做print(soup) 看看,来自服务器的原始响应是什么。
    猜你喜欢
    • 2016-07-27
    • 1970-01-01
    • 1970-01-01
    • 2017-04-28
    • 1970-01-01
    • 2022-10-01
    • 1970-01-01
    • 2018-11-07
    • 2016-01-18
    相关资源
    最近更新 更多