BS4 获取 python 中元素的完整 CSS 选择器答案

【问题标题】：BS4 get the full CSS selector for an element in pythonBS4 获取 python 中元素的完整 CSS 选择器
【发布时间】：2020-12-12 07:57:30
【问题描述】：

我正在使用 python 编写一个网络爬虫，我想在其中使用 BS4 获取一些元素。我想获取元素的完整 CSS 选择器链接

page_r = requests.get(page_url)
page_soup = BeautifulSoup(page_r.content, 'html.parser')
elements = page_soup.find('body').find_all()
for element in elements:
    print("CSS selector for this element")
    # here I want to print full CCS selector like body>section>div:nth-of-type(3)>p:nth-of-type(4)

我如何使用 BS4 或可以做到这一点的函数来获得它？

【问题讨论】：

标签： python web-scraping beautifulsoup css-selectors

【解决方案1】：

您可以尝试使用此代码打印页面元素的 CSS 选择器：

from bs4 import BeautifulSoup


def nth_of_type(elem):
    count, curr = 0, 0
    for i, e in enumerate(elem.find_parent().find_all(recursive=False), 1):
        if e.name == elem.name:
            count += 1
        if e == elem:
            curr = i
    return '' if count == 1 else ':nth-of-type({})'.format(curr)

def get_css_selector(elem):
    rv = [elem.name + nth_of_type(elem)]
    while True:
        elem = elem.find_parent()
        if not elem or elem.name == '[document]':
            return ' > '.join(rv[::-1])
        rv.append(elem.name + nth_of_type(elem))


sample_html = '''
<html>
    <body>
        <div>Hello</div>
        <div>World</div>
        <div>
            <ul>
                <li>1</li>
                <li>2</li>
                <li>3</li>
            </ul>
        </div>
    </body>
</html>
'''

page_soup = BeautifulSoup(sample_html, 'html.parser')
elements = page_soup.find('body').find_all()
for element in elements:
    css_selector = get_css_selector(element)
    txt = page_soup.select_one(css_selector)
    print(css_selector)
    print(txt)
    print('-' * 80)

打印：

html > body > div:nth-of-type(1)
<div>Hello</div>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(2)
<div>World</div>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(3)
<div>
<ul>
<li>1</li>
<li>2</li>
<li>3</li>
</ul>
</div>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(3) > ul
<ul>
<li>1</li>
<li>2</li>
<li>3</li>
</ul>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(3) > ul > li:nth-of-type(1)
<li>1</li>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(3) > ul > li:nth-of-type(2)
<li>2</li>
--------------------------------------------------------------------------------
html > body > div:nth-of-type(3) > ul > li:nth-of-type(3)
<li>3</li>
--------------------------------------------------------------------------------

【讨论】：