获取 HTML 的最终结果文本

Question

我有以下字符串：

html = '<style>li { list-style-type: lower-alpha; }</style> <ol><li>hello</li></ol>'

是否有任何 Python 库可以将其转换为以下字符串？

'a. hello'

---

编辑：这需要适用于任意 HTML/CSS（因此使用ol 标签的type 属性、li 标签的value attr、CSS 的content、counters , 可能还有数百个用于列表和其他东西的 HTML/CSS 模式）。

编辑 2：我尝试了 Lynx，如果不是因为 Lynx 显然无法处理 list-style-type 和其他常见的 CSS 问题，那实际上会起作用。

Answer 1

据我所知，没有一个库，但是您仍然可以使用 beautifulsoup 和内置的字符串。这是一种边缘情况，因为只有这么多不同的列表样式类型。

您正在解析的字符串是结构化的，如果存在这些特殊的列表样式类型，它会告诉您它们在样式标签中的类型。

使用 BeautifulSoup 和内置字符串

from bs4 import BeautifulSoup
import string


html = """
<style>li { list-style-type: lower-alpha; }</style> <ol><li>hello</li><li>hi</li><li>Hey</li></ol>
"""

soup = BeautifulSoup(html, "lxml")

# Some obscure hidden tag
hidden_tag = "list-style-type:"

# The style that was listed if there are any
style = None
# First parse the style tags
for s in soup.find_all("style"):
    # If the style tag is for a list, and the hidden tag type is there
    if "li" in s.text and hidden_tag in s.text:
        text = s.text
        # Grab whatever is from the end of the hidden tag, to the first instance of ";" 
        # and strip white space
        style = text[text.find(hidden_tag) + len(hidden_tag):text.find(";")].strip()
# Create a list of the different style types you could encounter.
# For this, I just used lower/upper-alpha but you could use ascii character codes to represent many other styles

style_sets = {
    "lower-alpha": list(string.ascii_lowercase),
    "upper-alpha": list(string.ascii_uppercase),
    # Etc. for whatever possible styles you might encounter
}

# Iterate through the soup as you normally would using bs4.
for ol in soup.find_all("ol"):
    ind = 0
    for li in ol.find_all("li"):
        # If the style type is found we print it using that style type
        if style is not None:
            print("{}. {}".format(style_sets[style][ind], li.text))
            ind += 1
        else:
            print(li.text)

这会给你最终的字符串：

a. hello
b. hi
c. Hey
a. hello2
b. hi2
c. Hey2

Answer 2

您可以通过使用像Selenium Webdriver 这样的无头浏览器来做到这一点，因为我们需要使用Window.getComputedStyle() 来查看哪些ol li 项目具有lower-alpha 的lower-alpha 值list-style-type。无法获取列表项算术/alpha 索引的文本。

我们可以根据 CSS 和 HTML 参数生成这些数字。 HTML 列表可能会变得非常复杂，因为它们可能包含超过 26 个项目，其中字母必须是 aa.、ab. 等。还有 start 和 reversed ol attributes。 start 定义订单从哪里开始，例如对于<ol start="3">，计数将从字母c 开始。 reversed 属性以逆序显示列表计数 c.、b.、a. 等。我们需要解决这两种情况。

使用 Chrome Webdriver 安装 Selenium 的说明

1. 使用pip 安装 Selenium：

   pip3 install selenium
   

2. 下载 Chrome Webdriver here 并添加到您的系统 PATH。请注意选择本地 Chrome 安装的版本。最新的 Chrome 是80.0.3987。

用于抓取有序列表项并生成 lower-alpha 计数器的 Python 脚本

在此脚本中，我使用了一个实时 URL，但您可以查看脚本底部，我解释了如何使用 'data:text/html;' 向 webdriver 传递一些自定义 HTML。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import string


def get_content(link):
    driver.get(link)

    # Get all page ordered lists
    for ol in WebDriverWait(driver,5).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, "ol"))):
        # Get all items from the current ordered list
        list_items = ol.find_elements_by_css_selector("li")
        list_items_count = len(list_items)

        # Get the list start attribute, will return 1 if not present
        ol_start = int(ol.get_attribute("start"))

        # Get the list reversed attribute, will return None if not present
        ol_reversed = ol.get_attribute("reversed")

        # Print information about the ordered list
        print("OL with %s items starting at %s, reversed: %s" % (
            len(list_items), 
            ol_start, 
            "yes" if ol_reversed else "no"))

        # Counter for the letters.
        # If the list is reversed begin count from the last item to the first,
        # else count from first (start) to last
        li_letter = list_items_count if ol_reversed else ol_start

        # Keep count how many list items found with lower-alpha list-style-type
        list_items_found = 0

        for li in list_items:
            # Execute javascript getComputedStyle to get the list item computed style
            list_style_type = driver.execute_script("return window.getComputedStyle(arguments[0])['list-style-type']", li)

            # If the list item computed style 'list-style-type' has 'lower-alpha' value
            if list_style_type == "lower-alpha":
                # Print generated alpha counter and the item text
                print("%s. %s" % (get_alpha_num(li_letter), li.text))

                # If the list is reversed, decrease letter by 1, else increase it
                li_letter += -1 if ol_reversed else 1

                # Keep counting how many items found with 'lower-alpha'
                list_items_found += 1

        # If no items with 'lower-alpha' found do something
        if list_items_found == 0:
            print("No list items found with 'lower-alpha' list style type")
        print()

# Function to convert numbers to letters 1 => a, 26 => aa
def get_alpha_num(num):
    letters = string.ascii_lowercase
    letters_count = len(letters)
    result = ''
    cnum = num - 1

    while(cnum // letters_count > 0):
        cnum //= letters_count
        result += list(letters)[cnum - 1]

    result += list(letters)[((num - 1) % letters_count)]
    return result

if __name__ == '__main__':
    URL = 'https://zikro.gr/dbg/html/lists.html'

    # If you want to parse HTML code from a string
    # then you can use a 'data:text/html;' URL with the HTML contents like this:
    # 
    # html_content = '<style>li { list-style-type: lower-alpha; }</style> <ol><li>hello</li><li>there</li></ol>'
    # URL = "data:text/html;charset=utf-8,{html_content}".format(html_content=html_content)
    # 
    # Will result to this:
    #  OL with 2 items starting at 1, reversed: no
    #  a. hello
    #  b. there

    chrome_options = Options()

    # Make headless
    # chrome_options.add_argument("--headless")

    with webdriver.Chrome(options=chrome_options) as driver:
       get_content(URL)

结果

如果您尝试解析具有多个列表的页面，例如this one：

#ol-a-css li {
  list-style-type: lower-alpha;
}

<ul>
  <li>Unordered list item 1</li>
  <li>Unordered list item 2</li>
  <li>Unordered list item 3</li>
  <li>Unordered list item 4</li>
</ul>

<ol>
  <li>Simple ordered list item 1</li>
  <li>Simple ordered list item 2</li>
  <li>Simple ordered list item 3</li>
  <li>Simple ordered list item 4</li>
</ol>

<ol type="a">
  <li>Lower alpha, ordered list item 1</li>
  <li>Lower alpha, ordered list item 2</li>
  <li>Lower alpha, ordered list item 3</li>
  <li>Lower alpha, ordered list item 4</li>
</ol>

<ol type="a" start="3">
  <li>Lower alpha start=3, ordered list item 1</li>
  <li>Lower alpha start=3, ordered list item 2</li>
  <li>Lower alpha start=3, ordered list item 3</li>
  <li>Lower alpha start=3, ordered list item 4</li>
</ol>

<ol type="a" reversed>
  <li>Lower alpha reversed, ordered list item 1</li>
  <li>Lower alpha reversed, ordered list item 2</li>
  <li>Lower alpha reversed, ordered list item 3</li>
  <li>Lower alpha reversed, ordered list item 4</li>
</ol>

<ol id="ol-a-css">
  <li>Lower alpha CSS, ordered list item 1</li>
  <li>Lower alpha CSS, ordered list item 2</li>
  <li>Lower alpha CSS, ordered list item 3</li>
  <li>Lower alpha CSS, ordered list item 4</li>
  <li>Lower alpha CSS, ordered list item 5</li>
  <li>Lower alpha CSS, ordered list item 6</li>
  <li>Lower alpha CSS, ordered list item 7</li>
  <li>Lower alpha CSS, ordered list item 8</li>
  <li>Lower alpha CSS, ordered list item 9</li>
  <li>Lower alpha CSS, ordered list item 10</li>
  <li>Lower alpha CSS, ordered list item 11</li>
  <li>Lower alpha CSS, ordered list item 12</li>
  <li>Lower alpha CSS, ordered list item 13</li>
  <li>Lower alpha CSS, ordered list item 14</li>
  <li>Lower alpha CSS, ordered list item 15</li>
  <li>Lower alpha CSS, ordered list item 16</li>
  <li>Lower alpha CSS, ordered list item 17</li>
  <li>Lower alpha CSS, ordered list item 18</li>
  <li>Lower alpha CSS, ordered list item 19</li>
  <li>Lower alpha CSS, ordered list item 20</li>
  <li>Lower alpha CSS, ordered list item 21</li>
  <li>Lower alpha CSS, ordered list item 22</li>
  <li>Lower alpha CSS, ordered list item 23</li>
  <li>Lower alpha CSS, ordered list item 24</li>
  <li>Lower alpha CSS, ordered list item 25</li>
  <li>Lower alpha CSS, ordered list item 26</li>
  <li>Lower alpha CSS, ordered list item 27</li>
  <li>Lower alpha CSS, ordered list item 28</li>
  <li>Lower alpha CSS, ordered list item 29</li>
  <li>Lower alpha CSS, ordered list item 30</li>
  <li>Lower alpha CSS, ordered list item 31</li>
</ol>

你会得到这样的结果：

OL with 4 items starting at 1, reversed: no
No list items found with 'lower-alpha' list style type

OL with 4 items starting at 1, reversed: no
a. Lower alpha, ordered list item 1
b. Lower alpha, ordered list item 2
c. Lower alpha, ordered list item 3
d. Lower alpha, ordered list item 4

OL with 4 items starting at 3, reversed: no
c. Lower alpha start=3, ordered list item 1
d. Lower alpha start=3, ordered list item 2
e. Lower alpha start=3, ordered list item 3
f. Lower alpha start=3, ordered list item 4

OL with 4 items starting at 1, reversed: yes
d. Lower alpha reversed, ordered list item 1
c. Lower alpha reversed, ordered list item 2
b. Lower alpha reversed, ordered list item 3
a. Lower alpha reversed, ordered list item 4

OL with 31 items starting at 1, reversed: no
a. Lower alpha CSS, ordered list item 1
b. Lower alpha CSS, ordered list item 2
c. Lower alpha CSS, ordered list item 3
d. Lower alpha CSS, ordered list item 4
e. Lower alpha CSS, ordered list item 5
f. Lower alpha CSS, ordered list item 6
g. Lower alpha CSS, ordered list item 7
h. Lower alpha CSS, ordered list item 8
i. Lower alpha CSS, ordered list item 9
j. Lower alpha CSS, ordered list item 10
k. Lower alpha CSS, ordered list item 11
l. Lower alpha CSS, ordered list item 12
m. Lower alpha CSS, ordered list item 13
n. Lower alpha CSS, ordered list item 14
o. Lower alpha CSS, ordered list item 15
p. Lower alpha CSS, ordered list item 16
q. Lower alpha CSS, ordered list item 17
r. Lower alpha CSS, ordered list item 18
s. Lower alpha CSS, ordered list item 19
t. Lower alpha CSS, ordered list item 20
u. Lower alpha CSS, ordered list item 21
v. Lower alpha CSS, ordered list item 22
w. Lower alpha CSS, ordered list item 23
x. Lower alpha CSS, ordered list item 24
y. Lower alpha CSS, ordered list item 25
z. Lower alpha CSS, ordered list item 26
aa. Lower alpha CSS, ordered list item 27
ab. Lower alpha CSS, ordered list item 28
ac. Lower alpha CSS, ordered list item 29
ad. Lower alpha CSS, ordered list item 30
ae. Lower alpha CSS, ordered list item 31