使用 Selenium 获取“ul”标签内的文本？

Question

请帮我找到解决方案以获取“ul”标签内的文本。

我想获取以逗号分隔的信息，例如：“含有酶活性 B 族维生素、膳食补充剂、非转基因 LE 认证”

网站链接：https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051

图片：enter image description here

这是 HTML 代码：

<ul>
  <li>Contains Enzymatically Active B-Vitamins
  </li>
  <li>Dietary Supplement
  </li>
  <li>Non-GMO LE Certified
  </li>
</ul>

Answer 1

您始终可以获取所有元素li，从所有这些元素中获取文本并使用", ".join(elements)

---

小例子的代码

text = '''
<ul>
  <li>Contains Enzymatically Active B-Vitamins
  </li>
  <li>Dietary Supplement
  </li>
  <li>Non-GMO LE Certified
  </li>
</ul>'''

import selenium.webdriver

driver = selenium.webdriver.Firefox()

driver.get("data:text/html;charset=utf-8," + text)

elements = driver.find_elements_by_tag_name('li')

elements = [i.text for i in elements]

print(", ".join(elements)) 

Answer 2

from selenium import webdriver
from shutil import which
chrome_path = which('chromedriver.exe')

driver = webdriver.Chrome(executable_path = chrome_path)

li_eliments = driver.find_elements_by_tag_name('li')

elements = []
for e in li_eliments.text:
    elements.append(e)
print(", ".join(elements)) 

Answer 3

应该这样做：

from selenium import webdriver

link = 'https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051'

with webdriver.Chrome() as driver:
    driver.get(link)
    elements = ', '.join([item.text for item in driver.find_elements_by_css_selector("[itemprop='description'] > ul:nth-of-type(1) > li")])
    print(elements)

输出：

Contains Enzymatically Active B-Vitamins, Dietary Supplement, Non-GMO LE Certified 

Answer 4

好吧，Selenium 用于网络自动化，但数据抓取（就像您正在尝试做的那样）更多地用于请求和漂亮的汤。有关于使用 Selenium 的帖子，但使用这些更容易，因此您不必像使用 selenium 一样启动网络浏览器。

r = requests.get("https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051")
soup = BeautifulSoup(r.content, 'html.parser')
list_items = soup.find('div', itemprop="description")
found = str(re.findall(r'itemprop="description"><ul><li>(\D+)', str(list_items)))

这只需一秒钟，而其他方法可能需要更长的时间来加载浏览器并导航到网站以获取此信息。一旦你得到这些信息并使用正则表达式找到合适的标签，你就可以使用正则表达式来清理它只是文本。

newfound = re.sub(r"</li>|[\[']", '', found)
newfound2 = re.sub(r"<li>", ', ', newfound)
stripped = newfound2.split('\\xa0', 1)[0]

itemprop="description"><ul><li> 和\xa0 行都来自查看页面源并在那里找到列表元素。 以下是一些关于正则表达式的信息：https://www.guru99.com/python-regular-expressions-complete-tutorial.html

Answer 5

提取文本，例如含有酶活性 B 族维生素，膳食补充剂，使用 Selenium 和 python 您可以使用以下任一 Locator Strategies：

• 使用CSS_SELECTOR 并打印列表：

  driver.get('https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051')
  print([my_elem.text for my_elem in driver.find_elements_by_css_selector("div[itemprop='description']>ul li")])
  

• 控制台输出：

  ['Contains Enzymatically Active B-Vitamins', 'Dietary Supplement', 'Non-GMO LE Certified ', 'Promotes healthy metabolism of glucose, fat & alcohol', 'Supports the healthy energy production your body needs', 'Encourages healthy organ function, cognitive health & more', 'Helps inhibit potential vitamin B deficiency']
  

• 使用XPATH 并以逗号分隔的字符串打印元素：

  driver.get('https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051')
  print(', '.join([my_elem.text for my_elem in driver.find_elements_by_xpath("//div[@itemprop='description']/ul//li")]))
  

• 控制台输出：

  Contains Enzymatically Active B-Vitamins, Dietary Supplement, Non-GMO LE Certified , Promotes healthy metabolism of glucose, fat & alcohol, Supports the healthy energy production your body needs, Encourages healthy organ function, cognitive health & more, Helps inhibit potential vitamin B deficiency
  

---

提取文本，例如含有酶活性 B 族维生素，膳食补充剂理想情况下，您必须为WebDriverWait 诱导visibility_of_all_elements_located()，您可以使用以下任一Locator Strategies：

• 使用CSS_SELECTOR 并打印列表：

  driver.get('https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051')
  print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div[itemprop='description']>ul li")))])
  

• 控制台输出：

  ['Contains Enzymatically Active B-Vitamins', 'Dietary Supplement', 'Non-GMO LE Certified ', 'Promotes healthy metabolism of glucose, fat & alcohol', 'Supports the healthy energy production your body needs', 'Encourages healthy organ function, cognitive health & more', 'Helps inhibit potential vitamin B deficiency']
  

• 使用XPATH 并以逗号分隔的字符串打印元素：

  driver.get('https://ca.iherb.com/pr/Life-Extension-BioActive-Complete-B-Complex-60-Vegetarian-Capsules/67051')
  print(', '.join([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@itemprop='description']/ul//li")))]))
  

• 控制台输出：

  Contains Enzymatically Active B-Vitamins, Dietary Supplement, Non-GMO LE Certified , Promotes healthy metabolism of glucose, fat & alcohol, Supports the healthy energy production your body needs, Encourages healthy organ function, cognitive health & more, Helps inhibit potential vitamin B deficiency
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC