使用 Selenium Python 将数据收集为元组答案

【问题标题】：Collecting data as a tuple using Selenium Python使用 Selenium Python 将数据收集为元组
【发布时间】：2017-06-07 03:56:40
【问题描述】：

我在使用 Selenium Python 3.6 将数据收集为元组时遇到了麻烦。这是我要收集数据的页面 (http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I) 我想在页面上部的搜索菜单中收集“制造商（制造商）”数据。

我用selenium webdrivet设置了虚拟页面，用这段代码来收集和选择第一个下拉菜单的列表：

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException

from bs4 import BeautifulSoup
from time import sleep


link = 'http://www.bobaedream.co.kr/cyber/CyberCar.php?gubun=I'
driver = webdriver.PhantomJS()
driver.set_window_size(1920, 1080)
driver.get(link)
sleep(.75)

soup = BeautifulSoup(driver.page_source, "html.parser", from_encoding='utf-8')

manufacturers = [
    ('%s', '%s') % (o.text, o.get_attribute('href'))
    for o
    in driver.find_elements_by_css_selector("#layer_maker ul.list li a")
    if o.text != '전체']

for manufacturer in manufacturers:
    driver.execute_script("o.get_attribute('href')")

而且，这是我收到的错误消息：

/Library/Frameworks/Python.framework/Versions/3.6/bin/python3.6 /Users/chongwonshin/PycharmProjects/Crawler_test/dump.py
/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/bs4/__init__.py:146: UserWarning: You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.
  warnings.warn("You provided Unicode markup but also provided a value for from_encoding. Your from_encoding will be ignored.")
Traceback (most recent call last):
  File "/Users/chongwonshin/PycharmProjects/Crawler_test/dump.py", line 23, in <module>
    in driver.find_elements_by_css_selector("#layer_maker ul.list li a")
  File "/Users/chongwonshin/PycharmProjects/Crawler_test/dump.py", line 24, in <listcomp>
    if o.text != '전체']
TypeError: unsupported operand type(s) for %: 'tuple' and 'tuple'

Process finished with exit code 1

请帮忙。

【问题讨论】：

标签： python html selenium beautifulsoup web-crawler

【解决方案1】：

我想这就是你需要的：

[
('%s' % o.text, '%s' % o.get_attribute('href'))
for o
in driver.find_elements_by_css_selector("#layer_maker ul.list li a")
if o.text != '전체']

或者只是

[
(o.text, o.get_attribute('href'))
for o
in driver.find_elements_by_css_selector("#layer_maker ul.list li a")
if o.text != '전체']

请注意，% 也是 Python 中的“模”运算符，不能应用于元组

【讨论】：