【发布时间】:2018-12-26 15:05:32
【问题描述】:
大家好,我有一个任务是从this link 做一些抓取工作,问题是我正在尝试通过点击进入页面的最终外观,这是包含一些数据的表格,但有时它只需要一个单击,另一方面,它需要多个。在那种情况下,我尝试通过使用 if 语句进行控制,其中我从页面的最终视图 is_enabled() 中说了 if 元素(我也尝试过使用 is_displayed()) 转到打印最终页面视图的函数。请如果你能帮助我,因为我不知道如何解决这个问题了。这是我的代码
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import StaleElementReferenceException
from selenium.common.exceptions import ElementNotInteractableException
from bs4 import BeautifulSoup
import time
import urllib.request
import re
ignored_exceptions=(NoSuchElementException,StaleElementReferenceException)
#Function that will do writing because I don't have these atrb in final page look as data
def write(make,model,body,nesto):
source = driver.page_source
soup = BeautifulSoup(source, "lxml")
table = soup.find('table')
table_rows = table.find_all('tr')
for tr in table_rows:
td = tr.find_all('td')
scrape_list = [make,model,body,td]
print(scrape_list)
def not_normal():
another_back = driver.find_element_by_class_name("btn btn-back pull-left kba_step_back")
another_back.click()
item_type_list = driver.find_element_by_class_name("kba_vehicle_type_group_id")
item = item_type_list.find_element_by_tag_name("li")
make = driver.find_element_by_class_name("make")
model = driver.find_element_by_class_name("model")
body = driver.find_element_by_class_name("body")
item.click()
another_back.click()
write(make.text,model.text,body.text,item.text)
while True:
try:
button_back = wait.until(EC.element_to_be_clickable((By.ID, "back")))
button_back.click()
except TimeoutException:
break
driver = webdriver.Chrome("/home/cutie/Desktop/felgenoutlet/bin/chromedriver")
driver.get("https://www.felgenoutlet.com/en")
wait = WebDriverWait(driver, 3)
#Opens webpage
first_list = driver.find_element_by_class_name("kba_vehicle_trade_group_id")
wait.until(EC.presence_of_all_elements_located((By.TAG_NAME,'li')))
cars = first_list.find_elements_by_tag_name("li")
for car in cars:
car.click()
try:
final_view=driver.find_element_by_class_name("header_entry_list")
if final_view.is_enabled():
not_normal()
continue
except:
print("not found", end='\n')
second_list = driver.find_element_by_class_name("kba_vehicle_model_group_id")
second_list_items= wait.until(EC.presence_of_all_elements_located((By.TAG_NAME,'li')))
second_list_items = second_list.find_elements_by_tag_name("li")
for model in second_list_items:
model.click()
try:
final_view=driver.find_element_by_class_name("header_entry_list")
if final_view.is_enabled():
not_normal()
continue
except:
third_list = driver.find_element_by_class_name("kba_vehicle_body_group_id")
third_list_items= wait.until(EC.presence_of_all_elements_located((By.TAG_NAME,'li')))
third_list_items = third_list.find_elements_by_tag_name("li")
for body in third_list_items:
body.click()
try:
final_view=driver.find_element_by_class_name("header_entry_list")
if final_view.is_enabled():
not_normal()
continue
except:
fourth_list = driver.find_element_by_class_name("kba_vehicle_type_group_id")
fourth_list_items = wait.until(EC.presence_of_all_elements_located((By.TAG_NAME,'li')))
fourth_list_items = fourth_list.find_elements_by_tag_name("li")
for nesto in fourth_list_items:
make = driver.find_element_by_class_name("make").text
model = driver.find_element_by_class_name("model").text
body = driver.find_element_by_class_name("body").text
write(make,model,body,nesto.text)
nesto.click()
这是我有问题的 if 语句:
final_view=driver.find_element_by_class_name("header_entry_list")
if final_view.is_enabled():
not_normal()
continue
任何帮助将不胜感激 更新问题已解决,即将上传代码解决方案
【问题讨论】:
-
not_normal()是什么?为什么需要final_view.is_enabled()? -
not_normal() 是一个函数,当没有显示少于 4 次点击时我调用它,所以我可以点击它们,而 final_view 是包含最终页面元素的 selenium 对象,所以如果它是显示我调用了一个函数 not_normal()
标签: javascript css python-3.x selenium web-scraping