使用 python selenium/Beautifulsoup 从多个页面中提取数据答案

【问题标题】：Extract data from multiple page using python selenium/Beautifulsoup使用 python selenium/Beautifulsoup 从多个页面中提取数据
【发布时间】：2019-10-24 14:48:25
【问题描述】：

这是link，我想从SDS选项卡下提取数据表，大约有 40 页数据，但是在转到（单击下一个按钮链接）下一页后，我无法提取数据，这是代码。

driver = webdriver.Chrome("~/chromedriver.exe")
driver.get('https://www.international-pc.com/product/interfine-629')
soup_seln = BeautifulSoup(driver.page_source, 'lxml')
tables_sds = soup_seln.find("table", {"id": "documentTable-2"}).find("tbody")
tr_sds = tables_sds.find_all("tr")           
tr_sds[0].findAll('td')[0].get_text()                                 
Gives O/P --> 'INTERFINE 629 APPLE GREEN PART A'

点击下一页按钮链接后

driver.find_element_by_xpath("//a[@id='documentTable-2_next']").click()
tr_sds[0].findAll('td')[0].get_text()

但这并没有给我新的表格输出。关于如何完成的任何建议？

【问题讨论】：

您要提取哪些数据？
@DebanjanB SDS 标签下的表（列）数据。

标签： python selenium web-scraping beautifulsoup webdriverwait

【解决方案1】：

要从表格的 SDS 选项卡中提取所需数据，您可以使用以下解决方案：

代码块：

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, StaleElementReferenceException

options = webdriver.ChromeOptions() 
options.add_argument("start-maximized")
options.add_argument('disable-infobars')
driver=webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://www.international-pc.com/product/interfine-629")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='SDS']"))).click()
my_texts = [my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 5).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@class='display dataTable no-footer' and @id='documentTable-2']//tbody//tr//td[position() = 1]")))]
while True:
    try:
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.paginate_button.next#documentTable-2_next:not(.disabled)"))).click()
        time.sleep(3)
        my_new_text_elements = [my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH, "//table[@class='display dataTable no-footer' and @id='documentTable-2']//tbody//tr//td[position()=1]")))]
        my_texts.extend(my_new_text_elements)
    except (TimeoutException, StaleElementReferenceException) as e:
        break
print(my_texts)

控制台输出：

['INTERFINE 629 APPLE GREEN PART A', 'INTERFINE 629 APPLE GREEN PART A', 'INTERFINE 629 B15 MID BLUE PART A', 'INTERFINE 629 B15 MID BLUE PART A', 'INTERFINE 629 B15 MID BLUE PART A', 'INTERFINE 629 B15 MID BLUE PART A', 'INTERFINE 629 B44 LIGHT GREY BLUE PTA', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE LIGHT PART A', 'INTERFINE 629 BASE MEDIUM PART A', 'INTERFINE 629 BASE MEDIUM PART A', 'INTERFINE 629 BASE MEDIUM PART A', 'INTERFINE 629 BASE MEDIUM PART A', 'INTERFINE 629 BASE RED PART A', 'INTERFINE 629 BASE RED PART A', 'INTERFINE 629 BASE RED PART A', 'INTERFINE 629 BASE RED PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE ULTRA DEEP PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BASE YELLOW PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLACK PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE GREY PART A', 'INTERFINE 629 BLUE JADE PART A', 'INTERFINE 629 BLUE JADE PART A', 'INTERFINE 629 BLUE PART A', 'INTERFINE 629 BLUE PART A', 'INTERFINE 629 BLUED WHITE PART A', 'INTERFINE 629 BLUED WHITE PART A', 'INTERFINE 629 BRIGHT RED ORANGE PART A', 'INTERFINE 629 BRIGHT RED ORANGE PART A', 'INTERFINE 629 CAT YELLOW PART A', 'INTERFINE 629 CAT YELLOW PART A', 'INTERFINE 629 CAT YELLOW PART A', 'INTERFINE 629 CATERPILLAR YELLOW PT A', 'INTERFINE 629 CATERPILLAR YELLOW PT A', 'INTERFINE 629 CEMENT GREY PART A', 'INTERFINE 629 CEMENT GREY PART A', 'INTERFINE 629 CLAY BROWN (RAL8003)PT A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CLOUD GREY PART A', 'INTERFINE 629 CORFLOWER PART A', 'INTERFINE 629 CORFLOWER PART A', 'INTERFINE 629 CURING AGENT', 'INTERFINE 629 DARK BLUE PART A', 'INTERFINE 629 DEEP BASE PART A', 'INTERFINE 629 FINISH BLUE GREEN', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 FLOTILLA GRAY PART A', 'INTERFINE 629 G21 JADE PART A', 'INTERFINE 629 G25 OLIVE PART A', 'INTERFINE 629 GARDENIA PART A', 'INTERFINE 629 GARDENIA PART A', 'INTERFINE 629 GARDENIA PART A', 'INTERFINE 629 GARDENIA PART A', 'INTERFINE 629 GARDENIA PART A', 'INTERFINE 629 GLOAMING PART A', 'INTERFINE 629 GLOAMING PART A', 'INTERFINE 629 GLOAMING PART A', 'INTERFINE 629 GLOAMING PART A', 'INTERFINE 629 GOLDEN TAN PART A', 'INTERFINE 629 GOLDEN TAN PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GOLDEN YELLOW PART A', 'INTERFINE 629 GRAPHITE GREY PART A', 'INTERFINE 629 GRAPHITE GREY PART A', 'INTERFINE 629 GRASS GREEN(RAL6010)PT A', 'INTERFINE 629 GREEN PART A', 'INTERFINE 629 GREEN PART A', 'INTERFINE 629 GREY PART A', 'INTERFINE 629 GREY PART A', 'INTERFINE 629 HALIBURTON RED PART A', 'INTERFINE 629 HS FINISH AUTUMN CLOUD', 'INTERFINE 629 HS FINISH BLUE BELL BASE', 'INTERFINE 629 HS FINISH CREAM BASE', 'INTERFINE 629 HS FINISH FISHER GREEN BASE', 'INTERFINE 629 HS FINISH FISHER GREY BASE', 'INTERFINE 629 HS FINISH GLOAMING BASE', 'INTERFINE 629 HS FINISH GOLDEN YELLOW BASE', 'INTERFINE 629 HS FINISH GREY BASE', 'INTERFINE 629 HS FINISH MARBLE GREY BASE', 'INTERFINE 629 HS FINISH MIST GREY BASE', 'INTERFINE 629 HS FINISH MOCHA BASE', 'INTERFINE 629 HS FINISH POPPY RED BASE', 'INTERFINE 629 HS FINISH POWDER BLUE BASE', 'INTERFINE 629 HS FINISH TANGERINE', 'INTERFINE 629 HS FINISH YELLOW BASE', 'INTERFINE 629 HS LIGHT GREY BASE', 'INTERFINE 629 HS LIGHT GREY BASE', 'INTERFINE 629 HS LIGHT GREY BASE', 'INTERFINE 629 HS LIGHT GREY BASE', 'INTERFINE 629 HS YELLOW BASE', 'INTERFINE 629 JADE PART A', 'INTERFINE 629 JADE PART A', 'INTERFINE 629 JADE PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 JET BLACK PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT BOX GREY PART A', 'INTERFINE 629 LIGHT GREY BLUE PART A', 'INTERFINE 629 LIGHT GREY BLUE PART A', 'INTERFINE 629 LIGHT GREY PART A', 'INTERFINE 629 LIGHT GREY PART A', 'INTERFINE 629 LIGHT IVORY PART A', 'INTERFINE 629 LIGHTBOX GREY PART A', 'INTERFINE 629 LIGHTBOX GREY PART A', 'INTERFINE 629 MAUVE PART A', 'INTERFINE 629 MIST GREEN PART A', 'INTERFINE 629 MIST GREEN PART A', 'INTERFINE 629 MIST GREY PART A', 'INTERFINE 629 MIST GREY PART A', 'INTERFINE 629 N14 WHITE PART A', 'INTERFINE 629 N14 WHITE PART A', 'INTERFINE 629 N14 WHITE PART A', 'INTERFINE 629 N35 LIGHT GREY PART A', 'INTERFINE 629 N35 LIGHT GREY PART A', 'INTERFINE 629 N35 LIGHT GREY PART A', 'INTERFINE 629 N35 LIGHT GREY PART A', 'INTERFINE 629 N52 MID GREY PART A', 'INTERFINE 629 N52 MID GREY PART A', 'INTERFINE 629 N52 MID GREY PART A', 'INTERFINE 629 N52 MID GREY PART A', 'INTERFINE 629 NASSCO WHITE PART A', 'INTERFINE 629 NASSCO WHITE PART A', 'INTERFINE 629 OATMEAL PART A', 'INTERFINE 629 OFF WHITE PART A', 'INTERFINE 629 OFF WHITE PART A', 'INTERFINE 629 OFF WHITE PART A', 'INTERFINE 629 OFF WHITE PART A', 'INTERFINE 629 OFF WHITE PART A', 'INTERFINE 629 OYSTER PART A', 'INTERFINE 629 OYSTER PART A', 'INTERFINE 629 OYSTER WHITE PART A', 'INTERFINE 629 PARAKEET PART A', 'INTERFINE 629 PARAKEET PART A', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 PART B', 'INTERFINE 629 POWDER BLUE PART A', 'INTERFINE 629 POWDER BLUE PART A', 'INTERFINE 629 RAINFOREST GREEN PART A', 'INTERFINE 629 RAL1013OYSTER WHITE PT A', 'INTERFINE 629 RAL1013OYSTER WHITE PT A', 'INTERFINE 629 RAL1021 YELLOW PART A', 'INTERFINE 629 RAL1021 YELLOW PART A', 'INTERFINE 629 RAL1023 TRAFF.YELLOW PAR', 'INTERFINE 629 RAL1023 TRAFF.YELLOW PTA', 'INTERFINE 629 RAL1023 TRAFF.YELLOW PTA', 'INTERFINE 629 RAL1023TRAFFICYELLOW PTA', 'INTERFINE 629 RAL1023TRAFFICYELLOW PTA', 'INTERFINE 629 RAL1023TRAFFICYELLOW PTA', 'INTERFINE 629 RAL1023TRAFFICYELLOW PTA', 'INTERFINE 629 RAL2004 PURE ORANGE PT A', 'INTERFINE 629 RAL2004 PURE ORANGE PT A', 'INTERFINE 629 RAL2004 PURE ORANGE PTA', 'INTERFINE 629 RAL2004 PURE ORANGE PTA', 'INTERFINE 629 RAL3000 FLAME RED PART A', 'INTERFINE 629 RAL3000 FLAME RED PART A', 'INTERFINE 629 RAL3000 FLAME RED PT A', 'INTERFINE 629 RAL3000 FLAME RED PT A', 'INTERFINE 629 RAL3002 CARMINE RED PT A', 'INTERFINE 629 RAL3020 TRAFFIC RED PT A', 'INTERFINE 629 RAL3020 TRAFFIC RED PT A', 'INTERFINE 629 RAL3020 TRAFFIC RED PT A', 'INTERFINE 629 RAL3020 TRAFFIC RED PT A', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL3020 TRAFFIC RED PTA', 'INTERFINE 629 RAL5002 ULTRAMAR BLUE PA', 'INTERFINE 629 RAL5002 ULTRAMAR BLUE PA', 'INTERFINE 629 RAL5003 SAPPHIRE BLUE PA', 'INTERFINE 629 RAL5003 SAPPHIRE BLUE PA', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5009 AZURE BLUE PT A', 'INTERFINE 629 RAL5015 SKY BLUE PART A', 'INTERFINE 629 RAL5015 SKY BLUE PART A', 'INTERFINE 629 RAL5015 SKY BLUE PT A', 'INTERFINE 629 RAL5015 SKY BLUE PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6002 LEAF GREEN PT A', 'INTERFINE 629 RAL6011 RESEDA GREEN PTA', 'INTERFINE 629 RAL6017 MAY GREEN PART A', 'INTERFINE 629 RAL6017 MAY GREEN PART A', 'INTERFINE 629 RAL6018 YELLOW GREEN PAR', 'INTERFINE 629 RAL6018 YELLOW GREEN PAR', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6021 PALE GREEN PT A', 'INTERFINE 629 RAL6028 PINE GREEN PT A', 'INTERFINE 629 RAL6028 PINE GREEN PT A', 'INTERFINE 629 RAL7005 MOUSE GREY PT A', 'INTERFINE 629 RAL7005 MOUSE GREY PT A', 'INTERFINE 629 RAL7030 STONE GREY PTA', 'INTERFINE 629 RAL7030 STONE GREY PTA', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 GREY PART A', 'INTERFINE 629 RAL7032 PEBBLE GREY PT A', 'INTERFINE 629 RAL7035 LIGHT GREY PT A', 'INTERFINE 629 RAL7035 LIGHT GREY PT A', 'INTERFINE 629 RAL7035 LIGHT GREY PTA', 'INTERFINE 629 RAL7035 LIGHT GREY PTA', 'INTERFINE 629 RAL7036 PLATINUMGREY PTA', 'INTERFINE 629 RAL7036 PLATINUMGREY PTA', 'INTERFINE 629 RAL7037 DUSTY GREY PARTA', 'INTERFINE 629 RAL7037 DUSTY GREY PARTA', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7037 DUSTY GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL7038 AGATE GREY PT A', 'INTERFINE 629 RAL8001 OCHRE BROWN PT A', 'INTERFINE 629 RAL8001 OCHRE BROWN PT A', 'INTERFINE 629 RAL9002 GREY WHITE PT A', 'INTERFINE 629 RAL9002 GREY WHITE PT A', 'INTERFINE 629 RAL9002 GREY WHITE PT A', 'INTERFINE 629 RAL9002 GREY WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9010 PURE WHITE PT A', 'INTERFINE 629 RAL9016 TRAFFIC WHITE PA', 'INTERFINE 629 RAL9016 TRAFFIC WHITE PA', 'INTERFINE 629 RIVERGUM PART A', 'INTERFINE 629 RIVERGUM PART A', 'INTERFINE 629 RIVERGUM PART A', 'INTERFINE 629 RIVERGUM PART A', 'INTERFINE 629 ROSE PINK PART A', 'INTERFINE 629 ROSE PINK PART A', 'INTERFINE 629 SAND PART A', 'INTERFINE 629 SAND PART A', 'INTERFINE 629 SIGNAL RED PART A', 'INTERFINE 629 SIGNAL RED PART A', 'INTERFINE 629 SIGNAL WHITE PART A', 'INTERFINE 629 STORM GREY PART A', 'INTERFINE 629 STORM GREY PART A', 'INTERFINE 629 STORM GREY PART A', 'INTERFINE 629 STORM GREY PART A', 'INTERFINE 629 SUNFLOWER PART A', 'INTERFINE 629 SUNFLOWER PART A', 'INTERFINE 629 SUNFLOWER PART A', 'INTERFINE 629 SUNFLOWER PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TANGERINE PART A', 'INTERFINE 629 TELEGREY PART A', 'INTERFINE 629 TRAFFIC BLACK PART A', 'INTERFINE 629 TRAFFIC YELLOW PART A', 'INTERFINE 629 VIVID YELLOW PART A', 'INTERFINE 629 WARATAH PART A', 'INTERFINE 629 WARATAH PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 WHITE PART A', 'INTERFINE 629 Y12 WATTLE PART A', 'INTERFINE 629 Y12 WATTLE PART A', 'INTERFINE 629 Y12 WATTLE PART A', 'INTERFINE 629 Y12 WATTLE PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y14 GOLDEN YELLOW PART A', 'INTERFINE 629 Y15 SUNFLOWER PART A', 'INTERFINE 629 Y34 CREAM PART A', 'INTERFINE 629 Y44 SAND PART A', 'INTERFINE 629 YELLOW GREY PART A', 'INTERFINE 629 YELLOW GREY PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 YELLOW PART A', 'INTERFINE 629 ZINC YELLOW PART A', 'INTERFINE 629 ZINC YELLOW PART A']

【讨论】：

它现在按要求工作。谢谢。如果可以的话，请告诉我可以从中学习这些东西的任何链接，这将很有帮助。谢谢
@deepesh 简而言之，作为一个经验法则，当您需要读取任何属性时，例如text、innerHTML 等元素必须使用visibility_of_all_elements_located()，如果你想click() 任何元素必须使用element_to_be_clickable()
在这种情况下，我曾经在使用 selenium 时使用find_elements_by_xpath()。再次感谢您的投入和时间。
请不要假设任何东西会自动出现/可见/可交互。您可以随意使用所有基于 Selenium 的方法。
是的，目前正在进行的项目，现在我遇到了selenium 的新功能/方法，现在我真的需要深入学习它们......感谢您的所有帮助跨度>

【解决方案2】：

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time
import random
from bs4 import BeautifulSoup
driver = webdriver.Chrome()
driver.get('https://www.international-pc.com/product/interfine-629')
soup_seln = BeautifulSoup(driver.page_source, 'lxml')
time.sleep(5)
x=driver.find_element_by_xpath("//*[@id='cookie-notification']/div/article/div[3]/a[1]")
x.click()


tables_sds = soup_seln.find("table", {"id": "documentTable-2"}).find("tbody")
tr_sds = tables_sds.find_all("tr")
for i in range(len(tr_sds)):
    print('td')[0].get_text()+"                   "+tr_sds[i].findAll('td')[1].get_text()+"                           "+tr_sds[i].findAll('td')[2].get_text())

while True:
    time.sleep(5)
    x=driver.find_element_by_xpath("//*[@id='documentTable-2_next']")
    x.click()
    tables_sds = soup_seln.find("table", {"id": "documentTable-2"}).find("tbody")
    tr_sds = tables_sds.find_all("tr")
    for i in range(len(tr_sds)):
        print(tr_sds[i].findAll('td')[0].get_text()+"                   "+tr_sds[i].findAll('td')[1].get_text()+"                           "+tr_sds[i].findAll('td')[2].get_text())

不需要点击同意，但您有 5 秒时间，您必须手动点击 SDS 按钮一次。如果您发现运行此代码有任何困难，我会帮助您，我的示例结果是：

INTERFINE 629 APPLE GREEN PART A
INTERFINE 629 APPLE GREEN PART A
INTERFINE 629 B15 MID BLUE PART A
INTERFINE 629 B15 MID BLUE PART A
INTERFINE 629 B15 MID BLUE PART A
INTERFINE 629 B15 MID BLUE PART A
INTERFINE 629 B44 LIGHT GREY BLUE PTA
INTERFINE 629 BASE LIGHT PART A
INTERFINE 629 BASE LIGHT PART A
INTERFINE 629 BASE LIGHT PART A
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia
INTERFINE 629 BASE LIGHT PART A                   Malayalam                           Malaysia
INTERFINE 629 APPLE GREEN PART A                   Chinese                           China
INTERFINE 629 APPLE GREEN PART A                   English                           China
INTERFINE 629 B15 MID BLUE PART A                   Japanese                           Japan
INTERFINE 629 B15 MID BLUE PART A                   English                           Japan
INTERFINE 629 B15 MID BLUE PART A                   Chinese                           Singapore
INTERFINE 629 B15 MID BLUE PART A                   English                           Singapore
INTERFINE 629 B44 LIGHT GREY BLUE PTA                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Thailand
INTERFINE 629 BASE LIGHT PART A                   English                           Australia

【讨论】：

@Janil Patel，请注意，您打印的值没有改变，实际上它们在每 10 个元素之后重复。