美汤元素未发现异常

Question

我正在使用 Python 3.6 和 Beautiful Soup 制作爬虫。这是我的代码

当我运行它时，我得到 element not found 异常，为什么？我想要做的是选择 uri 并单击名称 uri 以便打开一个新页面

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import re
import pandas as pd
import os
url = "https://www.codechef.com/ratings/all?order=asc&page=3&sortBy=global_rank"


# create a new Firefox session
driver = webdriver.Firefox()
driver.get(url)
soup_level1=BeautifulSoup(driver.page_source, 'lxml')

datalist = [] #empty list
x = 488 
for link in soup_level1.find_all('a', id=re.compile(r"^ember")):

    elemnt2222=driver.find_element_by_xpath("//*[@id='ember"+str(493)+"']/td[2]/div[2]/a")

    python_button = elemnt2222
    python_button.click() #click link

Answer 1

您不必单击。只需从锚点获取href。导航到该 URL。

from selenium import webdriver
from selenium.webdriver.common.keys import Keys    
from bs4 import BeautifulSoup
import re
import pandas as pd
import os
import time

url = "https://www.codechef.com/ratings/all?order=asc&page=3&sortBy=global_rank"
driver = webdriver.Firefox()
driver.get(url)
#Takes some time to load
time.sleep(5)
soup=BeautifulSoup(driver.page_source, 'lxml')

links = soup.select('div.user-name > a')
for link in links:
  print(link.get('href'))

这会给你这个结果

/users/sumeet_varma
/users/fjzzq2002
/users/dreamoon4
/users/y0105w49
/users/nblt
/users/dzhulgakov
/users/uwi
/users/Fcdkbear
/users/austin990301
/users/KADR
/users/adkroxx
/users/kostroma
/users/fhlasek
/users/argos
/users/watcher
/users/nafis
/users/scli
/users/mister
/users/iwiwi
/users/aurinegro

之后你可以导航到

https://www.codechef.com/users/username