尝试使用 python 网络爬虫

Question

我第一次尝试创建网络爬虫。该脚本不会在没有错误代码的情况下运行。

我正在使用的教程：

https://oxylabs.io/blog/python-web-scraping

我已完成所有步骤，并通过其他 youtube 教程进行了进一步的步骤，但没有任何效果；(

错误代码：

Traceback (most recent call last):
  File "C:\Users\hpy03\PycharmProjects\webtest\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
    self.process = subprocess.Popen(cmd, env=self.env,
  File "C:\Users\hpy03\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\hpy03\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 1307, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/hpy03/PycharmProjects/webtest/test_scraping.py", line 5, in <module>
    driver = webdriver.Chrome(executable_path='c:\path\to\windows\webdriver\executable.exe')
  File "C:\Users\hpy03\PycharmProjects\webtest\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
    self.service.start()
  File "C:\Users\hpy03\PycharmProjects\webtest\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'executable.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

Answer 1

您似乎没有正确引用 chromedriver。如果您在命令提示符中运行 chromedriver -v 并返回一个版本，那么只需从您的代码中删除 executable_path=...：

之前：

driver = webdriver.Chrome(executable_path='c:\path\to\windows\webdriver\executable.exe')

之后：

driver = webdriver.Chrome()

否则，您可以将 chromedriver.exe 与您的代码放在同一文件夹中，并像这样引用它：

driver = webdriver.Chrome(executable_path='./chromedriver.exe')

如果您没有 chromedriver.exe，请查看here。