【发布时间】:2020-01-11 22:49:42
【问题描述】:
我正在关注这个关于网络抓取的教程https://www.linkedin.com/pulse/how-easy-scraping-data-from-linkedin-profiles-david-craven/。 python脚本正在生成错误,我已经尝试将目录添加到PATH,当我将路径回显到屏幕时它会显示,但现在它应该只有一个时显示“/Users/owner/Users/owner”路径中的“用户/所有者”。
我在 mac os High Sierra 中使用 bash,并且我是一名数据科学专业的学生,所以 DevOps 对我来说是一个挑战,同时我也在学习如何将代码发布到 StackOverflow,但我正在尝试记录我的步骤,以便更容易解决此问题。
- 我 pip 安装了 selenium
- 我将 chromedriver 下载到我的网页抓取脚本文件的目录并双击运行
- 我以为我使用 'export PATH=$PATH:~opt/bin:~/Users/owner/sbox/test/pandas_sqlite_dbase/chromedriver' 将目录添加到我的 PATH,这是我从 http://osxdaily.com/2014/08/14/add-new-path-to-path-command-line/ 找到的方向
- 我更新了画中画
- 我要运行脚本的目录是'/Users/owner/sbox/test/pandas_sqlite_dbase'
- 还有另一篇 SO 帖子 Can a website detect when you are using selenium with chromedriver? 谈到了如何自动检测和禁用带有 selenium 的 chromedriver...所以我是否要尝试使用过时的代码库?
- 我可以发布我的整个 PATH 或提供其他信息。
from selenium import webdriver
driver = webdriver.Chrome('~/Users/owner/sbox/test/pandas_sqlite_dbase/googlechrome')
driver.get('https://www.linkedin.com')
现在我收到一个回溯错误
Traceback (most recent call last):
File "/Users/owner/anaconda3/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 76, in start
stdin=PIPE)
File "/Users/owner/anaconda3/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/Users/owner/anaconda3/lib/python3.7/subprocess.py", line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '~/Users/owner/sbox/test/pandas_sqlite_dbase/googlechrome': '~/Users/owner/sbox/test/pandas_sqlite_dbase/googlechrome'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/owner/sbox/test/pandas_sqlite_dbase/scraping_tutorial.py", line 7, in <module>
driver = webdriver.Chrome('~/Users/owner/sbox/test/pandas_sqlite_dbase/googlechrome')
File "/Users/owner/anaconda3/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/Users/owner/anaconda3/lib/python3.7/site-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'googlechrome' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
[Finished in 0.7s with exit code 1]
[shell_cmd: python -u "/Users/owner/sbox/test/pandas_sqlite_dbase/scraping_tutorial.py"]
[dir: /Users/owner/sbox/test/pandas_sqlite_dbase]
[path: /usr/bin:/bin:/usr/sbin:/sbin]
【问题讨论】:
-
更新 - 如果我在 webdriver.Chrome(此处删除文本)之后删除括号内的代码,则脚本有效。
标签: python python-3.x web-scraping path selenium-chromedriver