我正在尝试在“https://dribbble.com/jobs”上抓取公司的名称答案

【问题标题】：I am trying to scrape the names of the companies on "https://dribbble.com/jobs"我正在尝试在“https://dribbble.com/jobs”上抓取公司的名称
【发布时间】：2020-04-28 11:54:09
【问题描述】：

当我尝试以下代码时，它只会再次返回第一项。我是 python 新手，所以不胜感激。

import pandas as pd 
from selenium import webdriver
from time import sleep
driver = webdriver.Chrome('/Users/user/Downloads/chromedriver_win32/chromedriver')
driver.get('https://dribbble.com/jobs')
assert 'Dribbble' in driver.title 
columns = ['company']
count = 0
jobs = pd.DataFrame(columns=columns)
for item in range(10): 
            company_elem = "job-board-job-title"
            company = driver.find_element_by_class_name(company_elem).text
            item+=1
            jobs.loc[item] = [company]
driver.close()

【问题讨论】：

标签： python selenium webdriver screen-scraping

【解决方案1】：

import pandas as pd
from selenium import webdriver
# from time import sleep
driver = webdriver.Chrome(r'E:\data\python\pycharm\chromedriver_win32\chromedriver.exe')
driver.get('https://dribbble.com/jobs')
assert 'Dribbble' in driver.title
columns = ['company']
count = 0
jobs = pd.DataFrame(columns=columns)
# for item in range(10):
# company_elem = "job-board-job-title"
companies = driver.find_elements_by_class_name("job-board-job-title")
for i in companies:
     print(i.text)
# count += 1
# jobs.loc[count] = [company]
# print(jobs)
driver.close()

现在对您的代码进行了一些更改，您需要获取所有元素，而不是使用 find 元素的 1，使用 find 元素。 https://selenium-python.readthedocs.io/locating-elements.html#locating-elements-by-class-name

【讨论】：

【解决方案2】：

你只能改变一行，所以不是

company = driver.find_element_by_class_name(company_elem).text

试试

company = driver.find_elements_by_class_name(company_elem)[item].text

输出将是：

                                     company
1       UI Designer / Animator / Illustrator
2                    Senior Product Designer
3          Freelance Senior Graphic Designer
4    Senior Product Designer, Internal Tools
5                       Sr. Product Designer
6                            User Researcher
7                           Product Designer
8                            Design Director
9                           Product Designer
10  Supply Chain Account Manager - 200163935

【讨论】：