【发布时间】:2021-03-16 23:03:57
【问题描述】:
我有这个数据框:
CurrentThermostatTemp DateAppointment
City
Abergele 20.0 2018-07-04
Abergele 25.0 2018-06-21
Abergele 21.0 2018-08-29
Abergele 22.0 2018-04-30
Abergele 19.0 2018-05-24
Abergele 20.0 2018-07-04
Abergele 21.0 2018-05-16
Abergele 22.0 2018-05-29
Abergele 10.0 2018-07-04
Abergele 18.0 2018-05-09
Abergele 20.0 2018-05-01
Abergele 19.0 2018-04-17
Abergele 21.0 2018-05-29
Abingdon 22.0 2018-03-22
Abingdon 20.0 2018-03-22
Abingdon 19.0 2018-06-18
Abingdon 0.0 2018-04-26
Abingdon 14.0 2017-08-08
Altrincham 20.0 2018-01-18
Altrincham 25.0 2018-04-25
Altrincham 20.0 2018-03-14
Altrincham 21.0 2018-01-23
Altrincham 18.0 2018-09-17
Altrincham 21.0 2018-01-23
Altrincham 20.0 2018-04-03
Altrincham 21.0 2018-04-16
Altrincham 20.0 2018-06-15
Altrincham 18.0 2018-03-14
Altrincham 18.0 2018-04-04
Altrincham 21.0 2018-03-22
... ... ...
Wrexham 20.0 2018-04-26
Wrexham 20.0 2018-04-18
Wrexham 22.0 2018-07-10
Wrexham 18.0 2018-06-04
Wrexham 20.0 2018-03-05
Wrexham 21.0 2018-03-19
Wrexham 20.0 2018-03-26
Wrexham 18.0 2018-03-29
Wrexham 25.0 2018-02-27
Wrexham 20.0 2018-02-05
Wrexham 18.0 2018-01-15
Wrexham 17.0 2018-02-01
Wrexham 20.0 2018-05-09
Wrexham 18.0 2018-05-02
Wrexham 15.0 2018-02-22
Wrexham 15.0 2018-04-20
Wrexham 20.0 2018-08-20
Wrexham 18.0 2018-09-05
Wrexham 20.0 2018-05-18
Wrexham 17.0 2018-03-07
Wrexham 21.0 2018-04-04
Wrexham 21.0 2018-04-05
Wrexham 18.0 2018-06-15
Wrexham 20.0 2018-04-05
Wrexham 20.0 2018-05-30
Wrexham 15.0 2018-05-25
Wrexham 20.0 2018-04-18
Wrexham 21.0 2018-04-09
Yelverton 0.0 2018-06-11
Yelverton 21.0 2018-02-22
[6159 rows x 2 columns]
我想通过 Safari 的网络驱动程序从互联网上提取当天和城市的当地温度,我希望它创建一个列以将其附加到数据框中。我用于测试的页面是https://www.wunderground.com,尽管我可以使用任何我想要的。这是我尝试过的:
import pandas as pd
import numpy as np
import datetime as dt
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver import Safari
from selenium.webdriver.support.ui import Select
dates = []
cities = []
for index, row in Tempvs.iterrows():
t = row["DateAppointment"].to_pydatetime()
c = index
cities.append(c)
dates.append(t)
for i in range(len(dates)):
driver = Safari()
wait = WebDriverWait(driver, 40)
url = "https://www.wunderground.com/weather/gb/{}".format(str(cities[i]))
driver.get(url)
elem=wait.until(EC.presence_of_element_located((By.XPATH,"//a[./span[.='History']]"))) #Press on historic data
driver.execute_script("arguments[0].click();", elem)
selectMonth=Select(wait.until(EC.presence_of_element_located((By.ID,"monthSelection")))) #select month
selectMonth = Select(driver.find_element_by_id("monthSelection"))
selectMonth.select_by_visible_text(str(dates[i].strftime("%B")))
selectDay = Select(driver.find_element_by_id("daySelection")) #select day
selectDay.select_by_visible_text(str(dates[i].day))
selectYear = Select(driver.find_element_by_id("yearSelection")) #select year
selectYear.select_by_visible_text(str(dates[i].year))
View = driver.find_element_by_id("dateSubmit").click() #view data
avrgTemp=wait.until(EC.presence_of_element_located((By.XPATH,
'//*[@id="inner-content"]/div[2]/div/div[3]/div/div/lib-city-history-summary/div/div[2]/table/tbody[1]/tr[3]/td[1]'))).text #extract daily average temp from that day and place
print(avrgTemp) #have a list of all these temperatures to later append it to the dataframe
我遇到了很多错误,但我得到的主要错误是:
selenium.common.exceptions.SessionNotCreatedException: Message: Could not create a session: The Safari instance is already paired with another WebDriver session.
我知道这可能需要一些日志时间来处理,但我可以在收集数据时让我的计算机处于待机状态。我将不胜感激任何帮助。 谢谢。
【问题讨论】:
标签: python dataframe selenium for-loop webdriver