【问题标题】:Extract historic daily local temperatures from cities in the UK从英国城市中提取历史每日当地温度
【发布时间】:2021-03-16 23:03:57
【问题描述】:

我有这个数据框:

            CurrentThermostatTemp DateAppointment
City                                             
Abergele                     20.0      2018-07-04
Abergele                     25.0      2018-06-21
Abergele                     21.0      2018-08-29
Abergele                     22.0      2018-04-30
Abergele                     19.0      2018-05-24
Abergele                     20.0      2018-07-04
Abergele                     21.0      2018-05-16
Abergele                     22.0      2018-05-29
Abergele                     10.0      2018-07-04
Abergele                     18.0      2018-05-09
Abergele                     20.0      2018-05-01
Abergele                     19.0      2018-04-17
Abergele                     21.0      2018-05-29
Abingdon                     22.0      2018-03-22
Abingdon                     20.0      2018-03-22
Abingdon                     19.0      2018-06-18
Abingdon                      0.0      2018-04-26
Abingdon                     14.0      2017-08-08
Altrincham                   20.0      2018-01-18
Altrincham                   25.0      2018-04-25
Altrincham                   20.0      2018-03-14
Altrincham                   21.0      2018-01-23
Altrincham                   18.0      2018-09-17
Altrincham                   21.0      2018-01-23
Altrincham                   20.0      2018-04-03
Altrincham                   21.0      2018-04-16
Altrincham                   20.0      2018-06-15
Altrincham                   18.0      2018-03-14
Altrincham                   18.0      2018-04-04
Altrincham                   21.0      2018-03-22
...                           ...             ...
Wrexham                      20.0      2018-04-26
Wrexham                      20.0      2018-04-18
Wrexham                      22.0      2018-07-10
Wrexham                      18.0      2018-06-04
Wrexham                      20.0      2018-03-05
Wrexham                      21.0      2018-03-19
Wrexham                      20.0      2018-03-26
Wrexham                      18.0      2018-03-29
Wrexham                      25.0      2018-02-27
Wrexham                      20.0      2018-02-05
Wrexham                      18.0      2018-01-15
Wrexham                      17.0      2018-02-01
Wrexham                      20.0      2018-05-09
Wrexham                      18.0      2018-05-02
Wrexham                      15.0      2018-02-22
Wrexham                      15.0      2018-04-20
Wrexham                      20.0      2018-08-20
Wrexham                      18.0      2018-09-05
Wrexham                      20.0      2018-05-18
Wrexham                      17.0      2018-03-07
Wrexham                      21.0      2018-04-04
Wrexham                      21.0      2018-04-05
Wrexham                      18.0      2018-06-15
Wrexham                      20.0      2018-04-05
Wrexham                      20.0      2018-05-30
Wrexham                      15.0      2018-05-25
Wrexham                      20.0      2018-04-18
Wrexham                      21.0      2018-04-09
Yelverton                     0.0      2018-06-11
Yelverton                    21.0      2018-02-22

[6159 rows x 2 columns]

我想通过 Safari 的网络驱动程序从互联网上提取当天和城市的当地温度,我希望它创建一个列以将其附加到数据框中。我用于测试的页面是https://www.wunderground.com,尽管我可以使用任何我想要的。这是我尝试过的:

import pandas as pd 
import numpy as np 
import datetime as dt

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
from selenium.webdriver import Safari
from selenium.webdriver.support.ui import Select

dates = []
cities = []
for index, row in Tempvs.iterrows():
    t = row["DateAppointment"].to_pydatetime()
    c = index
    cities.append(c)
    dates.append(t)
    for i in range(len(dates)):

        driver = Safari()
        wait = WebDriverWait(driver, 40)
        url = "https://www.wunderground.com/weather/gb/{}".format(str(cities[i]))
        driver.get(url)

        elem=wait.until(EC.presence_of_element_located((By.XPATH,"//a[./span[.='History']]"))) #Press on historic data
        driver.execute_script("arguments[0].click();", elem) 

        selectMonth=Select(wait.until(EC.presence_of_element_located((By.ID,"monthSelection"))))   #select month
        selectMonth = Select(driver.find_element_by_id("monthSelection"))
        selectMonth.select_by_visible_text(str(dates[i].strftime("%B")))

        selectDay = Select(driver.find_element_by_id("daySelection"))   #select day
        selectDay.select_by_visible_text(str(dates[i].day))

        selectYear = Select(driver.find_element_by_id("yearSelection")) #select year
        selectYear.select_by_visible_text(str(dates[i].year))

        View = driver.find_element_by_id("dateSubmit").click()  #view data
        avrgTemp=wait.until(EC.presence_of_element_located((By.XPATH,
                            '//*[@id="inner-content"]/div[2]/div/div[3]/div/div/lib-city-history-summary/div/div[2]/table/tbody[1]/tr[3]/td[1]'))).text #extract daily average temp from that day and place

        print(avrgTemp)  #have a list of all these temperatures to later append it to the dataframe

我遇到了很多错误,但我得到的主要错误是:

selenium.common.exceptions.SessionNotCreatedException: Message: Could not create a session: The Safari instance is already paired with another WebDriver session.

我知道这可能需要一些日志时间来处理,但我可以在收集数据时让我的计算机处于待机状态。我将不胜感激任何帮助。 谢谢。

【问题讨论】:

    标签: python dataframe selenium for-loop webdriver


    【解决方案1】:

    移动driver = Safari()

    到文件顶部...导入之后的循环之前

    【讨论】:

      猜你喜欢
      • 2021-09-27
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2013-01-12
      • 2019-08-05
      • 2018-07-10
      • 1970-01-01
      相关资源
      最近更新 更多