如何在 python 中使用 pandas 和 excel
【问题讨论】：
标签： python
您需要/要求的数据都在 api 中。那时你不需要使用 Selnium。只需获取所有事件数据，您就可以根据需要操作数据框。
这将为您获取所有数据：
import requests
import pandas as pd
import datetime
today = datetime.datetime.now().strftime("%Y-%m-%d")
date_str = input("Input the date in the form of yyyy-mm-dd, hit Enter for today\n-> ")
if not date_str:
date = today
else:
date = datetime.datetime.strptime(date_str, "%Y-%m-%d").strftime("%Y-%m-%d")
url = 'https://push.api.bbci.co.uk/batch'
query = {
't':f'/data/bbc-morph-football-scores-match-list-data/endDate/{date}/startDate/{date}/todayDate/{today}/tournament/full-priority-order/version/2.4.6?timeout=5'}
jsonData = requests.get(url, params=query).json()
rows = []
matchData = jsonData['payload'][0]['body']['matchData']
for match in matchData:
tournamentDatesWithEvents = match['tournamentDatesWithEvents']
for k, v in tournamentDatesWithEvents.items():
for i in v:
events = i['events']
for event in events:
rows.append(event)
event_df = pd.json_normalize(rows)
输出：仅前 5 行的样本
Input the date in the form of yyyy-mm-dd, hit Enter for today
-> 2021-11-10
print(event_df.head(5).to_string())
eventKey startTime isTBC minutesElapsed minutesIntoAddedTime eventStatus eventStatusNote eventStatusReason eventOutcomeType eventType seriesWinner cpsId cpsLive officials tournamentInfo eventActions startTimeInUKHHMM comment href tournamentSlug homeTeam.key homeTeam.scores.score homeTeam.scores.halfTime homeTeam.scores.fullTime homeTeam.scores.extraTime homeTeam.scores.shootout homeTeam.scores.aggregate homeTeam.scores.aggregateGoalsAway homeTeam.formation homeTeam.eventOutcome homeTeam.name.first homeTeam.name.full homeTeam.name.abbreviation homeTeam.name.last awayTeam.key awayTeam.scores.score awayTeam.scores.halfTime awayTeam.scores.fullTime awayTeam.scores.extraTime awayTeam.scores.shootout awayTeam.scores.aggregate awayTeam.scores.aggregateGoalsAway awayTeam.formation awayTeam.eventOutcome awayTeam.name.first awayTeam.name.full awayTeam.name.abbreviation awayTeam.name.last eventProgress.period eventProgress.status venue.name.abbreviation venue.name.videCode venue.name.first venue.name.full venue.homeCountry tournamentName.first tournamentName.full tournamentName.abbreviation
0 EFBO2257629 2021-11-10T17:00:00+00:00 False NaN NaN post-event Result None regular Regular None None None [] None None 17:00 None None international-friendlies TFBB11658 0 0 0 None None None None None loss Kosovo Kosovo Kosovo None TFBB1843 2 0 0 None None None None None win Jordan Jordan Jordan None FULLTIME RESULT Stadiumi Fadil Vokrri Stadiumi Fadil Vokrri Stadiumi Fadil Vokrri Stadiumi Fadil Vokrri None International Friendlies International Friendlies Int Friendlies
1 EFBO2245946 2021-11-10T17:45:00+00:00 False 90.0 3.0 post-event Result None regular Regular None None None [] None None 17:45 None None womens-champions-league TFBB9121 4 3 4 None None None None None win Barcelona Femenino Barcelona Femenino Barcelona Femenino None TFBB8706 0 0 0 None None None None None loss TSG 1899 Hoffenheim Ladies TSG 1899 Hoffenheim Ladies TSG 1899 Hoffenheim Ladies None FULLTIME RESULT Estadi Johan Cruyff Estadi Johan Cruyff Estadi Johan Cruyff Estadi Johan Cruyff Spain Women's Champions League UEFA Women's Champions League Women's Champs Lge
2 EFBO2245947 2021-11-10T17:45:00+00:00 False 90.0 4.0 post-event Result None regular Regular None 59214028 yes [] None None 17:45 None /sport/football/59214028 womens-champions-league TFBB16926 1 0 1 None None None None None loss HB Køge Women HB Køge Women HB Køge Women None TFBB6272 5 1 5 None None None None None win Arsenal Women Arsenal Women Arsenal Women None FULLTIME RESULT Capelli Sport Stadion Capelli Sport Stadion Capelli Sport Stadion Capelli Sport Stadion Denmark Women's Champions League UEFA Women's Champions League Women's Champs Lge
3 EFBO2245948 2021-11-10T20:00:00+00:00 False 90.0 6.0 post-event Result None regular Regular None None None [] None None 20:00 None None womens-champions-league TFBB16019 0 0 0 None None None None None loss Benfica Women Benfica Women Benfica Women None TFBB17702 1 0 1 None None None None None win BK Häcken Women BK Häcken Women BK Häcken Women None FULLTIME RESULT Caixa Futebol Campus Caixa Futebol Campus Caixa Futebol Campus Caixa Futebol Campus Portugal Women's Champions League UEFA Women's Champions League Women's Champs Lge
4 EFBO2245949 2021-11-10T20:00:00+00:00 False 90.0 4.0 post-event Result None regular Regular None None None [] None None 20:00 None None womens-champions-league TFBB9100 2 0 2 None None None None None win Lyon Féminines Lyon Féminines Lyon Féminines None TFBB6597 1 1 1 None None None None None loss FC Bayern München Ladies FC Bayern München Ladies FC Bayern München Ladies None FULLTIME RESULT Groupama Stadium Groupama Stadium Groupama Stadium Groupama Stadium France Women's Champions League UEFA Women's Champions League Women's Champs Lge
现在可以拉出你想要的列了：
results = event_df[['homeTeam.name.full','homeTeam.eventOutcome','awayTeam.name.full','awayTeam.eventOutcome']]
输出：
print(results.to_string())
homeTeam.name.full homeTeam.eventOutcome awayTeam.name.full awayTeam.eventOutcome
0 Kosovo loss Jordan win
1 Barcelona Femenino win TSG 1899 Hoffenheim Ladies loss
2 HB Køge Women loss Arsenal Women win
3 Benfica Women loss BK Häcken Women win
4 Lyon Féminines win FC Bayern München Ladies loss
5 Walsall loss Forest Green Rovers win
6 Exeter City win Bristol Rovers loss
7 Wigan Athletic win Shrewsbury Town loss
8 Broomhill loss Bo'ness United win
9 Cumbernauld Colts loss Rangers B win
10 Dalbeattie Star loss East Kilbride win
11 East Stirlingshire tie University of Stirling tie
12 Grêmio win Fluminense loss
13 Athletico Paranaense win Ceará loss
14 Atlético Mineiro win Corinthians loss
15 Santos win Red Bull Bragantino loss
16 Palmeiras win Atlético Goianiense loss
编辑：附加代码
import pandas as pd
data = [['Liverpool','Manchester U'],
['Burnley' ,'West Ham'],
['Arsenal' ,'Chelsea'],
['Tottenham','Brentford']]
scraped_data = pd.DataFrame(data, columns = ['Home', 'Away'])
data = [['Liverpool', 'Manchester U', 'Liverpool'],
['TestTeamA', 'TestTeamB', 'TestTeamA'],
['Arsenal', 'Chelsea', 'Chelsea']]
excel_data = pd.DataFrame(data, columns = ['Home', 'Away', 'Result'])
result = scraped_data.merge(excel_data, how='left', on=['Home','Away']).dropna(subset=['Result'])
输出：
print(result)
Home Away Result
0 Liverpool Manchester U Liverpool
2 Arsenal Chelsea Chelsea
【讨论】：
这个url的表格不是那么好操作的，我建议使用selenium：
import pandas as pd
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
options = webdriver.ChromeOptions()
options.add_argument('--no-default-browser-check')
options.add_argument('--log-level=3')
options.add_argument('--headless')
service = Service('chromedriver.exe')
driver = webdriver.Chrome(options=options, service=service)
date = input(str("Input the date, hit Enter for today: "))
driver.get("https://www.bbc.com/sport/football/scores-fixtures/" + date)
games = driver.find_elements(By.XPATH, "//article[@data-event-id]")
ListHome = []
ListAway = []
ListTime = []
for game in games:
ListHome.append(game.find_element(By.XPATH,".//span[contains(@class,'team--time-home')]").text)
ListAway.append(game.find_element(By.XPATH,".//span[contains(@class,'team--time-away')]").text)
df = pd.DataFrame(list(zip(ListHome, ListAway)), columns =['Home', 'Away'])
df.to_excel('name.xlsx', index=False)
使用 pd.merge 与您的其他电子表格进行比较。
【讨论】：
您好，Wilian，感谢您的回答，
我刚刚尝试了你给我的代码，但它在下面显示了这个错误，因为我从未使用过 selenium，所以我无法理解这个问题。
DevTools listening on ws://127.0.0.1:24083/devtools/browser/c0807040-1041-4a0e-a9ed-f1add52c5484
Input the date, hit Enter for today:
Traceback (most recent call last):
File "c:\Users\dgayg\Desktop\pyBet\testmatch.py", line 19, in <module>
ListHome.append(game.find_element(By.XPATH,".//span[contains(@class,'team--time-home')]").text)
File "C:\Users\dgayg\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 718, in find_element
return self._execute(Command.FIND_CHILD_ELEMENT,
File "C:\Users\dgayg\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webelement.py", line 693, in _execute
return self._parent.execute(command, params)
File "C:\Users\dgayg\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 418, in execute
self.error_handler.check_response(response)
File "C:\Users\dgayg\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 243, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":".//span[contains(@class,'team--time-home')]"}
(Session info: headless chrome=95.0.4638.69)
Stacktrace:
Backtrace:
Ordinal0 [0x00880C43+2493507]
Ordinal0 [0x0081A4B1+2073777]
Ordinal0 [0x00722608+1058312]
Ordinal0 [0x0074CAA4+1231524]
Ordinal0 [0x00743621+1193505]
Ordinal0 [0x0076597A+1333626]
Ordinal0 [0x007435A6+1193382]
Ordinal0 [0x00765A2A+1333802]
Ordinal0 [0x00775038+1396792]
Ordinal0 [0x0076580B+1333259]
Ordinal0 [0x00742314+1188628]
Ordinal0 [0x0074316F+1192303]
GetHandleVerifier [0x00A07BF6+1548950]
GetHandleVerifier [0x00AB461C+2256060]
GetHandleVerifier [0x0090C13B+518107]
GetHandleVerifier [0x0090B1E0+514176]
Ordinal0 [0x0081F53D+2094397]
Ordinal0 [0x00823418+2110488]
Ordinal0 [0x00823552+2110802]
Ordinal0 [0x0082CE81+2150017]
BaseThreadInitThunk [0x7518FA29+25]
RtlGetAppContainerNamedObjectPath [0x76FA7A9E+286]
RtlGetAppContainerNamedObjectPath [0x76FA7A6E+238]
非常感谢您的宝贵时间！
【讨论】：