【发布时间】:2020-03-07 12:48:30
【问题描述】:
我正在尝试从“日期”列中的日期在 2019 年 1 月 11 日之前的数据框中删除所有行
数据框是通过抓取谷歌新闻(标题、日期、链接、出版商)生成的。完整代码如下:
from bs4 import BeautifulSoup
import requests
import html5lib
import pandas as pd
import datetime
headers = {'User-Agent': 'Mozilla/5.0'}
#URL Generator (scraping news for 'sega')
urlA= 'https://news.google.com/search?q='
urlB='sega'
urlC='&hl=en-US&gl=US&ceid=US%3Aen'
url=urlA+urlB+urlC
response=requests.get(url)
soup=BeautifulSoup(response.content,'html5lib')
print(soup)
T=[]
t=[]
L=[]
P=[]
#Collecting Data
for x in soup.find_all(class_='ipQwMb ekueJc RD0gLb'):
title=x.text
T.append(title)
print(title)
for r in soup.find_all(class_='SVJrMe'):
z=r.find('time')
if z is not None:
for y in r.find_all('time'):
time=y.get('datetime')
time=str(time).partition('T')
time=time[0]
time = datetime.datetime.strptime(time, "%Y-%m-%d").date()
print(time)
t.append(time)
else:
x='Not Specified'
t.append(x)
for z in soup.find_all(class_='VDXfz'):
links=z.get('href')
links =links[1::] #removing the dot (first character always a
dot in links which is not required)
urlx= 'https://news.google.com'
links= urlx+links
L.append(links)
for w in soup.find_all(class_='wEwyrc AVN2gc uQIVzc Sksgp'):
publisher = w.text
P.append(publisher)
#Checking length to see all is equal
print(len(T))
print(len(t))
print(len(P))
print(len(L))
df=pd.DataFrame({'Title':(T) , 'Date':(t), 'Publisher' : (P), 'Link': (L)})
print(df)
这是当前输出(仅前 12 行):
如您所见,数据框包含 11 月份之前的日期,我想做的是删除所有这些行。我已经将日期列转换为“dateTIME”格式(参见代码 [for r in soup.find...time=datetime.datetime.strip ....]。
请建议添加代码行以实现所需功能。如果需要任何说明,请告诉我。
【问题讨论】: