【发布时间】:2017-05-11 22:18:22
【问题描述】:
我编写了一个脚本,该脚本将访问网站并将数据下载到 csv。然后它会打开 pandas 数据框 a 并对将保存为 HTML 并通过电子邮件发送的数据进行排序。
import csv
import requests
from datetime import datetime, time, timedelta
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import pandas as pd
from bs4 import BeautifulSoup
# code to calculate the date parameters for pulling in wex data
date = {}
today = datetime.today()
end_day_delta = datetime.today() + timedelta(days=1)
start_day_delta = datetime.today() - timedelta(days=5)
date['today'] = today.strftime("%m/%d/%Y")
yesterday = datetime.today() - timedelta(days=5)
date['yesterday'] = yesterday.strftime("%m/%d/%Y")
end_year = date['end_year'] = end_day_delta.strftime("%Y")
end_month = date['end_month'] = end_day_delta.strftime("%m")
end_day = date['end_day'] = today.strftime("%d")
start_year = date['start_year'] = start_day_delta.strftime("%Y")
start_month = date['start_month'] = start_day_delta.strftime("%m")
start_day = date['start_day'] = start_day_delta.strftime("%d")
# open a connection to wex and download data to csv files
with requests.Session() as c:
url = 'https://www.wexonline.com/fleetpilot/main/fork_logon.cfm'
user_name = 'my_user_name'
pass_word = 'my_password'
c.get(url)
cfttoken = c.cookies['CFTOKEN']
login_data = dict(csrfmiddlewaretoken=cfttoken, login_id=user_name,
pword_id=pass_word)
c.post(url, data=login_data, headers={'Referer':
'https://www.wexonline.com/fleetpilot/main/logon.cfm'})
url1 = 'https://www.wexonline.com/fleetpilot/query/VehicleDownload.cfm?
page1 = c.get(url1)
file = 'card_data.csv'
with open(file, 'w') as temp_file:
temp_file.writelines(page1.text)
url2 = 'https://www.wexonline.com/fleetpilot/query/DriverDownload.cfm?
page2 = c.get(url2)
file = 'driver_data.csv'
with open(file, 'w') as temp_file:
temp_file.writelines(page2.text)
urla = 'https://www.wexonline.com/fleetpilot/query/TransactionDownload.cfm?
parm= 'more params'
urlb = '%2F'
urlc = '%2F'
urld = 'ugly url'
urle = '%2F'
urlf = '%2F'
urlg = 'ugly url'
page3 = c.get(urla + start_month + urlb + start_day + urlc + start_year + \
urld + end_month + urle + end_day + urlf + end_year + urlg)
file = 'fuel_data.csv'
with open(file, 'w') as temp_file:
temp_file.writelines(page3.text)
# more code below
请忽略代码的非 Python 特性。第一个目标是编写一些可以工作的代码(还有很长的路要走),然后学习如何让它更 Pythonic。网址也需要编码。你在哪里看到“丑陋的代码”,我把它放在了那里,因为 URL 真的很长,而且在这里更难阅读。
程序应采取的步骤:
- 登录网站
- 下载 3 个数据集
- 将这些保存到 csv
- 将这 3 个数据集打开到 pandas 数据帧中
- 打开一个静态数据集
- 合并、过滤并将这些数据集保存到 html
- 通过电子邮件发送 html
当我使用 Windows 10 任务计划程序朗姆酒时,它不会执行第 3 步。它也可能不会执行第 2 步。
这里是任务计划程序设置
我只编程了几个月,所以我不确定如何进一步解决问题。
【问题讨论】:
标签: python python-3.x pandas task