解析csv文件并根据相对大小将行写入文件

Question

我有一长串不同时间的天气变量，我已经过滤了这些变量以删除不符合某些标准的变量。例如，所有数据点仅位于上午 11 点（11 点）和下午 5 点（17 点）之间。 11 点到 17 点之间的数据代表一个事件，并非每天都包含一个事件。我正在尝试确定哪些日子展示了一个事件。我知道一个新事件正在发生，时间 (HH24) 列中的每个值都比之前的值低。例如。如果值 16（下午 5 点）后跟 11、12、13、14 或 15，则我知道数据已进入新的一天和新事件。

我正在尝试编写的代码将获取包含最后一个值（例如 17）的行并将其写入文件，然后写入下一行。这样，新的 csv 文件将包含每个事件的开始时间（和其他信息）和结束时间。我假设我需要使用 for 命令，但我不确定如何使用 csv writer 来应对这一特殊挑战。我的代码大纲如下，我需要帮助的部分遵循#

import csv

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input, open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted:
    reader = csv.DictReader(input, delimiter=",", skipinitialspace=True)
    fieldnames = reader.fieldnames
    writer_wanted = csv.DictWriter(wanted, fieldnames, delimiter=",")
    writer_wanted.writeheader()

    for line_number, row in enumerate(reader):
        try:
            if float(row["HH24"]) < #the value in the subsequent row:
                writer_wanted.writerow(row) # and also write subsequent row
        except:
            print "Failed to parse line", line_number
            print row

我的数据文件如下所示。我已经展示了 HH24 从高值到低值的过渡，所以你可以明白我的意思。

hd,Station Number,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Local standard time,Year Month Day Hours Minutes in YYYY,MM,DD,HH24,MI format in Universal coordinated time,Precipitation since last (AWS) observation in mm,Quality of precipitation since last (AWS) observation value,Air Temperature in degrees Celsius,Quality of air temperature,Air temperature (1-minute maximum) in degrees Celsius,Quality of air temperature (1-minute maximum),Air temperature (1-minute minimum) in degrees Celsius,Quality of air temperature (1-minute minimum),Wet bulb temperature in degrees Celsius,Quality of Wet bulb temperature,Wet bulb temperature (1 minute maximum) in degrees Celsius,Quality of wet bulb temperature (1 minute maximum),Wet bulb temperature (1 minute minimum) in degrees Celsius,Quality of wet bulb temperature (1 minute minimum),Dew point temperature in degrees Celsius,Quality of dew point temperature,Dew point temperature (1-minute maximum) in degrees Celsius,Quality of Dew point Temperature (1-minute maximum),Dew point temperature (1 minute minimum) in degrees Celsius,Quality of Dew point Temperature (1 minute minimum),Relative humidity in percentage %,Quality of relative humidity,Relative humidity (1 minute maximum) in percentage %,Quality of relative humidity (1 minute maximum),Relative humidity (1 minute minimum) in percentage %,Quality of Relative humidity (1 minute minimum),Wind (1 minute) speed in km/h,Wind (1 minute) speed quality,Minimum wind speed (over 1 minute) in km/h,Minimum wind speed (over 1 minute) quality,Wind (1 minute) direction in degrees true,Wind (1 minute) direction quality,Standard deviation of wind (1 minute),Standard deviation of wind (1 minute) direction quality,Maximum wind gust (over 1 minute) in km/h,Maximum wind gust (over 1 minute) quality,Visibility (automatic - one minute data) in km,Quality of visibility (automatic - one minute data),Mean sea level pressure in hPa,Quality of mean sea level pressure,Station level pressure in hPa,Quality of station level pressure,QNH pressure in hPa,Quality of QNH pressure,#
    hd,40842,2000,3,22,13,40,2000,3,22,13,40,2000,3,22,13,40,0,N,20.4,N,20.5,N,20.4,N,20.2,N,20.2,N,20.1,N,20.1,N,20.1,N,20,N,98,N,,N,,N,9,N,8,N,18,N,7,N,11,N,,N,1013.3,N,1012.2,N,1013.3,N,#
    hd,40842,2000,3,22,13,47,2000,3,22,13,47,2000,3,22,13,47,0,N,20.5,N,20.5,N,20.5,N,20.2,N,20.2,N,20.2,N,20.1,N,20.1,N,20,N,97,N,,N,,N,4,N,0,N,56,N,75,N,5,N,,N,1013.2,N,1012.1,N,1013.2,N,#
    hd,40842,2000,3,23,11,0,2000,3,23,11,0,2000,3,23,11,0,0,N,23.4,N,23.4,N,23.3,N,21.3,N,21.4,N,21.3,N,20.2,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,66,N,2,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#
    hd,40842,2000,3,23,11,1,2000,3,23,11,1,2000,3,23,11,1,0,N,23.4,N,23.4,N,23.4,N,21.4,N,21.4,N,21.3,N,20.3,N,20.3,N,20.2,N,82,N,,N,,N,8,N,5,N,68,N,3,N,9,N,,N,1013.6,N,1012.5,N,1013.6,N,#

Answer 1

您想在日期更改时写一行，所以我认为最好创建一个日期变量进行比较。 （我在评论中提到了为什么仅比较“HH24”值无法确定您何时到达新日期）

跟踪和写出上一个行（因为您已经处理过它）比后续行要容易得多，所以这就是您应该思考的方式关于进行。 类似以下的内容应该会有所帮助（未经测试）：

...
import datetime
prev_row = None
for line_number, row in enumerate(reader):
    try:
        dt = datetime.date(year=row["Year"], month=row["Month"], day=row["day"])
        if prev_row is not None and dt > prev_row['dt']:
            writer_wanted.writerow(prev_row['row'])
            writer_wanted.writerow(row)
        prev_row = {'row':row, 'dt':dt}
    except:
        print "Failed to parse line", line_number
        print row

编辑：

你的程序中这一行的第一部分：

with open("weather_out_2000_2006_time_filtered_and_speed_filtered.csv", "rb") as input

打开命名的.csv 文件进行输入（因为mode 是'rb' - docs here）。

同一行的下一部分：

open("X:\weatherresults\seabreezeevents.csv", "wb") as wanted

打开该命名文件以进行输出（'wb' 模式） - 请参阅与上述相同的参考。

此时，变量名input和wanted现在都引用了file type的对象。

您的程序接下来使用csv 模块以帮助解析逗号分隔文本文件的特定方式读取文件；并将该引用分配给reader 变量。 同样，它将变量writer_wanted 分配给csv.DictWriter，这将有助于在将行写入输出文件时进行格式化，由wanted 引用。

之后，每次读取一行：

for line_number, row in enumerate(reader):

一次一行将由以下人员写出：

writer_wanted.writerow(row)

如果您想了解更多细节，最好的办法是阅读一些 Python 教程（Google 是您的朋友）。