【问题标题】:use re when reading csv file读取 csv 文件时使用 re
【发布时间】:2018-05-30 07:23:04
【问题描述】:

我有一个关键字 healthy_list 列表,我想在 csv 文件的列中进行检查。如果列表中至少有一个关键字出现,那么我将整行写入一个新的 csv 文件。

我使用 re.search 检查关键字,然后记录行号,然后使用 csv.writer 编写新的 csv。但是包含关键字的许多行没有显示在我的新 csv 文件中。请问有cmets吗?

healthy_new=[]
with open("Data 2017.csv","rb") as f:
    csvreader=csv.reader(f,delimiter=",")
    next(csvreader)
    for line, row in enumerate(csvreader):
        for word in healthy_list:
            try:
                if  (re.search(word,row[4].lower()) ):
                    healthy_new.append(line)
            except ValueError:
                continue 

healthy_new=list(set(healthy_new))

....

f = open("Data 2017.csv", "r")
reader = csv.reader(f)

data = open("healthy_new_output.csv", "w")
w = csv.writer(data, delimiter=',')
for idx, row in enumerate(reader):
    idx+=-1
    if idx in healthy_new:
        my_row = row
        w.writerow(my_row)

编辑: 一些数据切片 2017.csv Data 2017.csv

健康列表:

 [...'diet', 'low-fat', 'light', 'diet', 'salad', 'salads', 'baked', 'grilled', 'whole grain']

【问题讨论】:

  • 我们能得到'Data 2017.csv'healthy_list的例子吗?
  • 如果你只想要一个包含测试的字符串,为什么还要使用re
  • @Megalng 这是迄今为止我学到的唯一方法...您建议哪种方法?

标签: python regex csv


【解决方案1】:

如果需要,您可以使用 pandas 将它们过滤掉,然后使用pandas.DataFrame.to_csv 方法将其输出到 csv。

以下是其工作原理的基本说明:

数据 2017.csv

name,age,description
Andy,15,Having a bad stomach
Bobby,21,Having a good stomach and a little flu
Connie,22,Not having anything particularly bad
Derry,12,Bad stomach & lightheaded

这是如何工作的基本说明:

In []: df = pd.read_csv('Data 2017.csv')

In []: word_flags = ['bad', 'flu', 'lightheaded']

In []: df_filtered = df.loc[:, :][df.description.str.contains("|".join(word_flags), re.IGNORECASE)]

In []: df_filtered
Out[]: 
     name  age                             description
0    Andy   15                    Having a bad stomach
1   Bobby   21  Having a good stomach and a little flu
2  Connie   22    Not having anything particularly bad
3   Derry   12               Bad stomach & lightheaded

In []: word_flags = ['flu', 'foo', 'bar']

In []: df_filtered = df.loc[:, :][df.description.str.contains("|".join(word_flags), re.IGNORECASE)]

In []: df_filtered
Out[]: 
    name  age                             description
1  Bobby   21  Having a good stomach and a little flu

df_filtered.to_csv("Filtered Data 2017.csv", index=False)

所以现在你有了这个:

name,age,description
Bobby,21,Having a good stomach and a little flu

要具体解决您的问题,请参见下面的代码段:

In []: word_flags = ['bad', 'flu', 'lightheaded']

In []: df2 = pd.DataFrame()

In []: for col in df.select_dtypes(object):
    ...:     df2 = pd.concat([df2, df[df[col].str.contains("|".join(word_flags), flags=re.IGNORECASE)]])
    ...:     

In []: df2
Out[]: 
     name  age                             description
0    Andy   15                    Having a bad stomach
1   Bobby   21  Having a good stomach and a little flu
2  Connie   22    Not having anything particularly bad
3   Derry   12               Bad stomach & lightheaded

In []: word_flags = ['flu', 'foo', 'bar']

In []: df2 = pd.DataFrame()

In []: for col in df.select_dtypes(object):
    ...:     df2 = pd.concat([df2, df[df[col].str.contains("|".join(word_flags), flags=re.IGNORECASE)]])
    ...:     

In []: df2
Out[]: 
    name  age                             description
1  Bobby   21  Having a good stomach and a little flu

但是,此方法仅在您将过滤器定义为仅过滤掉特定列时才有效。假设您这样定义word_flags

In []: word_flags = ['flu', 'foo', 'bar', 'bobby']

这会产生重复记录,需要进一步清理。

In []: df2 = pd.DataFrame()

In []: for col in df.select_dtypes(object):
    ...:     df2 = pd.concat([df2, df[df[col].str.contains("|".join(word_flags), flags=re.IGNORECASE)]])
    ...:     

In []: df2
Out[]: 
    name  age                             description
1  Bobby   21  Having a good stomach and a little flu
1  Bobby   21  Having a good stomach and a little flu

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2019-02-21
    • 1970-01-01
    • 2021-10-02
    • 2018-05-24
    • 2015-06-12
    • 1970-01-01
    • 2011-10-28
    相关资源
    最近更新 更多