从 csv 文件中读取数据，同时密切关注文件夹的更改答案

【问题标题】：Read the data from the csv file while keeping a watch on the folder for changes从 csv 文件中读取数据，同时密切关注文件夹的更改
【发布时间】：2020-01-25 15:07:15
【问题描述】：

我有一个代码可以监视文件夹的任何更改。它会检查新添加和删除的文件，并在发生此类情况时及时显示文件名。这是我的代码：

import os, time
import pandas as pd
import glob
path_to_watch = os.path.abspath('C:/Folder for violation csv/')
before = dict ([(f, None) for f in os.listdir (path_to_watch)])
while 1:
  time.sleep (2)
  after = dict ([(f, None) for f in os.listdir (path_to_watch)])
  added = [f for f in after if not f in before]
  removed = [f for f in before if not f in after]
  if added: print ("Added: ", ", ".join (added))
  if removed: print ("Removed: ", ", ".join (removed))
  before = after

现在我想在这段代码中再包含一件事，我想读取正在添加到文件夹中的文件中的数据。我尝试使用from_dict 将字典转换为DataFrame，然后使用pandas 使用read_csv 读取csv，但它似乎不起作用。这是我更新的代码：

import os, time
import pandas as pd
import glob
path_to_watch = os.path.abspath('C:/Folder for violation csv/')
before = dict ([(f, None) for f in os.listdir (path_to_watch)])
while 1:
  time.sleep (2)
  after = dict ([(f, None) for f in os.listdir (path_to_watch)])
  added = [f for f in after if not f in before]
  removed = [f for f in before if not f in after]
  if added: 
    print ("Added: ", ", ".join (added))
  df = pd.DataFrame.from_dict(path_to_watch, orient = 'index')
  df1 = pd.read_csv(df)
  if removed: print ("Removed: ", ", ".join (removed))
  before = after

如果您能帮我得到想要的结果，我将不胜感激。

【问题讨论】：

你为什么用from_dict？，希望你需要df = pd.read_csv(os.path.join(path_to_watch, added))
我尝试使用您的代码，但出现错误：TypeError: join() argument must be str or bytes, not 'list'。抱歉，如果这听起来像是业余错误，我是 python 新手

标签： python pandas csv dataframe dictionary

【解决方案1】：

首先，最好使用pathlib包进行访问文件系统。
使用set收集文件名。

from pathlib import Path

path_to_watch = Path('C:/Folder for violation csv/')
before = set(path_to_watch.glob('**/*'))
while 1:
  time.sleep (2)
  after = set(path_to_watch.glob('**/*')
  added = after - before
  removed = before - after
  for f in added:
     df = pd.read_csv(f)
  before = after

【讨论】：

我不知道为什么，但while 1: 行的语法无效。
我错过了一个右括号。前一阵就上线了。现在修好了。
嗨，@abhilb，我现在收到此错误：NotImplementedError: Non-relative patterns are unsupported
我不确定 csv 文件的内容？您在读取 csv 文件时遇到错误吗？错误在哪一行。
我也忘了设置 before = after.. 现在添加了那行。

【解决方案2】：

试试这个，

import os, time
import pandas as pd
import glob
path_to_watch = os.path.abspath('C:/Folder for violation csv/')
before = dict ([(f, None) for f in os.listdir (path_to_watch)])
while 1:
  time.sleep (2)
  after = dict ([(f, None) for f in os.listdir (path_to_watch)])
  added = [f for f in after if not f in before]
  removed = [f for f in before if not f in after]
  if added: 
    print ("Added: ", ", ".join (added))
  for file_ in added:
    df = pd.read_csv(os.path.join(path_to_watch, file_))
  if removed: print ("Removed: ", ", ".join (removed))
  before = after

【讨论】：