在 read_csv() python 中出现 StopIteration 错误答案

【问题标题】：Getting StopIteration error in read_csv() python在 read_csv() python 中出现 StopIteration 错误
【发布时间】：2016-11-20 18:32:11
【问题描述】：

我有一个名为 summaryDF 的数据框，如下所示（为 MCVE 修改了标题）：

         f1         precision recall  threshold
A        0.600321   0.739130  0.111   0.001
B        0.096692   0.826087  0.2321  0.001
C        0.072539   0.875000  0.4531  0.001
D        0.128079   0.722222  0.2311  0.001

然后我正在运行一个批处理脚本，该脚本运行一个机器学习管道，其中包含 3 个具有不同 threshold 超参数的文件，并且在第三个文件的末尾，我想将精度、F1 和召回结果附加到文件的底部。因此，例如，下一次迭代应该使文件看起来像：

         f1         precision recall  threshold
A        0.600321   0.739130  0.111   0.001
B        0.096692   0.826087  0.2321  0.001
C        0.072539   0.875000  0.4531  0.001
D        0.128079   0.722222  0.2311  0.001
A        0.121211   0.312214  0.3213  0.005
B        0.214422   0.432332  0.2311  0.005
C        0.081111   0.121211  0.1322  0.005
D        0.012121   0.212444  0.6546  0.005

这是我当前的代码：

with open(data/output/test/summaryEval.csv, 'w+') as summaryCSV:
    try:
        df_csv = pd.read_csv(summaryCSV,encoding='utf-8',engine='python')
    except ValueError as v:
        print v
        summaryDF.to_csv(path_or_buf=summaryCSV, encoding='utf-8')
        summaryCSV.close()
    else:
        summaryDF = pd.concat([df_csv,summaryDF],axis=1)
        summaryDF.to_csv(path_or_buf=summaryCSV,encoding='utf-8',mode='a')
        summaryCSV.close()

出了什么问题？错误是：

Traceback (most recent call last):
  File "src/main/logisticBagOfWords.py", line 477, in <module>
    df_csv = pd.read_csv(summaryCSV,encoding='utf-8',engine='python')
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 474, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 250, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 566, in __init__
    self._make_engine(self.engine)
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 711, in _make_engine
    self._engine = klass(self.f, **self.options)
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 1429, in __init__
    self.columns, self.num_original_columns = self._infer_columns()
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 1644, in _infer_columns
    line = self._buffered_line()
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 1771, in _buffered_line
    return self._next_line()
  File "/Users/dhruv/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 1802, in _next_line
    orig_line = next(self.data)
StopIteration

【问题讨论】：

标签： python-2.7 csv parsing dataframe append

【解决方案1】：

问题是以相同的模式打开文件，无论是否有文件与文件中已经存在信息。因此，我根据这些情况以不同的模式打开。

    try:
        if os.stat(sys.argv[5]).st_size > 0:
            with open(sys.argv[5], 'a') as f:
                summaryDF.to_csv(path_or_buf=f,encoding='utf-8',mode='a',header=False)
                f.close()
        else:
            print "empty file"
            with open(sys.argv[5], 'w+') as f:
                summaryDF.to_csv(path_or_buf=f,encoding='utf-8')
                f.close()
    except OSError:
        print "No file"
        with open(sys.argv[5], 'w+') as f:
            summaryDF.to_csv(path_or_buf=f,encoding='utf-8')
            f.close()

【讨论】：