csv中的双引号元素无法用熊猫读取

Question

我有一个输入文件，其中每个值都存储为字符串。 它位于一个 csv 文件中，每个条目都包含在双引号中。

示例文件：

"column1","column2", "column3", "column4", "column5", "column6"
"AM", "07", "1", "SD", "SD", "CR"
"AM", "08", "1,2,3", "PR,SD,SD", "PR,SD,SD", "PR,SD,SD"
"AM", "01", "2", "SD", "SD", "SD"

只有六列。我需要在 pandas read_csv 中输入哪些选项才能正确读取？

我目前正在尝试：

import pandas as pd
df = pd.read_csv(file, quotechar='"')

但这给了我错误信息： CParserError: Error tokenizing data. C error: Expected 6 fields in line 3, saw 14

这显然意味着它忽略了 '"' 并将每个逗号解析为一个字段。 但是，对于第 3 行，第 3 到第 6 列应该是带有逗号的字符串。 ("1,2,3", "PR,SD,SD", "PR,SD,SD", "PR,SD,SD")

如何让 pandas.read_csv 正确解析？

谢谢。

Answer 1

这会奏效。它回退到 python 解析器（因为你有非常规分隔符，例如它们是逗号，有时是空格）。如果你只有逗号，它会使用 c-parser 并且更快。

In [1]: import csv

In [2]: !cat test.csv
"column1","column2", "column3", "column4", "column5", "column6"
"AM", "07", "1", "SD", "SD", "CR"
"AM", "08", "1,2,3", "PR,SD,SD", "PR,SD,SD", "PR,SD,SD"
"AM", "01", "2", "SD", "SD", "SD"

In [3]: pd.read_csv('test.csv',sep=',\s+',quoting=csv.QUOTE_ALL)
pandas/io/parsers.py:637: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators; you can avoid this warning by specifying engine='python'.
  ParserWarning)
Out[3]: 
     "column1","column2" "column3"   "column4"   "column5"   "column6"
"AM"                "07"       "1"        "SD"        "SD"        "CR"
"AM"                "08"   "1,2,3"  "PR,SD,SD"  "PR,SD,SD"  "PR,SD,SD"
"AM"                "01"       "2"        "SD"        "SD"        "SD"