导入带有“当其中一些包含”以及逗号的值的 CSV 文件

Question

我想我一直在搜索，但如果我遗漏了什么 - 请告诉我。

我正在尝试导入 CSV 文件，其中所有非数值都用“. 我遇到了一个问题：

 df = pd.read_csv(file.csv)

CSV 示例：

"Business focus","Country","City","Company Name"
"IT","France","Lyon","Societe General"
"Mining","Russia","Moscow","Company "MoscowMining" Owner1, Owner2, Owner3"
"Agriculture","Poland","Warsaw","Company" Jankowski,A,B""

由于其中包含多个引号和逗号，在这种情况下，pandas 看到的列数超过了 4 列（例如 5 或 6）。

我已经试过玩了

df = pd.read_csv(file.csv, quotechar='"', quoting=2)

得到了

ParserError: Error tokenizing data (...)

有效的方法是跳过坏行

error_bad_lines=False

但我宁愿以某种方式考虑所有数据，而不是忽略它。

非常感谢您的帮助！

Answer 1

这似乎是格式错误的 CSV 数据，因为值中的 '"' 字符应该被转义。我经常看到这些值通过将它们加倍或以 \ 作为前缀来转义。请参阅 https://en.wikipedia.org/wiki/Comma-separated_values#cite_ref-13

我要做的第一件事是修复导出这些文件的任何内容。但是，如果您不能这样做，您可以通过转义作为值一部分的 " 来解决此问题。

您最好的选择可能是假设一个 " 仅在它是一个值的结尾时才跟在（或前面）一个逗号或换行符。然后您可以执行类似的正则表达式（从内存中工作，所以可能不是100% - 但应该给你正确的想法。你必须适应任何你手边的正则表达式库）

s/([^,\n])"([^,\n])/$1""$2/g

因此，如果您要运行示例文件，尽管它会像这样被转义：

"Business focus","Country","City","Company Name"
"IT","France","Lyon","Societe General"
"Mining","Russia","Moscow","Company ""MoscowMining"" Owner1, Owner2, Owner3"
"Agriculture","Poland","Warsaw","Company"" Jankowski,A,B"""

或使用以下

s/([^,\n])"([^,\n])/$1\"$2/g

文件会像这样被转义：

"Business focus","Country","City","Company Name"
"IT","France","Lyon","Societe General"
"Mining","Russia","Moscow","Company \"MoscowMining\" Owner1, Owner2, Owner3"
"Agriculture","Poland","Warsaw","Company\" Jankowski,A,B\""

根据您的 CSV 解析器，其中之一应该被接受并按预期工作。

如果正如@exe 所建议的那样，您的 CSV 解析器还需要转义值中的逗号，您可以应用类似的正则表达式来替换逗号。

Answer 2

如果我理解你需要的是在熊猫读取 csv 之前加上引号和逗号。

像这些：

"Business focus","Country","City","Company Name"
"IT","France","Lyon","Societe General"
"Mining","Russia","Moscow","Company \"MoscowMining\" Owner1\, Owner2\, Owner3"
"Agriculture","Poland","Warsaw","Company\" Jankowski\,A\,B\""