csv 文件中字符串的不正确表示

Question

我在Win7，Python2.7。

有字符串。 原图：

A. P. Møller 马士基

UTF-8：

s = 'A. P. M\xc3\xb8ller M\xc3\xa6rsk'

我需要把它写成 csv。 试试这个：

with open('14.09 Anbefalte aksjer.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow([s])

知道了：

A. P. Møller Mærsk

尝试使用 UnicodeWriter：

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    def __init__(self, f, dialect=csv.excel, encoding="utf-8", **kwds):
        # Redirect output to a queue
        self.queue = StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

s = 'A. P. M\xc3\xb8ller M\xc3\xa6rsk'.decode('utf8')
with open('14.09 Anbefalte aksjer.csv', 'w') as csvfile:
    writer = UnicodeWriter(csvfile)
    writer.writerow([s])

又来了：

A. P. Møller Mærsk

试试 unicodecsv：

再说一遍：

A. P. Møller Mærsk

怎么了？怎样才能写对？

Answer 1

Windows 使用记事本或 Excel 等工具采用默认的 Window 区域设置编码，因此对于 UTF-8，字节顺序标记（BOM、U+FEFF）必须在文件开头进行编码。 Python为此提供了编码utf-8-sig。另请注意，通过使用#coding:utf8 并将源文件保存为 UTF-8，您可以将字符串直接声明为 Unicode 字符串。最后，与csv 模块一起使用的文件应在 Python 2.7 上以wb 的形式打开，否则您将看到在 Windows 上编写换行符时出现问题。

#coding:utf8
import csv
from StringIO import StringIO
import codecs

class UnicodeWriter:
    """
    A CSV writer which will write rows to CSV file "f",
    which is encoded in the given encoding.
    """

    # Use utf-8-sig encoding here.
    def __init__(self, f, dialect=csv.excel, encoding="utf-8-sig", **kwds):
        # Redirect output to a queue
        self.queue = StringIO()
        self.writer = csv.writer(self.queue, dialect=dialect, **kwds)
        self.stream = f
        self.encoder = codecs.getincrementalencoder(encoding)()

    def writerow(self, row):
        self.writer.writerow([s.encode("utf-8") for s in row])
        # Fetch UTF-8 output from the queue ...
        data = self.queue.getvalue()
        data = data.decode("utf-8")
        # ... and reencode it into the target encoding
        data = self.encoder.encode(data)
        # write to the target stream
        self.stream.write(data)
        # empty queue
        self.queue.truncate(0)

    def writerows(self, rows):
        for row in rows:
            self.writerow(row)

s = u'A. P. Møller Mærsk' # declare as Unicode string.
with open('14.09 Anbefalte aksjer.csv', 'wb') as csvfile:
    writer = UnicodeWriter(csvfile)
    writer.writerow([s])

输出：

A. P. Møller Mærsk

Answer 2

您看到的是 mojibake：表示以一种字符编码编码的 Unicode 文本的字节以另一种（不兼容的）字符编码显示。

如果''.decode('utf8') 没有引发AttributeError，则意味着您不在Python 3 上（尽管您的问题是这样说的）。在 Python 2 上，csv 不直接支持 Unicode，您必须手动编码：

#!/usr/bin/env python2
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

text = "A. P. Møller Mærsk"
with open('14.09 Anbefalte aksjer.csv', 'wb') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow([text.encode('utf-8')])

如果 text 包含未损坏的数据，UnicodeWriter 和 unicodecsv 模块应该也能正常工作。