ElementTree Unicode 编码/解码错误答案

【问题标题】：ElementTree Unicode Encode/Decode ErrorElementTree Unicode 编码/解码错误
【发布时间】：2014-06-26 03:06:52
【问题描述】：

对于一个项目，我应该增强一些 XML 并将其存储在一个文件中。我遇到的问题是我不断收到以下错误：

Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Bart\Dropbox\Studie\2013-2014\BSc-KI\cite_parser\parser.py", line 193, in parse_references
    outputXML = ET.tostring(root, encoding='utf8', method='xml')
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 1126, in tostring
    ElementTree(element).write(file, encoding, method=method)
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 820, in write
    serialize(write, self._root, encoding, qnames, namespaces)
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 939, in _serialize_xml
    _serialize_xml(write, e, encoding, qnames, None)
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 939, in _serialize_xml
    _serialize_xml(write, e, encoding, qnames, None)
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 939, in _serialize_xml
    _serialize_xml(write, e, encoding, qnames, None)
 ECLI:NL:RVS:2012:BY1564
 File "C:\Python27\lib\xml\etree\ElementTree.py", line 937, in _serialize_xml
    write(_escape_cdata(text, encoding))
  File "C:\Python27\lib\xml\etree\ElementTree.py", line 1073, in _escape_cdata
    return text.encode(encoding, "xmlcharrefreplace")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 80: ordinal not in range(128)

该错误由以下人员生成：

outputXML = ET.tostring(root, encoding='utf8', method='xml')

在寻找此问题的解决方案时，我发现了一些建议，说我应该将 .decode('utf-8') 添加到函数中，但这会导致写入函数出现编码错误（首先是解码），因此无法正常工作。 .

编码错误：

Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Bart\Dropbox\Studie\2013-2014\BSc-KI\cite_parser\parser.py", line 197, in parse_references
    myfile.write(outputXML)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xeb' in position 13559: ordinal not in range(128)

由以下代码生成：

outputXML = ET.tostring(root, encoding='utf8', method='xml').decode('utf-8')

来源（或至少相关部分）：

# URL encodes the parameters
encoded_parameters = urllib.urlencode({'id':ecli})

# Opens XML file
feed = urllib2.urlopen("http://data.rechtspraak.nl/uitspraken/content?"+encoded_parameters, timeout = 3)

# Parses the XML
ecliFile = ET.parse(feed)

# Fetches root element of current tree
root = ecliFile.getroot()

# Write the XML to a file without any extra indents or newlines
outputXML = ET.tostring(root, encoding='utf8', method='xml')

# Write the XML to the file
with open(file, "w") as myfile:
    myfile.write(outputXML)

最后但并非最不重要的是 XML 示例的 URL：http://data.rechtspraak.nl/uitspraken/content?id=ECLI:NL:RVS:2012:BY1542

【问题讨论】：

异常的完整回溯是什么？我敢打赌，触发它的不是 ElementTree 本身。
我刚刚为这两个异常添加了完整的回溯 :)
我无法重现该问题，无论如何 Python 2.7.6 都无法重现。
UnicodeDecodeError 不合适；这意味着树中有 字节字符串数据，而不是预期的 Unicode。您是否操纵了树，添加了元素？如果是这样，请确保添加 Unicode 字符串，而不是字节字符串。
我已将其作为答案；它可能对遇到此异常的其他人有所帮助。

标签： python python-2.7 unicode elementtree

【解决方案1】：

异常是由字节字符串值引起的。

回溯中的

text 应该是一个 unicode 值，但如果它是一个纯字节字符串，Python 将首先将其隐式解码（使用 ASCII 编解码器）为 Unicode，这样您就可以 encode 再次。

是解码失败了。

因为您实际上并未向我们展示您在 XML 树中插入了什么，所以很难告诉您要修复什么，除了确保在插入文本时始终使用 Unicode 值。

演示：

>>> root.attrib['oops'] = u'Data with non-ASCII codepoints \u2014 (em dash)'.encode('utf8')
>>> ET.tostring(root, encoding='utf8', method='xml')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/xml/etree/ElementTree.py", line 1126, in tostring
    ElementTree(element).write(file, encoding, method=method)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/xml/etree/ElementTree.py", line 820, in write
    serialize(write, self._root, encoding, qnames, namespaces)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/xml/etree/ElementTree.py", line 932, in _serialize_xml
    v = _escape_attrib(v, encoding)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/xml/etree/ElementTree.py", line 1090, in _escape_attrib
    return text.encode(encoding, "xmlcharrefreplace")
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 31: ordinal not in range(128)
>>> root.attrib['oops'] = u'Data with non-ASCII codepoints \u2014 (em dash)'
>>> ET.tostring(root, encoding='utf8', method='xml')
'<?xml version=\'1.0\' encoding=\'utf8\'?> ...'

设置 bytestring 属性，包含 ASCII 范围之外的字节，会触发异常；使用 unicode 值来确保可以产生结果。

【讨论】：