解析不可读的字符

Question

我正在尝试解析 API 响应，但是响应数据包含给 python 一些麻烦的字符。

API 响应：electricity price | 19.52¢/kW·h (January 1, 2014) natural gas price | $11.05 per thousand cubic feet (January 15, 2014) heating oil price | $4.338/gal (March 17, 2014) propane price | $3.968/gal (March 17, 2014)

在“每千瓦时的美分”字符处引发错误。

完全错误：UnicodeEncodeError: 'ascii' codec can't encode character u'\xa2' in position 25: ordinal not in range(128)

显示在终端中的响应：electricity price | 19.52\xa2/kW\xb7h (January 1, 2014)\nnatural gas price | $11.05 per thousand cubic feet (January 15, 2014)\nheating oil price | $4.338/gal (March 17, 2014)\npropane price | $3.968/gal (March 17, 2014)

我将如何解析这些问题字符周围的数据？我不需要全文，只需要其中的数值。感谢您的帮助。

编辑：

导致错误的代码：

search('electricity price | {:d}', energy)

我也试过了：

search('electricity price | {:f}', energy)

结果相似。 energy 是一个存储上面列出的完整字符串的变量。

编辑 2：

包含 API 调用的完整代码：

client = wolframalpha.Client('apikey')
energy_query = 'utilities prices in ' + city + ' ' + state_abbr
res = client.query(energy_query)


energy = (next(res.results).text)

search('electricity price | {:d}', energy)

完整的追溯：

File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-    packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site- packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/nulife.py", line 120, in index
search('electricity price | {:d}', energy)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-    packages/parse.py", line 1041, in search
return Parser(format, extra_types=extra_types).search(string, pos, endpos)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-    packages/parse.py", line 678, in search
return self._generate_result(m)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-    packages/parse.py", line 699, in _generate_result
fixed_fields[n] = self._type_conversions[n](fixed_fields[n], m)
File "/Users/aaronpardes/Dropbox/Python/nuLife2/newlifenv/lib/python2.7/site-packages/parse.py", line 375, in f
if string[0] == '-':
TypeError: 'NoneType' object has no attribute '__getitem__'

Answer 1

energy 已经是一个 Unicode 对象；尝试在其上调用 .decode() 会首先触发隐含 encode（使用 ASCII，默认编解码器）：

>>> energy = u'19.52¢/kW·h'
>>> energy.decode('windows-1252')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/encodings/cp1252.py", line 15, in decode
    return codecs.charmap_decode(input,errors,decoding_table)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xa2' in position 5: ordinal not in range(128)

注意异常；解码 Unicode 字符串触发了UnicodeEncodeError。

正如设计的那样，wolframalpha library 使用 ElementTree 来解析 XML 响应，它总是为您提供 Unicode 对象。

在你更新后，我查看了parse library source code我担心你发现代码中有错误；它们不会在您提交的文字字符串中转义正则表达式元字符。如果您转义 | 字符它可以工作：

>>> search('electricity price \\| {:f}', u'electricity price | 19.52¢/kW·h')
<Result (19.52,) {}>

我已经打开了一个 bug report 和 parse 项目关于这个。

请注意，该库可能仅限于解析 ASCII 文本；至少不要尝试将¢/kW·h 匹配为单词字符。

更新：parse version 1.6.4 已发布，修复了此特定错误。