当使用标准输入而不是命令行参数时，编码 utf-8 错误

Question

我需要使我的脚本可用于 UNIX 命令，以便可以使用我的脚本一次处理 2 个输入文件。此脚本与命令行参数完美配合：

newlist = []
def f1()
 .... 
  return places 
  return persons
  return unknown

def f2(input_file):

  volume_id = sys.argv[3]   

  for line in input_data:  

     if any(place+'</dfn>' in line.decode('utf-8') for place in places):
         line = line.replace('"person"', '"place"')
         line = line.replace('id="', 'id="'+volume_id)
     elif any(unk+'</dfn>' in line.decode('utf-8') for unk in unknown):
         line = line.replace('"person"', '"undefined"')
         line = line.replace('id="', 'id="'+volume_id)
     elif 'class="person"' in line.decode('utf-8') and '<dfn' not in line:
         line = line.replace('class="person"', '')
         line = line.replace('id="', 'id="'+volume_id)
     elif 'id="' in line:
         line = line.replace('id="', 'id="'+volume_id)

     newlist.append(line)

  return  newlist                

def main():
   if len(sys.argv) < 4:
     print 'usage: ./myscript.py [file_in... file_out... volume_id]'
     sys.exit(1)

   else:

    filename = sys.argv[1]
    filename_out = sys.argv[2]

    tree = etree.parse(filename)
    extract(tree)

    input_file = open(filename, 'rU')
    change_class(input_file)

    file_new = open(filename_out, 'w')
    for x in newlist:

      if '\n' in x:                   
         x = x.replace('\n', '')                
      print>>file_new, x

当我尝试向它添加标准输入标准输出时，我首先遇到了先读取相同输入文件的问题，因此进行了一些更改，以便它实际上只打开一次。我修改了以下内容：

  def f2(input_data) #instead of input_file

我修改了 main():

        filename = sys.argv[1]
        filename_out = sys.argv[2]

        if filename == '-':
           input_file = sys.stdin

        else:
            input_file = open(filename, 'rU')


        if filename_out == '-':
            filename_out = sys.stdout
            file_new = filename_out
        else:
            file_new = open(filename_out, 'w')

        input_data = input_file.read()

        tree = etree.fromstring(input_data)
        extract(tree)


        change_class(input_data)

        for x in newlist:

            if '\n' in x:                   
               x = x.replace('\n', '')                
            print>>file_new, x

我从命令行运行程序： ./myscript.py --volumeid 输出文件

现在我遇到了编码问题：

Traceback (most recent call last):
  File "./exportXMLstd.py", line 192, in <module>
    main()
  File "./exportXMLstd.py", line 182, in main
    change_class(input_data)
  File "./exportXMLstd.py", line 135, in change_class
    if any(place+'</dfn>' in line.decode('utf-8') for place in places):
  File "./exportXMLstd.py", line 135, in <genexpr>
    if any(place+'</dfn>' in line.decode('utf-8') for place in places):
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe2 in position 0: unexpected end of data

我做错了什么？

Answer 1

这个怎么样：

if filename == '-':
    input_file = sys.stdin

else:
    input_file = open(filename, 'rb')

tree = etree.fromstring(input_file.read())
...

我认为 XML 源很可能是 utf-8（无论它来自标准输入还是来自文件）