UnicodeDecodeError 发生在 .p 文件中（pickle）

Question

我给出了 .p 文件的路径并尝试加载该文件。 但是发生了这个错误 “UnicodeDecodeError：‘ascii’编解码器无法解码位置 0 的字节 0xf0：序数不在范围内 (128)”

def main(params):

  # load the checkpoint
  checkpoint_path = params['checkpoint_path']
  print ('loading checkpoint %s' % (checkpoint_path, ))
  #with open(checkpoint_path, 'rb') as pickle_file:
  #    checkpoint = pickle.load(pickle_file)
  checkpoint = pickle.load(open(checkpoint_path, 'rb'))


if __name__ == "__main__":
  parser = argparse.ArgumentParser()
  parser.add_argument('-checkpoint_path', default="D:\\neuraltalk\\cv\\flickr8k_cnn_lstm_v1.p", type=str, help='the input checkpoint')
  parser.add_argument('-r', '--root_path', default='D:\intermediate-cnn-features\images', type=str, help='folder with the images, tasks.txt file, and corresponding vgg_feats.mat file')
  parser.add_argument('-b', '--beam_size', type=int, default=1, help='beam size in inference. 1 indicates greedy per-word max procedure. Good value is approx 20 or so, and more = better.')

  args = parser.parse_args()
  params = vars(args) # convert to ordinary dict
  print ('parsed parameters:')
  print (json.dumps(params, indent = 2))
  main(params)

我试着像这样修复它。

with open(checkpoint_path, 'rb', encoding='utf-8') as pickle_file:
  checkpoint = pickle.load(pickle_file)

但是发生了另一个错误.. “ValueError：二进制模式不接受编码参数”

我该怎么办？

Answer 1

pickle是二进制协议，所以使用二进制模式。正如您的错误所说，“二进制模式不采用编码参数”：

with open(checkpoint_path, 'rb') as pickle_file:
  checkpoint = pickle.load(pickle_file)

您的其他错误“UnicodeDecodeError”只会在文本模式下发生。确保使用'rb'。如果您仍有问题，请使用可重现的示例更新您的问题，并附上示例输入和错误消息的完整回溯。

完整演示：

import pickle

data = {'key':[1,2,3], 'key2':[4,5,6]}

with open('out.p','wb') as f:
    pickle.dump(data,f)

with open('out.p','rb') as f:
    data2 = pickle.load(f)

print(data == data2)

True

Answer 2

我自己找到了答案。

您可以通过添加此代码来解决此问题

with open(checkpoint_path, 'rb') as pickle_file:
      checkpoint = pickle.load(pickle_file, encoding='latin1')