PyInstaller 可执行文件无法获取 TorchScript 的源代码

Question

我正在尝试使包含 PyTorch 的脚本的 Windows 可执行。我的脚本的导入是：

import numpy.core.multiarray  # which is a workaround for "ImportError: numpy.core.multiarray failed to import"
import six # which is workaround for "ModuleNotFoundError: No module named 'six'"
import torch
import torch.nn as nn
import warnings
import argparse
import json
import math
import numpy as np
import jsonschema
import os
from datetime import datetime
from sklearn.mixture import GaussianMixture
from scipy.io import wavfile
from scipy.signal import get_window
from scipy.signal import spectrogram

我正在使用命令：

pyinstaller --hidden-import pkg_resources.py2_warn extractor.py

PyInstaller 在创建 .exe 时不会抛出错误，但是当我运行 .exe 时，我得到：

Traceback (most recent call last):
  File "site-packages\torch\_utils_internal.py", line 46, in get_source_lines_and_file
  File "inspect.py", line 967, in getsourcelines
  File "inspect.py", line 798, in findsource
OSError: could not get source code

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "extractor.py", line 3, in <module>
    import torch
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\ProgramData\Anaconda3\envs\forexe2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages\torch\__init__.py", line 367, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\ProgramData\Anaconda3\envs\forexe2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages\torch\distributions\__init__.py", line 112, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\ProgramData\Anaconda3\envs\forexe2\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "site-packages\torch\distributions\von_mises.py", line 55, in <module>
  File "site-packages\torch\jit\__init__.py", line 1287, in script
  File "site-packages\torch\jit\frontend.py", line 164, in get_jit_def
  File "site-packages\torch\_utils_internal.py", line 53, in get_source_lines_and_file
OSError: Can't get source for <function _rejection_sample at 0x0000000006892F70>. TorchScript requires source access in order to carry out compilation, make sure original .py files are available. Original error: could not get source code
[5704] Failed to execute script extractor

我不明白。这可能与this question 类似。是什么导致了问题？

我正在使用 conda env，通过 pip 安装了 torch（这是正确连接 torch 的一种解决方法）。

• Windows 10
• Python 3.8.2
• 火炬 1.5.0+cu101
• torchvision 0.6.0+cu101（也尝试使用 0.2.2）
• PyInstaller 3.6

Answer 1

Torch 是开源的，所以你可以在 Torch GitHub 上搜索函数_rejection_sample。这将有问题的文件标识为torch.distributions.von_mises。如果您的程序没有使用torch.distributions 模块，您可以通过更改pyinstaller 生成的.spec 文件来简单地排除它。

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None
excluded_modules = ['torch.distributions'] # <<< ADD THIS LINE

a = Analysis(['C:/your/path/here'],
             pathex=['C:\\your\\path\\here'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=excluded_modules,    # <<< CHANGE THIS LINE
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
# remaining code omitted for brevity

您应该只需要在上面指定的两个地方进行更改。其余的应该已经在那里了。

然后，build from the .spec file 使用

pyinstaller your_file.spec

将来，在构建项目时考虑使用debug flags。这也标识了导致问题的文件的位置。