当我尝试模拟它时，Sphinx / ReadTheDocs 仍然尝试构建一个模块

Question

我正在尝试在 ReadTheDocs 上使用 Sphinx 构建文档，因为 a project 依赖于一堆 pydata 包，其中一些依赖于 ReadTheDocs 上不可用的 C 库（即 python-snappy 包，它提供snappy 模块，实现了我们在 Apache Parquet 文件中使用的 Google 的 snappy 压缩算法。

文档suggests that the solution 是模拟那些python 模块，以便autodoc 可以导入它们，即使代码不存在，使用conf.py 中的autodoc_mock_imports 参数所以我得到了

autodoc_mock_imports = ['snappy', 'python-snappy']

在我的 Sphinx conf.py 中（我认为它真的应该只是 snappy 在那里，但我不是 100% 确定它不与 snappy 一起工作，所以我添加了包名称以及模块名称......它仍然没有工作）。

但是，文档构建仍在尝试编译 snappy 并且可以预见会失败，因为 C 语言头文件不可用。当构建收集需求时，它找到python-snappy，我认为在项目中setup.py 并说：

Collecting python-snappy (from pudl==0.1.dev398+g5e075d4)

随后构建失败：

Building wheels for collected packages: pudl, python-snappy
  Building wheel for pudl (setup.py): started
  Building wheel for pudl (setup.py): finished with status 'done'
  Stored in directory: /home/docs/checkouts/readthedocs.org/user_builds/catalyst-cooperative-pudl/.cache/pip/wheels/a4/5f/b6/1f8213aeb5876af1c140b54dce3466b845d989ca1101da875a
  Building wheel for python-snappy (setup.py): started
  Building wheel for python-snappy (setup.py): finished with status 'error'
  ERROR: Complete output from command /home/docs/checkouts/readthedocs.org/user_builds/catalyst-cooperative-pudl/envs/python-packaging/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-z6c1cop3/python-snappy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-r54uwk2d --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/snappy
  copying snappy/__init__.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/__main__.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/snappy_formats.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/snappy.py -> build/lib.linux-x86_64-3.7/snappy
  copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-3.7/snappy
  running build_ext
  building 'snappy._snappy' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/snappy
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/docs/.pyenv/versions/3.7.3/include/python3.7m -c snappy/snappymodule.cc -o build/temp.linux-x86_64-3.7/snappy/snappymodule.o
  snappy/snappymodule.cc:31:10: fatal error: snappy-c.h: No such file or directory
   #include <snappy-c.h>
            ^~~~~~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for python-snappy
  Running setup.py clean for python-snappy
Successfully built pudl
Failed to build python-snappy

python-snappy 已从包含在文档构建中的requirements.txt 中注释掉，但当然它仍包含在install_requires 参数中的项目setup.py 中。

为什么模拟不起作用？我如何让它停止尝试构建这个无法成功的包？在我的本地系统（安装了“libsnappy-dev”包）上，文档构建似乎工作正常。

可以在此处找到失败构建的完整输出：

https://readthedocs.org/projects/catalyst-cooperative-pudl/builds/9376117/

Answer 1

这里的问题是，我还明确告诉 Read The Docs 在.readthedocs.yml 配置文件中使用pip 构建和安装我的pudl 包：

# Set the version of Python and requirements required to build your docs
python:
  version: 3.7
  install:
    - requirements: docs/requirements.txt
    - method: pip
      path: .
  system_packages: true

这意味着它试图通过运行存储库顶层中的setup.py 来创建pudl 分发，但由于缺少snappy C 标头，因此可以预见地失败了。

解决方案是根据READTHEDOCS 环境变量有条件地从setup.py 中的install_requires 参数中排除python-snappy 包，使依赖关系由autodoc_mock_imports 中指定的模拟模块来满足conf.py:

install_requires = [
    'coloredlogs',
    'datapackage',
    'dbfread',
    'goodtables',
    'jupyter',
    'jupyterlab',
    'matplotlib',
    'nbval',
    'networkx>=2.2',
    'numpy',
    'pandas>=0.24',
    'psycopg2',
    'pyarrow>=0.14.0',
    'pyyaml',
    'scikit-learn>=0.20',
    'scipy',
    'sqlalchemy>=1.3',
    'sqlalchemy-postgres-copy',
    'tableschema',
    'timezonefinder',
    'xlsxwriter',
]

# We are installing the PUDL module to build the docs, but the C libraries
# required to build snappy aren't available on RTD, so we need to exclude it
# from the installed dependencies here, and mock it for import in docs/conf.py
# using the autodoc_mock_imports parameter:
if not os.getenv('READTHEDOCS'):
    install_requires.append('python-snappy')

另一种可能的解决方案是将pudl 包目录添加到conf.py 中的sys.path，以便可以导入该包，即使它没有安装。但是，我使用 setuptools_scm 自动生成版本号以供打包和文档使用，为此，pudl 包需要实际构建为分发版并安装。