【问题标题】:Failed to build pyarrow (for python 3.7) with an error message ERROR: Could not build wheels for pyarrow which use PEP 517无法构建 pyarrow(适用于 python 3.7)并显示错误消息错误:无法为使用 PEP 517 的 pyarrow 构建轮子
【发布时间】:2020-07-30 05:43:57
【问题描述】:

我正在尝试在我的虚拟环境中安装 apache-beam[gcp]。

     My python version is 3.7
     My pip3 version is 20.0.2

何时尝试安装(不带 sudo)并运行以下命令:-

    pip3 install apache-beam[gcp]

我从 pyarrow 社区发现了一条关于 python 3.8 (ARROW-7076) 的类似错误的说明,但 3.7 没有。当我遇到困难时,我将不胜感激!

看起来这个问题在 apache jira 仪表板中打开 (AIRAVATA-3305)

  https://issues.apache.org/jira/browse/AIRAVATA-3305? 
  jql=text%20~%20%22pip%20install%20python%203.7%22

添加完整的错误消息(现在以 root 身份运行,但得到相同的错误消息):

  copying pyarrow/include/arrow/io/hdfs.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  copying pyarrow/include/arrow/io/interfaces.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  copying pyarrow/include/arrow/io/memory.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  copying pyarrow/include/arrow/io/mman.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  copying pyarrow/include/arrow/io/slow.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  copying pyarrow/include/arrow/io/test_common.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/io
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/api.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/dictionary.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/feather.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/json_integration.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/json_simple.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/message.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/options.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/reader.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/test_common.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  copying pyarrow/include/arrow/ipc/writer.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/ipc
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/api.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/chunked_builder.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/chunker.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/converter.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/options.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/parser.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/rapidjson_defs.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/reader.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  copying pyarrow/include/arrow/json/test_common.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/json
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/api.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/arrow_to_pandas.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/benchmark.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/common.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/config.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/datetime.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/decimal.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/deserialize.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/extension_type.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/flight.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/helpers.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/inference.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/init.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/io.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/iterators.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/numpy_convert.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/numpy_interop.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/numpy_to_arrow.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/platform.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/pyarrow.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/pyarrow_api.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/pyarrow_lib.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/python_to_arrow.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/serialize.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/type_traits.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  copying pyarrow/include/arrow/python/visibility.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/python
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/extension_type.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/generator.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/gtest_common.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/gtest_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/random.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  copying pyarrow/include/arrow/testing/util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/testing
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/align_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/atomic_shared_ptr.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/base64.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/basic_decimal.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/bit_stream_utils.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/bit_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/bpacking.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/checked_cast.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compare.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compiler_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_brotli.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_bz2.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_lz4.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_snappy.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_zlib.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/compression_zstd.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/config.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/cpu_info.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/decimal.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/delimiting.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/double_conversion.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/formatting.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/functional.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/hash_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/hashing.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/int_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/io_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/iterator.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/key_value_metadata.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/logging.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/macros.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/make_unique.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/memory.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/neon_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/optional.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/parallel.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/parsing.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/print.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/range.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/rle_encoding.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/sort.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/sse_util.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/stopwatch.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/string.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/string_builder.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/string_view.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/task_group.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/thread_pool.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/time.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/trie.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/type_traits.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/ubsan.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/uri.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/utf8.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/variant.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/vector.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/visibility.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  copying pyarrow/include/arrow/util/windows_compatibility.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/util
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  copying pyarrow/include/arrow/vendored/datetime.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  copying pyarrow/include/arrow/vendored/optional.hpp -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  copying pyarrow/include/arrow/vendored/string_view.hpp -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  copying pyarrow/include/arrow/vendored/variant.hpp -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  copying pyarrow/include/arrow/vendored/xxhash.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  copying pyarrow/include/arrow/vendored/datetime/date.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  copying pyarrow/include/arrow/vendored/datetime/ios.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  copying pyarrow/include/arrow/vendored/datetime/tz.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  copying pyarrow/include/arrow/vendored/datetime/tz_private.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  copying pyarrow/include/arrow/vendored/datetime/visibility.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/datetime
  creating build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/bignum-dtoa.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/bignum.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/cached-powers.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/diy-fp.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/double-conversion.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/fast-dtoa.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/fixed-dtoa.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/ieee.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/strtod.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  copying pyarrow/include/arrow/vendored/double-conversion/utils.h -> build/lib.linux-aarch64-3.6/pyarrow/include/arrow/vendored/double-conversion
  creating build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/__init__.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/common.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libarrow.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libarrow_cuda.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libarrow_dataset.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libarrow_flight.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libarrow_fs.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libgandiva.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  copying pyarrow/includes/libplasma.pxd -> build/lib.linux-aarch64-3.6/pyarrow/includes
  creating build/lib.linux-aarch64-3.6/pyarrow/tensorflow
  copying pyarrow/tensorflow/plasma_op.cc -> build/lib.linux-aarch64-3.6/pyarrow/tensorflow
  creating build/lib.linux-aarch64-3.6/pyarrow/tensorflow/plasma_op.so.dSYM
  creating build/lib.linux-aarch64-3.6/pyarrow/tensorflow/plasma_op.so.dSYM/Contents
  copying pyarrow/tensorflow/plasma_op.so.dSYM/Contents/Info.plist -> build/lib.linux-aarch64-3.6/pyarrow/tensorflow/plasma_op.so.dSYM/Contents
  copying pyarrow/tests/pyarrow_cython_example.pyx -> build/lib.linux-aarch64-3.6/pyarrow/tests
  creating build/lib.linux-aarch64-3.6/pyarrow/tests/data
  creating build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/README.md -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.jsn.gz -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.emptyFile.orc -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.test1.jsn.gz -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.test1.orc -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.jsn.gz -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/TestOrcFile.testDate1900.orc -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/decimal.jsn.gz -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  copying pyarrow/tests/data/orc/decimal.orc -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/orc
  creating build/lib.linux-aarch64-3.6/pyarrow/tests/data/parquet
  copying pyarrow/tests/data/parquet/v0.7.1.all-named-index.parquet -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/parquet
  copying pyarrow/tests/data/parquet/v0.7.1.column-metadata-handling.parquet -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/parquet
  copying pyarrow/tests/data/parquet/v0.7.1.parquet -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/parquet
  copying pyarrow/tests/data/parquet/v0.7.1.some-named-index.parquet -> build/lib.linux-aarch64-3.6/pyarrow/tests/data/parquet
  running build_ext
  creating /tmp/pip-install-jar_d6v5/pyarrow/build/temp.linux-aarch64-3.6
  -- Running cmake for pyarrow
  cmake -DPYTHON_EXECUTABLE=/usr/bin/python3  -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_FLIGHT=off -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=off -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_S3=off -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /tmp/pip-install-jar_d6v5/pyarrow
  error: command 'cmake' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pyarrow
  Building wheel for google-apitools (setup.py) ... done
  Created wheel for google-apitools: filename=google_apitools-0.5.28-py3-none-any.whl size=131642 sha256=53006fe1611f352b29680065003fd0568a63aa046b676bd54bf3780743ffaedd
  Stored in directory: /root/.cache/pip/wheels/49/56/1c/73a513e437099b768ededdcb95106a58f5cdd048fb27ff640b
  Building wheel for docopt (setup.py) ... done
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=19852 sha256=99e8eed8943e34a2bf2457d3a6f7f269e95eafa9386d2f0766c32d881fb1df40
  Stored in directory: /root/.cache/pip/wheels/3f/2a/fa/4d7a888e69774d5e6e855d190a8a51b357d77cc05eb1c097c9
  Building wheel for grpc-google-iam-v1 (setup.py) ... done
  Created wheel for grpc-google-iam-v1: filename=grpc_google_iam_v1-0.12.3-py3-none-any.whl size=15434 sha256=bd5259daf0a66e2a34afac785bb7177d843166633433393cb1907740528f4b3f
  Stored in directory: /root/.cache/pip/wheels/76/65/cd/392da05e43270f143b6c5076ba88d39144abff586792593e7c
  Building wheel for googleapis-common-protos (setup.py) ... done
  Created wheel for googleapis-common-protos: filename=googleapis_common_protos-1.51.0-py3-none-any.whl size=74527 sha256=bb94fa9ac3db45115fd71380b4ca671c7d1074d75a9a8eb25e93148843ba63c9
  Stored in directory: /root/.cache/pip/wheels/35/8d/af/a922cb18800b31fadac3523cadf6c1efdf233b788fe7a4da70
Successfully built apache-beam crcmod dill fastavro future hdfs avro-python3 google-apitools docopt grpc-google-iam-v1 googleapis-common-protos
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly

【问题讨论】:

  • 能否分享pip install -vvv pyarrow的未截断日志。这将改善调试/编写更有用的答案。

标签: python-3.x virtualenv apache-beam pyarrow


【解决方案1】:

您是否有不能使用sudo pip3 install apache-beam[gcp] 的原因?当我试图复制这个问题时,这非常有效。一般来说,当使用 PEP 517 的包阻止您从源代码构建它时,就会出现此问题。使用 root shell 可以回避所有这些问题。

或者,总是下载源代码并自己构建。

【讨论】:

  • 我不想在根级别安装 beam 并将其保留在我的开发环境中。
【解决方案2】:

pyarrow 可用作 Python 3.7 的 manylinux2010,根本不需要在 Linux 上进行任何编译。有两种情况可能会阻止pip 拿起manylinux 轮:

  • 您的pip 版本太旧,无法了解新的manylinux 标签,请尝试将pip 更新到最新版本。
  • 您是 Alpine Linux 的用户,它使用不同的 libc 实现,因此默认情况下不支持 manylinux 轮子。

您还可以安装必要的编译要求,例如 cmakeboost、.. 但这比弄清楚为什么 pip 没有安装相应的轮子要复杂得多。

【讨论】:

  • 从上面可以看到,pip版本是20.0.2,是最新的。
  • 我在 Nividia Nano.4.9.140-tegra Arch Linux64 上使用 aarch-linux
  • 添加了 stackoverflow 允许我添加的最大错误消息数
  • @Uwe L. Korn,您能否提供有关在 windows 机器上的 python 3.7 中安装 pyarrow 的见解?
  • @san 究竟是什么?也许为此提出一个新问题。
【解决方案3】:

最后我设法解决了我的安装问题。这就是我所做的:

  1. 在安装之前检查 apache-beam 依赖项: https://beam.apache.org/documentation/sdks/python-dependencies/

  2. 在 conda 中启动您的虚拟环境(我以用户身份而不是以 root 身份运行)

     (base) $ source activate <myenv>
    
  3. 我将上面所有的依赖包信息复制到一个requirement.txt文件中,如下所示:-(剪切并粘贴)

    absl-py==0.9.0
    astor==0.8.1
    atomicwrites==1.3.0
    attrs==19.3.0
    avro-python3==1.8.2
    beautifulsoup4==4.8.1
    bs4==0.0.1
    cachetools==3.1.1
    certifi==2019.11.28
    chardet==3.0.4
    crcmod==1.7
    Cython==0.29.13
    dill==0.3.1.1
    docopt==0.6.2
    fastavro==0.21.24
    fasteners==0.15
    freezegun==0.3.12
    future==0.17.1
    gast==0.2.2
    google-api-core==1.16.0
    google-apitools==0.5.28
    google-auth==1.11.3
    google-auth-oauthlib==0.4.1
    google-cloud-bigquery==1.24.0
    google-cloud-bigtable==1.0.0
    google-cloud-core==1.1.0
    google-cloud-datastore==1.7.4
    google-cloud-dlp==0.13.0
    google-cloud-language==1.3.0
    google-cloud-pubsub==1.0.2
    google-cloud-spanner==1.13.0
    google-cloud-videointelligence==1.13.0
    google-cloud-vision==0.42.0
    google-pasta==0.2.0
    google-resumable-media==0.5.0
    googleapis-common-protos==1.51.0
    grpc-google-iam-v1==0.12.3
    grpcio==1.24.3
    grpcio-gcp==0.2.2
    guppy3==3.0.9
    h5py==2.10.0
    hdfs==2.5.8
    httplib2==0.12.0
    idna==2.9importlib-metadata==1.5.0
    joblib==0.14.1
    Keras-Applications==1.0.8
    Keras-Preprocessing==1.1.0
    Markdown==3.2.1
    mock==2.0.0
    monotonic==1.5
    more-itertools==8.2.0
    nltk==3.4.5
    nose==1.3.7
    nose-xunitmp==0.4.1
    numpy==1.17.3
    oauth2client==3.0.0
    oauthlib==3.1.0
    opt-einsum==3.2.0
    packaging==20.3
    pandas==0.25.2
    pbr==5.4.4
    Pillow==6.2.1
    pip==20.0.2
    pluggy==0.13.1
    protobuf==3.11.1
    protorpc==0.12.0
    py==1.8.1
    pyarrow==0.16.1
    pyasn1==0.4.8
    pyasn1-modules==0.2.8
    pydot==1.4.1
    pymongo==3.9.0
    pyparsing==2.4.6
    pytest==4.6.6
    python-dateutil==2.8.1
    python-gflags==3.1.2
    python-snappy==0.5.4
    pytz==2019.3
    PyYAML==3.13
    requests==2.23.0
    requests-oauthlib==1.3.0
    rsa==4.0
    scikit-learn==0.21.3
    scipy==1.4.1
    setuptools==46.0.0
    six==1.14.0
    soupsieve==2.0
    tensorboard==2.1.1
    tensorflow==2.1.0
    tensorflow-estimator==2.1.0
    termcolor==1.1.0
    typing-extensions==3.7.4.1
    urllib3==1.25.8
    wcwidth==0.1.8
    Werkzeug==1.0.0
    wheel==0.34.2
    wrapt==1.12.1
    zipp==3.1.0
    
    1. 我不得不交替使用 conda 和 pip 安装,因为其中一些软件包可用于各自的存储库:

      (myenv) $conda install --name myenv --yes --file requirements.txt

这将安装 conda 可以从其存储库中找到的内容。对于 pyarrow,conda 是我唯一的出路,因为我无法在我的 Nano arch linux 64 位上使用 pip 进行 cmake 构建。

  1. 您可以使用 pip 运行相同的程序来安装使用 conda 可能失败的内容

     (myenv) $pip install -r requirements.txt 
    
  2. 现在安装 apache-beam[gcp]:

    pip install apache-beam[gcp]==2.20.0
    

这可能不是最好的安装方式,我只是注意到我在安装 apache-beam 的许多天令人沮丧的经历之后所采取的步骤。 我最初认为运行 pip install apache-beam[gcp] 会理清依赖关系等,但事实并非如此。我必须声明我在同一台机器上有多个虚拟环境,并且我试图以用户和 root 身份安装 apache-beam,因此可能会搞砸。

可能这些步骤也适用于其他 linux 环境。

【讨论】:

    猜你喜欢
    • 2020-11-09
    • 2021-11-16
    • 2021-11-27
    • 2020-12-23
    • 2020-12-31
    • 2020-11-01
    • 2021-04-14
    • 2020-08-05
    • 2021-02-28
    相关资源
    最近更新 更多