在 Linux Alpine 上安装 DPDK 时出错答案

【问题标题】：Errors when installing DPDK on Linux Alpine在 Linux Alpine 上安装 DPDK 时出错
【发布时间】：2018-10-10 20:22:14
【问题描述】：

我正在尝试在 Alpine Linux 上安装 DPDK。我在 Docker 中运行 Alpine Linux。以下是我正在使用的步骤：

docker pull alpine
docker run -it alpine /bin/sh
wget https://git.dpdk.org/dpdk-stable/snapshot/dpdk-stable-16.11.8.tar.gz
tar -xzf dpdk-stable-16.11.8.tar.gz
cd dpdk-stable-16.11.8
apk add --no-cache make gcc libc-dev bsd-compat-headers linux-headers musl-dev
apk add numactl-dev --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
make install T=x86_64-native-linuxapp-gcc

安装失败并出现以下错误和警告：

make[5]: 'depdirs' 无事可做。

配置完成

== 构建库

== 构建 lib/librte_compat

== 构建 lib/librte_eal

== 构建 lib/librte_eal/common

== 构建 lib/librte_eal/linuxapp

== 构建 lib/librte_eal/linuxapp/eal

CC eal_hugepage_info.o

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c：在函数'clear_hugedir'中：

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:229:8：警告：函数“openat”的隐式声明；您指的是 'popen' 吗？ [-Wimplicit-function-declaration] fd = openat（dir_fd，dirent->d_name，O_RDONLY）； ^~~~~~ 打开

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:229:8：警告：“openat”的嵌套外部声明 [-Wnested-externs]

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:229:39: error: 'O_RDONLY' undeclared (第一次在这个函数中使用) fd = openat（dir_fd，dirent->d_name，O_RDONLY）； ^~~~~~~~ /dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:229:39：注意：每个未声明的标识符对于它出现的每个函数只报告一次

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c：在函数'eal_hugepage_info_init'中：

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:320:26：警告：函数“open”的隐式声明；您指的是 'popen' 吗？ [-Wimplicit-function-declaration] hpi->lock_descriptor = open(hpi->hugedir, O_RDONLY); ^~~~ 打开

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:320:26：警告：“open”的嵌套外部声明 [-Wnested-externs]

/dpdk/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_hugepage_info.c:320:45: error: 'O_RDONLY' undeclared (第一次在这个函数中使用) hpi->lock_descriptor = open(hpi->hugedir, O_RDONLY); ^~~~~~~~

make[7]: *** [/dpdk/dpdk-stable-16.11.8/mk/internal/rte.compile-pre.mk:140: eal_hugepage_info.o] 错误1

make[6]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: eal] 错误2

make[5]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: linuxapp] 错误2

make[4]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: librte_eal] 错误2

make[3]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.sdkbuild.mk:80: lib] 错误2

make[2]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.sdkroot.mk:127: all] 错误2

make[1]: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.sdkinstall.mk:86: pre_install] 错误2

make: *** [/dpdk/dpdk-stable-16.11.8/mk/rte.sdkroot.mk:101: install] 错误2

更新 1

我修复了一些错误，但现在出现了新错误。以下是更新的步骤：

docker pull alpine
docker run -it alpine /bin/sh
wget https://git.dpdk.org/dpdk-stable/snapshot/dpdk-stable-16.11.8.tar.gz
tar -xzf dpdk-stable-16.11.8.tar.gz
cd dpdk-stable-16.11.8
apk add --no-cache make gcc libc-dev bsd-compat-headers linux-headers musl-dev
apk add numactl-dev --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted
将#include <fcntl.h> 添加到lib/librte_eal/common/include/rte_common.h
将CFLAGS += -D_GNU_SOURCE 添加到lib/librte_eal/linuxapp/eal/Makefile
make install T=x86_64-native-linuxapp-gcc

现在安装失败并出现以下错误：

make[5]: 'depdirs' 无事可做。配置完成

== 构建库

== 构建 lib/librte_compat

== 构建 lib/librte_eal

== 构建 lib/librte_eal/common

== 构建 lib/librte_eal/linuxapp

== 构建 lib/librte_eal/linuxapp/eal

CC eal_vfio_mp_sync.o

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c：在函数'vfio_mp_sync_send_fd'中：

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c:75:17：错误：“struct cmsghdr”没有名为“__cmsg_data”的成员 memcpy((chdr).__cmsg_data, &(fd), sizeof(fd));\ ^

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c:187:2：注意：在宏“FD_TO_CMSGHDR”的扩展中 FD_TO_CMSGHDR(fd, *chdr); ^~~~~~~~~~~~~~~~

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c：在函数'vfio_mp_sync_receive_fd'中：

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c:78:24：错误：“struct cmsghdr”没有名为“__cmsg_data”的成员 memcpy(&(fd), (chdr).__cmsg_data, sizeof(fd)) ^

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c:228:2：注意：在宏“CMSGHDR_TO_FD”的扩展中 CMSGHDR_TO_FD(*chdr, fd); ^~~~~~~~~~~~~~~~

make[7]: *** [/dpdk-stable-16.11.8/mk/internal/rte.compile-pre.mk:140: eal_vfio_mp_sync.o] 错误1

make[6]: *** [/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: eal] 错误2

make[5]: *** [/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: linuxapp] 错误 2

make[4]: *** [/dpdk-stable-16.11.8/mk/rte.subdir.mk:63: librte_eal] 错误2

make[3]: *** [/dpdk-stable-16.11.8/mk/rte.sdkbuild.mk:80:lib] 错误2

make[2]: *** [/dpdk-stable-16.11.8/mk/rte.sdkroot.mk:127: all] 错误2

make[1]: *** [/dpdk-stable-16.11.8/mk/rte.sdkinstall.mk:86: pre_install] 错误2

make: *** [/dpdk-stable-16.11.8/mk/rte.sdkroot.mk:101: install] 错误2

如果你看一下/usr/include/x86_64-linux-gnu/bits/socket.h，struct cmsghdr的定义如下：

/* Structure used for storage of ancillary data object information.  */
struct cmsghdr
{
    size_t cmsg_len;
    /* Length of data in cmsg_data plus length
       of cmsghdr structure.
       !! The type should be socklen_t but the
       definition of the kernel is incompatible
       with this.  */
    int cmsg_level;     /* Originating protocol.  */
    int cmsg_type;      /* Protocol specific type.  */
#if __glibc_c99_flexarr_available
    __extension__ unsigned char __cmsg_data __flexarr; /* Ancillary data.  */
#endif
};

__glibc_c99_flexarr_available 是什么？我可以以某种方式提供它吗？

更新 2

我尝试使用this 而不是musl-dev，但错误仍然相同。

然后按照建议，我在我的 Alpine Linux 上安装了 glibc 来使用它而不是 musl。按照步骤here。现在我收到这样的错误：

在 /dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal.c:48:0: 中包含的文件中：

/usr/include/limits.h:124:26: 致命错误：limits.h: 没有这样的文件或目录

#include_next

【问题讨论】：

标签： docker alpine dpdk

【解决方案1】：

DPDK 似乎与 musl libc 不兼容，因此您可能很难一一修复错误。必须手动将预处理器定义和包含添加到 CFLAGS 似乎是个坏兆头。

最后一个错误就是这样的libc不兼容：

/dpdk-stable-16.11.8/lib/librte_eal/linuxapp/eal/eal_vfio_mp_sync.c:78:24: 错误：“struct cmsghdr”没有名为“__cmsg_data”的成员 memcpy(&(fd), (chdr ).__cmsg_data, sizeof(fd))

在我的 glibc (Ubuntu 16.04) 中，cmsghdr 在bits/socket.h 中定义如下：

/* Structure used for storage of ancillary data object information.  */
struct cmsghdr
  {
    size_t cmsg_len;            /* Length of data in cmsg_data plus length
                                   of cmsghdr structure.
                                   !! The type should be socklen_t but the
                                   definition of the kernel is incompatible
                                   with this.  */
    int cmsg_level;             /* Originating protocol.  */
    int cmsg_type;              /* Protocol specific type.  */
#if (!defined __STRICT_ANSI__ && __GNUC__ >= 2) || __STDC_VERSION__ >= 199901L
    __extension__ unsigned char __cmsg_data __flexarr; /* Ancillary data.  */
#endif
  };

注意最后__cmsg_data的定义。

在 musl 中，cmsghdr 定义为：

struct cmsghdr {
        socklen_t cmsg_len;
        int __pad1;
        int cmsg_level;
        int cmsg_type;
};

注意缺少的__cmsg_data 字段。

【讨论】：

如果我使用this 而不是musl-dev，它也不起作用，即它显示相同的错误。
它可能仍会选择 musl 标头；该图像应提供兼容的glibc.so 链接，但我不确定 glibc 标头设置
但我绝对同意 glibc 构建可能是最好的前进方式
您对进一步的步骤有何建议？
首先我会尝试找出兼容的 glibc 版本是什么，具有__cmsg_data 定义的版本，以及如何在 Alpine 上获得它。我认为最坏的情况是必须从源代码构建和安装 glibc。

【解决方案2】：

最后，经过一段时间和许多变通办法，我设法在 Alpine Linux 上安装了 DPDK，即在运行 Alpine Linux 的 Docker 容器上。

这是我创建的 Dockerfile（cmets 是内联的）：

FROM alpine:latest

ARG DPDK_VER=16.11.8
ARG KERNEL_VER=4.9.4

RUN wget https://git.dpdk.org/dpdk-stable/snapshot/dpdk-stable-$DPDK_VER.tar.gz && \
    tar -xzf dpdk-stable-$DPDK_VER.tar.gz && \
    \
    # Install Alpine packages
    apk add --no-cache make gcc linux-headers bsd-compat-headers binutils coreutils diffutils && \
    apk add --no-cache gettext bash grep sed texinfo perl util-linux vim build-base libexecinfo libexecinfo-dev && \
    apk add --no-cache abuild bc cmake ncurses-dev ca-certificates wget && \
    apk add numactl-dev --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/testing/ --allow-untrusted && \
    \
    # Enable usage of High Precision Timers
    sed -i "/\b\(CONFIG_RTE_LIBEAL_USE_HPET\|n\)\b/d" /dpdk-stable-$DPDK_VERSION/config/common_base && \
    sed -ie '/CONFIG_RTE_LOG_HISTORY=256/a CONFIG_RTE_LIBEAL_USE_HPET=y' /dpdk-stable-$DPDK_VERSION/config/common_base && \
    \
    # Add following lines / workarounds to header files 
    sed -ie '/int cmsg_type;/a unsigned char __cmsg_data;' /usr/include/bits/socket.h && \
    sed -i.old '1s;^;#include <fcntl.h>\n;' /dpdk-stable-$DPDK_VER/lib/librte_eal/common/include/rte_common.h && \
    sed -ie '/#define _VNIC_DEVCMD_H_/a typedef u32 u_int32_t;' /dpdk-stable-$DPDK_VER/drivers/net/enic/base/vnic_devcmd.h && \
    \
    # Prevent usage of the header files on /usr/include/fortify path by deleting them
    # Instead of them header files from /usr/include path will be used
    rm -rf /usr/include/fortify/* && \
    \
    # Add CFLAGS += -D_GNU_SOURCE to the following Makefiles
    cd /dpdk-stable-$DPDK_VER && \
    sed -ie '/CFLAGS/a CFLAGS += -D_GNU_SOURCE' lib/librte_ring/Makefile && \
    sed -ie '/CFLAGS/a CFLAGS += -D_GNU_SOURCE' lib/librte_mempool/Makefile && \
    sed -ie '/CFLAGS/a CFLAGS += -D_GNU_SOURCE' lib/librte_mbuf/Makefile && \
    sed -ie '/CFLAGS/a CFLAGS += -D_GNU_SOURCE' lib/librte_timer/Makefile && \
    sed -ie '/CFLAGS/a CFLAGS += -D_GNU_SOURCE' lib/librte_net/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_eal/linuxapp/eal/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_ether/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_cryptodev/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_vhost/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_hash/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_lpm/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_acl/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_ip_frag/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_sched/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_distributor/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_kni/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_port/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_table/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_pipeline/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' lib/librte_reorder/Makefile && \
    \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/af_packet/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/bonding/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/cxgbe/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/e1000/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/ena/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/enic/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/fm10k/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/i40e/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/ixgbe/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/bnxt/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/null/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/qede/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/ring/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/virtio/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/vmxnet3/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/net/vhost/Makefile && \
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' drivers/crypto/null/Makefile && \
    \
    # Enable tests for compiling
    sed -ie '/EXECENV_LDLIBS  += -ldl/a EXECENV_LDLIBS  += -lexecinfo' mk/exec-env/linuxapp/rte.vars.mk && \
    sed -ie '/#include <string.h>/a #include <features.h>' lib/librte_eal/linuxapp/eal/eal_pci_uio.c && \
    sed -ie '/#define OFF_MAX/a static inline void outb_p (unsigned char __value, unsigned short int __port) { __asm__ __volatile__ ("outb %b0,%w1\\noutb %%al,$0x80": :"a" (__value), "Nd" (__port)); }' lib/librte_eal/linuxapp/eal/eal_pci_uio.c && \
    sed -ie '/#define OFF_MAX/a static inline void outw_p (unsigned short int __value, unsigned short int __port) { __asm__ __volatile__ ("outw %w0,%w1\\noutb %%al,$0x80": :"a" (__value), "Nd" (__port)); }' lib/librte_eal/linuxapp/eal/eal_pci_uio.c && \
    sed -ie '/#define OFF_MAX/a static inline void outl_p (unsigned int __value, unsigned short int __port) { __asm__ __volatile__ ("outl %0,%w1\\noutb %%al,$0x80": :"a" (__value), "Nd" (__port)); }' lib/librte_eal/linuxapp/eal/eal_pci_uio.c && \
    # - test
    sed -ie '/LDLIBS += -lm/a LDLIBS += -lrte_pmd_ring -lrte_pmd_null' app/test/Makefile && \
    # - test-acl
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' app/test-acl/Makefile && \
    # - test-pipeline
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' app/test-pipeline/Makefile && \
    # - test-pmd
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' app/test-pmd/Makefile && \
    sed -ie '/DEPDIRS-y += lib drivers/a LDLIBS += -lrte_pmd_ixgbe' app/test-pmd/Makefile && \
    sed -ie '/typedef uint16_t streamid_t/a typedef unsigned int uint;' app/test-pmd/testpmd.h && \
    # - proc_info
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' app/proc_info/Makefile && \
    # - pdump
    sed -ie '/CFLAGS += $(WERROR_FLAGS)/a CFLAGS += -D_GNU_SOURCE' app/pdump/Makefile && \
    cd / && \
    \
    # Install missing Alpine Linux kernel modules
    wget -nv -P /srv https://www.kernel.org/pub/linux/kernel/v4.x/linux-$KERNEL_VER.tar.gz && \
    tar -C /srv -zxf /srv/linux-$KERNEL_VER.tar.gz && \
    cd /srv/linux-$KERNEL_VER && \
    make defconfig && \
    ([ ! -f /proc/1/root/proc/config.gz ] || zcat /proc/1/root/proc/config.gz > .config) && \
    echo 'CONFIG_USB=m' >> .config && \
    echo 'CONFIG_USB_HID=m' >> .config && \
    echo 'CONFIG_USB_SUPPORT=y' >> .config && \
    echo 'CONFIG_USB_COMMON=m' >> .config && \
    echo 'CONFIG_USB_ARCH_HAS_HCD=y' >> .config && \
    echo 'CONFIG_USB_DEFAULT_PERSIST=y' >> .config && \
    echo 'CONFIG_USBIP_CORE=m' >> .config && \
    echo 'CONFIG_USBIP_VHCI_HCD=m' >> .config && \
    echo 'CONFIG_USBIP_VHCI_HC_PORTS=8' >> .config && \
    echo 'CONFIG_USBIP_VHCI_NR_HCS=1' >> .config && \
    echo 'CONFIG_USBIP_HOST=m' >> .config && \
    sed -i'.bak' '/hcd->amd_resume_bug/{s/^/\/\//;n;s/^/\/\//}' ./drivers/usb/core/hcd-pci.c && \
    make oldconfig && \
    make modules_prepare && \
    make modules && \
    make modules_install && \
    make clean && \
    cd / && \
    \
    # Install DPDK
    mkdir /dpdk-build && \
    cd dpdk-stable-$DPDK_VER && \
    make install T=x86_64-native-linuxapp-gcc RTE_KERNELDIR=/srv/linux-$KERNEL_VER DESTDIR=/dpdk-build

我希望这会节省一些人的时间:)

【讨论】：