docker compose up 无法挂载卷

Question

我在运行 docker-compose up 时遇到这些错误

错误：airflow_webserver_1 无法启动服务网络服务器：OCI 运行时创建失败：container_linux.go:346：启动容器 进程导致“process_linux.go:449：容器初始化导致 \"rootfs_linux.go:58: 挂载 \\"/srv/airflow/dags\\" 到 rootfs \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged\\" 在 \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged/usr/local/airflow/dags\\" 导致 \\"not a directory\\"\""：未知：您是否尝试挂载 将目录复制到文件中（反之亦然）？检查指定主机是否 路径存在并且是预期的类型

错误：对于网络服务器无法启动服务网络服务器：OCI 运行时 创建失败：container_linux.go:346：启动容器进程 导致“process_linux.go:449：容器初始化导致 \"rootfs_linux.go:58: 挂载 \\"/srv/airflow/dags\\" 到 rootfs \\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a

我看过类似的帖子说我应该检查挂载的目录应该存在并且是目录（而不是文件）等等......

我在我的主机上检查了路径 /src/airflow/dags 存在并包含多个文件（我之前容器运行的 dag 的代码）。

我怎样才能摆脱这个错误，为什么会发生这种情况？我不介意从头开始重新启动容器，丢失以前的卷内容。

这里是 docker-compose 的内容：

版本：'2.1' 服务： postgres： 图片：postgres：9.6 重启：除非停止 环境： - POSTGRES_USER=气流 - POSTGRES_PASSWORD=气流 - POSTGRES_DB=气流

webserver:
    build: airflow-server
    restart: unless-stopped
    depends_on:
        - postgres
    environment:
        - LOAD_EX=n
        - EXECUTOR=Local
        - VIRTUAL_HOST=airflow.agatha.com
        - VIRTUAL_PORT=8080
    volumes:
        - /srv/airflow/dags:/usr/local/airflow/dags
        # Uncomment to include custom plugins
        # - /srv/airflow/plugins:/usr/local/airflow/plugins
    ports:
        - "8075:8080"
    command: webserver
    healthcheck:
        test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
        interval: 30s
        timeout: 30s
        retries: 3
    logging:
        driver: json-file
        options:
            max-size: 50m

网络： 默认： 外部的： 名称：nginx-proxy

（airflow-server 文件夹包含一个自定义 Dockerfile 用于复制一些配置文件）

这里是 Dockerfile

FROM puckel/docker-airflow
USER root

COPY proxy.conf /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update && apt-get install -y gnupg apt-transport-https procps

RUN mkdir /usr/local/airflow/ssh
COPY id_rsa_airflow_user /usr/local/airflow/ssh/id_rsa_airflow_user
RUN mkdir /usr/local/airflow/sparkScripts
COPY FileUnion.py /usr/local/airflow/sparkScripts
COPY ETLDag.py /usr/local/airflow/dags

USER airflow

完整的命令行输出：

> :~/docker/airflow$ sudo docker-compose up WARNING: The Docker Engine
> you're using is running in swarm mode.
> 
> Compose does not use swarm mode to deploy services to multiple nodes
> in a swarm. All containers will be scheduled on the current node.
> 
> To deploy your application across the swarm, use `docker stack
> deploy`.
> 
> Starting airflow_postgres_1 ... done Recreating airflow_webserver_1
> ... error
> 
> ERROR: for airflow_webserver_1  Cannot start service webserver: OCI
> runtime create failed: container_linux.go:348: starting container
> process caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\\"/srv/airflow/dags\\\" to rootfs
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\\"
> at
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\\"
> caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type
> 
> ERROR: for webserver  Cannot start service webserver: OCI runtime
> create failed: container_linux.go:348: starting container process
> caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\\"/srv/airflow/dags\\\" to rootfs
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\\"
> at
> \\\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\\"
> caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type ERROR: Encountered errors while
> bringing up the project.

还有airflow-serverdirectory 的内容（有些文件没有被使用）

~/docker/airflow$ ls -l airflow-server/
total 326788
-rw-rw-r-- 1 user user 5008 mar 25 12:16 Dockerfile
-rw-rw-r-- 1 user user 7302 mar 24 15:37 ETLDagCreator.py
-rw-rw-r-- 1 user user 5900 mar 24 16:16 ETLDag.py
-rw-rw-r-- 1 user user 2564 mar 24 15:19 FileUnion.py
-rw-r--r-- 1 user user 334559382 mar 18 11:08 hadoop-3.1.1.tar.gz
drwxrwxr-x 2 user user 4096 mar 19 10:42 hadoop-client-conf
-rwxrwxr-x 1 user user 2026 mar 18 10:40 hbase.service.keytab
-rwxrwxr-x 1 user user 132 mar 18 10:39 hdfs.headless.keytab
-rwxrwxr-x 1 user user 2002 mar 18 10:38 hive.service.keytab
-rw-rw-r-- 1 user user 625 mar 19 16:02 hosts
-rwxrwxr-x 1 user user 1679 mar 20 10:17 id_rsa_airflow_user
-rw-rw-r-- 1 user user 529 mar 18 10:33 krb5.conf
-rw-rw-r-- 1 user user 101 mar 18 10:45 proxy.conf
-rw-rw-r-- 1 user user 58 mar 18 16:47 resolv.conf

谢谢

Answer 1

如错误所示，您正在尝试将目录挂载到文件上。这不是 Linux 中的有效绑定挂载，因此 docker 无法创建容器。该文件是在您的图像中创建的：

COPY ETLDag.py /usr/local/airflow/dags

如果您想将其复制到名为 dags 的目录中，则需要一个尾部斜杠：

COPY ETLDag.py /usr/local/airflow/dags/

您试图在文件上挂载的目录来自您的卷挂载：

volumes:
    - /srv/airflow/dags:/usr/local/airflow/dags

您在 cmets 中显示的是一个目录：

$ ls -ld /srv/airflow/dags
drwxr-xr-x 2 root root 4096 mar 25 12:45 /srv/airflow/dags

这些都需要是文件或都是目录。你不能混合它们。