【发布时间】：2020-02-19 03:16:57
【问题描述】：
我正在尝试运行一个 hadoop 容器。我所做的是我从下载 hadoop https://hadoop.apache.org/releases.html 我下载了3.1.3文件https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/hadoop-3.1.3-src.tar.gz安装成功。我看到以下
debian@osboxes:~$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
python latest eeadc22d21a9 2 weeks ago 933MB
hadoop-build-1001 latest 1d1a162a72a6 3 months ago 2.02GB
hadoop-build latest 5c1480006f78 3 months ago 1.96GB
ubuntu xenial 5f2bf26e3524 3 months ago 123MB
现在当我尝试运行图像时
docker run -it --user debian 1d1a162a72a6
我得到错误
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "chdir to cwd (\"/root\") set in config.json failed: permission denied": unknown.
debian@osboxes:~$
我无法理解为什么会出现这些错误以及如何摆脱上述错误。
我以某种方式可以以 root 用户身份登录到容器，然后在里面我做了一个su debian，所以我能够以 debian 身份登录，
现在我有了想在其中运行的程序 mapper 和 reducer。
我当时就这样做了
RUN apt-get update
RUN apt-get install default-jdk wget -y
RUN apt-get install python3 -y
RUN wget
http://mirrors.estointernet.in/apache/hadoop/common/hadoop-2.10.0/hadoop-2.10.0.tar.gz
RUN tar -xzvf hadoop-2.10.0.tar.gz
ENV JAVA_HOME $(readlink -f /usr/bin/java | sed "s:bin/java::")
RUN mv hadoop-2.10.0 /usr/local/hadoop
ENV PATH /usr/local/hadoop/bin:$PATH
RUN rm -rf hadoop-2*
现在我已经在 hadoop 3 中安装了 hadoop-2.10.0，然后我执行了我的程序。 我必须在容器内编辑 debian 的 .bashrc。 我想知道如何在我从源代码构建和编译的 hadoop-3.1.3 中运行映射器和减速器程序，或者我在这里做错了什么？
有没有更好的方法来使用我花了几个小时编译和安装的 hadoop-3.1.3 环境。
更新 1
在我下载的hadoop源文件的构建脚本中有以下几行
docker build -t "hadoop-build-${USER_ID}" - <<UserSpecificDocker
FROM hadoop-build
RUN groupadd --non-unique -g ${GROUP_ID} ${USER_NAME}
RUN useradd -g ${GROUP_ID} -u ${USER_ID} -k /root -m ${USER_NAME}
RUN echo "${USER_NAME} ALL=NOPASSWD: ALL" > "/etc/sudoers.d/hadoop-build-${USER_ID}"
ENV HOME /home/${USER_NAME}
UserSpecificDocker
# By mapping the .m2 directory you can do an mvn install from
# within the container and use the result on your normal
# system. And this also is a significant speedup in subsequent
# builds because the dependencies are downloaded only once.
docker run --rm=true -t -i \
-v "${PWD}:/home/${USER_NAME}/hadoop${V_OPTS:-}" \
-w "/home/${USER_NAME}/hadoop" \
-v "${HOME}/.m2:/home/${USER_NAME}/.m2${V_OPTS:-}" \
-u "${USER_NAME}" \
"hadoop-build-${USER_ID}"
"start-build-env.sh"
所以我认为我有 hadoop 的开发环境。
但是我的问题在于容器。
当我做docker ps -a
我明白了
19a4eff3b5e5 1d1a162a72a6 "/bin/bash" 3 hours ago Exited (0) 3 hours ago assignment4
30933891f08c eeadc22d21a9 "python3" 3 hours ago Exited (0) 3 hours ago python
ccdab94fd32f 1d1a162a72a6 "/bin/bash" 9 hours ago Exited (0) 5 hours ago confident_wu
ce462ecfc5f2 1d1a162a72a6 "/bin/bash -v /home/…" 9 hours ago Exited (127) 9 hours ago stupefied_grothendieck
6123f134934c 1d1a162a72a6 "/bin/bash" 9 hours ago Exited (1) 9 hours ago stupefied_taussig
0ed23a8112a4 1d1a162a72a6 "/bin/bash" 10 hours ago Exited (0) 10 hours ago vigilant_bartik
c343731b7cde 1d1a162a72a6 "/bin/bash" 11 hours ago Exited (0) 10 hours ago gallant_ardinghelli
2f95d3b4c1b8 1d1a162a72a6 "/bin/bash" 11 hours ago Created nice_hermann
5ebe9f18c744 1d1a162a72a6 "/bin/bash" 27 hours ago Created pensive_easley
c1b43edfafb9 1d1a162a72a6 "/bin/bash" 27 hours ago Exited (1) 3 hours ago adoring_williams
42dea69d1d4e 1d1a162a72a6 "/bin/bash" 27 hours ago Created funny_austin
6f736902e650 1d1a162a72a6 "/bin/bash" 27 hours ago Exited (1) 27 hours ago strange_ride
09306e5ec5d1 1d1a162a72a6 "--name=kaushik" 32 hours ago Created pensive_shtern
699fb2a23e1c 1d1a162a72a6 "--name=kaushik" 32 hours ago Created sharp_feistel
9f7b29ab512e 1d1a162a72a6 "--name=kaushik" 32 hours ago Created elastic_payne2
25bfc74fab3b 1d1a162a72a6 "/bin/bash" 33 hours ago Exited (1) 31 hours ago festive_einstein
e658dd320297 1d1a162a72a6 "/bin/bash" 33 hours ago Up 3 hours objectiv:
我不知道创建了多少容器，并且我对容器内的环境进行了更改。重新启动后，我找不到对这些容器进行的任何更改。我的容器如何在重新启动后仍然存在？
【问题讨论】：
-
请只使用现有的 Hadoop docker 镜像，不要混合版本
-
您也不需要从容器中运行代码。您只需从任何地方配置它以使用 YARN 集群地址
-
我无法理解如何使用现有图像来运行映射器和化简器代码。我有代码，但我不清楚如何在 localhost 上拥有所有内容。
-
那就不用容器来学习了。 Hadoop 从未设计或打算在 Docker 中运行