SageMaker 训练作业未创建 /opt/ml/input/data/training 目录答案

【问题标题】：SageMaker Train job is not creating /opt/ml/input/data/training directorySageMaker 训练作业未创建 /opt/ml/input/data/training 目录
【发布时间】：2019-08-02 14:00:06
【问题描述】：

我正在尝试按照this tutorial 中的说明创建自定义算法。

当我运行训练作业时，它失败并出现错误 No such file or directory: '/opt/ml/input/data/training'。

根据文档，SageMaker 应在运行时创建这些文档并复制数据和工件。但这并没有发生。

请分享您对此的看法。

我的 DockerFile 内容，

# Build an image that can do training and inference in SageMaker
 # This is a Python 2 image that uses the nginx, gunicorn, flask stack
 # for serving inferences in a stable way.

 FROM ubuntu:16.04

 MAINTAINER Amazon AI <sage-learner@amazon.com


 RUN apt-get -y update && apt-get install -y --no-install-recommends \
          wget \
          python \
          nginx \
          ca-certificates \
     && rm -rf /var/lib/apt/lists/*

 # Here we get all python packages.
 # There's substantial overlap between scipy and numpy that we eliminate by
 # linking them together. Likewise, pip leaves the install caches populated which uses
 # a significant amount of space. These optimizations save a fair amount of space in the
 # image, which reduces start up time. RUN wget https://bootstrap.pypa.io/get-pip.py && python get-pip.py && \
     pip install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gevent gunicorn && \
         (cd /usr/local/lib/python2.7/dist-packages/scipy/.libs; rm *; ln ../../numpy/.libs/* .) && \
         rm -rf /root/.cache

 # Set some environment variables. PYTHONUNBUFFERED keeps Python from buffering our standard
 # output stream, which means that logs can be delivered to the user quickly. PYTHONDONTWRITEBYTECODE
 # keeps Python from writing the .pyc files which are unnecessary in this case. We also update
 # PATH so that the train and serve programs are found when the container is invoked.

 ENV PYTHONUNBUFFERED=TRUE ENV PYTHONDONTWRITEBYTECODE=TRUE ENV
 PATH="/opt/program:${PATH}"

 # Set up the program in the image COPY decision_trees /opt/program WORKDIR /opt/program

【问题讨论】：

标签： amazon-web-services aws-sdk amazon-sagemaker

【解决方案1】：

训练文件夹名称取决于您在 CreateTrainingJob 操作中提供的 InputDataConfig： https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html#SageMaker-CreateTrainingJob-request-InputDataConfig

如果频道名称为“xyz”，则会在该位置创建同名文件夹 (/opt/ml/input/data/xyz)

【讨论】：