如何将 Python 应用程序从 Jenkins 部署到 Amazon Elastic Beanstalk？

Question

我正在尝试通过 Jenkins 作业以编程方式部署到 Amazon Elastic Beanstalk。在我的开发机器上，这很简单：

eb deploy $(AWS_ELASTIC_BEANSTALK_ENVIRONMENT)

在 Jenkins 上，应该像将以下内容配置为构建命令一样简单：

virtualenv env && source env/bin/activate && pip install awsebcli
mkdir -p .elasticbeanstalk
cat << EOF > .elasticbeanstalk/config.yml 
branch-defaults:
  master:
    environment:  myenv
global:
  application_name:  myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
EOF
eb deploy myenv

但是，这会失败并出现以下跟踪：

[EnvInject] - Loading node environment variables.
Building remotely on standard Amazon Linux 2014.09 AMI (i-d39710df) (x) in workspace /media/ephemeral0/jenkins/workspace/My_Job
Fetching changes from the remote Git repository
Fetching upstream changes from git@github.com:myapp.git
Checking out Revision be45db94111f9ab49fe8031eb583307d2fb9921c (origin/master)
[My_Job] $ /bin/sh -xe /tmp/hudson8633484437962332339.sh
+ virtualenv env
New python executable in env/bin/python2.7
Not overwriting existing python script env/bin/python (you must use env/bin/python2.7)
Installing Setuptools..................................................................................................................done.
Installing Pip....................................................................................................................................done.
+ source env/bin/activate
++ deactivate nondestructive
++ unset pydoc
++ '[' -n '' ']'
++ '[' -n '' ']'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ '[' '!' nondestructive = nondestructive ']'
++ VIRTUAL_ENV=/media/ephemeral0/jenkins/workspace/My_Job/env
++ export VIRTUAL_ENV
++ _OLD_VIRTUAL_PATH=/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ PATH=/media/ephemeral0/jenkins/workspace/My_Job/env/bin:/usr/local/bin:/bin:/usr/bin:/opt/aws/bin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ '[' x '!=' x ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ '[' env = __ ']'
+++ basename /media/ephemeral0/jenkins/workspace/My_Job/env
++ PS1='(env)'
++ export PS1
++ alias 'pydoc=python -m pydoc'
++ '[' -n /bin/sh -o -n '' ']'
++ hash -r
+ pip install awsebcli
Requirement already satisfied (use --upgrade to upgrade): awsebcli in ./env/lib/python2.7/site-packages
Downloading/unpacking setuptools>=7.0 (from awsebcli)
  Running setup.py egg_info for package setuptools

Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.11 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): six==1.8.0 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): cement==2.4 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil>=2.2 in ./env/lib/python2.7/site-packages (from awsebcli)
Requirement already satisfied (use --upgrade to upgrade): jmespath>=0.4.1 in ./env/lib/python2.7/site-packages (from awsebcli)
Installing collected packages: setuptools
  Found existing installation: setuptools 0.9.7
    Uninstalling setuptools:
      Successfully uninstalled setuptools
  Running setup.py install for setuptools

    Installing easy_install script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
    Installing easy_install-2.7 script to /media/ephemeral0/jenkins/workspace/My_Job/env/bin
Successfully installed setuptools
Cleaning up...
+ mkdir -p .elasticbeanstalk
+ cat
+ cat .elasticbeanstalk/config.yml
branch-defaults:
  master:
    environment: myenv
global:
  application_name: myapp
  default_ec2_keyname: null
  default_platform: 64bit Amazon Linux 2014.09 v1.0.9 running Python 2.7
  default_region: us-west-2
  profile: eb-cli
  sc: git
+ eb deploy myenv
ERROR: The config profile (eb-cli) could not be found
Build step 'Execute shell' marked build as failure
Finished: FAILURE

目前尚不清楚为什么会发生这种情况，因为当我在项目的本地副本上运行上述内容时，它运行良好。

错误消息似乎没有太大帮助。目前还不清楚为什么在 Jenkins 机器上找不到 eb-cli。

所以再次总结一下我的问题：如何从 Jenkins 部署到 Amazon Elastic Beanstalk？上述方法是否正确，但细节有错误？还是有更简单的方法？

Answer 1

我通过 sshing 到 Jenkins 机器中解决了这个问题，运行 eb init，然后将生成的 .elasticbeanstalk/config.yml 与我在上面使用的 here-doc 中的比较。两者不同，因为我的开发机器和 Jenkins 机器上的安全配置文件不同。

我们可以重写这个脚本，使其对不同的config.yaml 文件更加健壮，如下所示：

virtualenv env && source env/bin/activate && pip install awsebcli
echo "1" | eb init myapp --region us-west-2 && eb use myenv && eb deploy myenv

注意，我们使用echo "1" | eb init myapp --region us-west-2 选择默认环境，因为eb init 不将环境作为位置参数，然后使用eb use myenv 选择我们想要的环境。

Answer 2

要更正 config profile (eb-cli) could not be found 错误，请将您用于部署到 EB 的凭据放入您的 jenkins 计算机上的 jenkins 用户的 ~/.aws/config。如果您在本地机器上构建部署，您应该能够直接从本地的~/.aws/config 拉取文件。它看起来像这样：

[profile eb-cli]
aws_access_key_id = (for your IAM user)
aws_secret_access_key = (for your IAM user)

Answer 3

我们突然遇到了这个问题，这可能相关也可能不相关，但我想为可能再次遇到此问题的任何人提供此问题。 eb-cli 似乎略有改变，不允许全局设置凭据。

我们的JenkinsFile 看起来像这样

 withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', 
 credentialsId: 'iam-creds']]) {
       sh "pip install awsebcli --upgrade --user"
       sh "~/.local/bin/eb use my-application"
       sh "~/.local/bin/eb deploy --verbose"
     }

我们的config.yml 看起来像这样

 global:
   application_name: my-application
   default_ec2_keyname: application-keyname
   ...
   profile: eb-cli
   sc: git
   workspace_type: Application

删除profile 键可以解决问题...但是，这不允许从本地计算机进行部署（除非有使用全局变量的方法）