AWS Data Pipeline 卡在 Waiting For Runner答案

【问题标题】：AWS Data Pipeline stuck on Waiting For RunnerAWS Data Pipeline 卡在 Waiting For Runner
【发布时间】：2019-02-20 09:10:25
【问题描述】：

我的目标是将 AWS RDS 上运行的 postgreSQL 数据库中的表复制到 Amazone S3 上的 .csv 文件中。为此，我使用 AWS 数据管道并找到以下 tutorial 但是当我按照所有步骤操作时，我的管道被卡在："WAITING FOR RUNNER" 参见屏幕截图。 AWS documentation 声明：

确保为 runsOn 或 workerGroup 设置有效值这些任务的字段

但是设置了“runs on”字段。知道为什么这条管道卡住了吗？

还有我的定义文件：

{
  "objects": [
    {
      "output": {
        "ref": "DataNodeId_Z8iDO"
      },
      "input": {
        "ref": "DataNodeId_hEUzs"
      },
      "name": "DefaultCopyActivity01",
      "runsOn": {
        "ref": "ResourceId_oR8hY"
      },
      "id": "CopyActivityId_8zaDw",
      "type": "CopyActivity"
    },
    {
      "resourceRole": "DataPipelineDefaultResourceRole",
      "role": "DataPipelineDefaultRole",
      "name": "DefaultResource1",
      "id": "ResourceId_oR8hY",
      "type": "Ec2Resource",
      "terminateAfter": "1 Hour"
    },
    {
      "*password": "xxxxxxxxx",
      "name": "DefaultDatabase1",
      "id": "DatabaseId_BWxRr",
      "type": "RdsDatabase",
      "region": "eu-central-1",
      "rdsInstanceId": "aqueduct30v05.cgpnumwmfcqc.eu-central-1.rds.amazonaws.com",
      "username": "xxxx"
    },
    {
      "name": "DefaultDataFormat1",
      "id": "DataFormatId_wORsu",
      "type": "CSV"
    },
    {
      "database": {
        "ref": "DatabaseId_BWxRr"
      },
      "name": "DefaultDataNode2",
      "id": "DataNodeId_hEUzs",
      "type": "SqlDataNode",
      "table": "y2018m07d12_rh_ws_categorization_label_postgis_v01_v04",
      "selectQuery": "SELECT * FROM y2018m07d12_rh_ws_categorization_label_postgis_v01_v04 LIMIT 100"
    },
    {
      "failureAndRerunMode": "CASCADE",
      "resourceRole": "DataPipelineDefaultResourceRole",
      "role": "DataPipelineDefaultRole",
      "pipelineLogUri": "s3://rutgerhofste-data-pipeline/logs",
      "scheduleType": "ONDEMAND",
      "name": "Default",
      "id": "Default"
    },
    {
      "dataFormat": {
        "ref": "DataFormatId_wORsu"
      },
      "filePath": "s3://rutgerhofste-data-pipeline/test",
      "name": "DefaultDataNode1",
      "id": "DataNodeId_Z8iDO",
      "type": "S3DataNode"
    }
  ],
  "parameters": []
}

【问题讨论】：

标签： amazon-web-services amazon-data-pipeline

【解决方案1】：

通常“WAITING FOR RUNNER”状态意味着它正在等待资源（例如 EMR 集群）。您似乎没有设置“工作组”字段。这意味着你已经指定了“做什么”，但没有指定“谁”应该做。

【讨论】：