【发布时间】:2021-06-28 16:25:42
【问题描述】:
我有一个临时 Emr 集群已准备就绪,我想在 emr 笔记本上运行一个简单的 pyspark 脚本。
有没有办法通过terraform创建和修改emr notebook?
提前致谢。
【问题讨论】:
标签: amazon-web-services pyspark terraform amazon-emr terraform-provider-aws
我有一个临时 Emr 集群已准备就绪,我想在 emr 笔记本上运行一个简单的 pyspark 脚本。
有没有办法通过terraform创建和修改emr notebook?
提前致谢。
【问题讨论】:
标签: amazon-web-services pyspark terraform amazon-emr terraform-provider-aws
是的,您可以从 Terraform 创建和修改 EMR 集群并选择要安装的工具,但这似乎是“艰难的方式”。更简单的是 Sagemaker Notebook 或使用新的 Glue Databrew 工具。
【讨论】:
As far as i know, AWS says "You create an EMR notebook using the Amazon EMR console. Creating notebooks using the AWS CLI or the Amazon EMR API is not supported." [AWS Documentation on creating EMR Notebook][1]
You can create a notebook via console, the notebook will be stored in S3 as .ipynb, by giving the relative path, you can execute notebook on the cluster. Refer boto3 for more info [Boto3 Documentation][2]
[1]: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-managed-notebooks-create.html
[2]: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/emr.html#EMR.Client.start_notebook_execution
【讨论】: