GCP Dataproc 的 cloud-sql-proxy 失败，nc：连接到 localhost 端口 3306 (tcp) 失败：连接被拒绝

Question

我正在使用 Terraform 创建一个 dataproc 集群，该集群使用 GCP cloudsql 实例作为 hivemetastore，terrafrm 项目创建集群及其所有先决条件（网络、服务帐户、cloudsql 实例和用户等）。
@提供了 987654321@ 来帮助解决这个问题，但是我无法让它工作，当集群创建时 cloud-sql-proxy.sh 失败并出现错误：

nc：连接到 localhost 端口 3306 (tcp) 失败：连接被拒绝

我把头撞在墙上，试图找出原因，但无法深入了解，所以希望这里有人能提供帮助。

我在https://github.com/jamiekt/democratising-dataproc 主持了 terraform 项目。重现问题非常简单，请按照以下步骤操作：

1. Install terraform 如果你还没有
2. 如果您还没有安装gcloud，请安装
3. 新建一个 GCP 项目
4. 为您的新项目启用Cloud Dataproc API
5. gcloud auth application-default login #creates a file containing credentials that terraform will use
6. git clone git@github.com:jamiekt/democratising-dataproc.git && cd democratising-dataproc
7. export GCP_PROJECT=name-of-project-you-just-created
8. make init
9. make apply

这应该成功启动一个网络、子网、cloudsql 实例、几个存储桶（其中一个包含 cloud-sql-proxy.sh）、一个服务帐户、一个防火墙，然后在尝试创建 dataproc 集群时失败.

如果有人可以看看并告诉我为什么会失败，我将非常感激。

Answer 1

您似乎没有在cloud-sql-proxy.sh.tmpl 模板（diff）中使用最新版本的cloud-sql-proxy.sh 脚本。

您可能想尝试使用 Dataproc Cloud SQL I/O and Hive Metastore 初始化操作中的最新脚本版本更新您的模板。

Answer 2

这里有很多问题现在已经解决了：

• service_account_scopes needed specifying and hive:hive.metastore.warehouse.dir property needed setting
• Terraform removes the default root user from MySQL so it has to be recreated. I was missing (amongst other things) the host attribute (host = '%')
• 其他各种东西

state of the repo at the time of posting this message 将按预期工作（即使用 Terraform 创建一个使用共享 hive 元存储的 dataproc 集群）。

感谢@igor-dvorzhak 的回复，您指向关于配置 Hive Metastore 以使用 Cloud SQL 的文章的链接让我走上了正轨。

Answer 3

按照here 的教程，我遇到了同样的问题

我看不出你给出的修复方法是什么。 似乎 hive 尝试通过 localhost 访问数据库我不明白为什么 这是来自 dataproc 的日志：

Synchronizing state of mysql.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable mysql
Removed /etc/systemd/system/multi-user.target.wants/mysql.service.
2020-10-07 14:05:49 URL:https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 [16361206/16361206] -> "cloud_sql_proxy.linux.amd64" [1]
Created symlink /etc/systemd/system/multi-user.target.wants/cloud-sql-proxy.service → /usr/lib/systemd/system/cloud-sql-proxy.service.
About to run 'nc -zv localhost 3306' with retries...
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 1 failed! Sleeping 1.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 2 failed! Sleeping 1.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 3 failed! Sleeping 2.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 4 failed! Sleeping 3.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 5 failed! Sleeping 5.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 6 failed! Sleeping 8.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 7 failed! Sleeping 13.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 8 failed! Sleeping 21.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 9 failed! Sleeping 34.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 10 failed! Sleeping 55.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 11 failed! Sleeping 89.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 12 failed! Sleeping 144.
Final attempt of 'nc -zv localhost 3306'...
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused