使用 oci 堡垒服务连接到私有 kubernetes 集群时出错答案

【问题标题】：Error connecting to private kubernetes cluster using oci bastion service使用 oci 堡垒服务连接到私有 kubernetes 集群时出错
【发布时间】：2022-01-04 23:10:43
【问题描述】：

我刚刚在 Oracle Cloud 上创建了一个私有 Kubernetes 集群。连接到集群 API 的常规方式是通过 Bastion 服务。我已按照本文中提到的确切步骤进行操作：https://www.ateam-oracle.com/using-oci-bastion-service-to-manage-private-oke-kubernetes-clusters

在执行 ssh 命令端口转发（文章中的第 4 步）后，shell 按预期阻塞，但运行 kubectl 没有得到任何合理的输出：

$ kubectl cluster-info

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Unable to connect to the server: net/http: TLS handshake timeout

这是将-v 传递给 ssh 时的输出：

OpenSSH_8.4p1, OpenSSL 1.1.1k  25 Mar 2021
debug1: Reading configuration data /home/praj/.ssh/config
debug1: Reading configuration data /usr/etc/ssh/ssh_config
debug1: /usr/etc/ssh/ssh_config line 24: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /usr/etc/ssh/ssh_config line 26: Applying options for *
debug1: Connecting to host.bastion.ap-mumbai-1.oci.oraclecloud.com [192.29.162.226] port 22.
debug1: Connection established.
debug1: identity file /home/praj/.ssh/id_rsa type 0
debug1: identity file /home/praj/.ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4
debug1: Remote protocol version 2.0, remote software version Go
debug1: no match: Go
debug1: Authenticating to host.bastion.ap-mumbai-1.oci.oraclecloud.com:22 as 'ocid1.bastionsession.oc1.ap-mumbai-1.amaaaaaafvm2mgaa5inuqsfwe73eitjgead23h2avusdwryx5hlz6orz7jea'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256-etm@openssh.com compression: none
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: kex: curve25519-sha256@libssh.org need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:JTeqM8qvS9EO9reRIF/qyllvs6px8Y69LEveK9NFzZc
debug1: Host 'host.bastion.ap-mumbai-1.oci.oraclecloud.com' is known and matches the RSA host key.
debug1: Found key in /home/praj/.ssh/known_hosts:13
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 4294967296 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: /home/praj/.ssh/id_rsa RSA SHA256:380ueVYrrzxGrkPRep4huj+pHdElPoz8iCTSYvKD5Hg explicit
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/praj/.ssh/id_rsa RSA SHA256:380ueVYrrzxGrkPRep4huj+pHdElPoz8iCTSYvKD5Hg explicit
debug1: Server accepts key: /home/praj/.ssh/id_rsa RSA SHA256:380ueVYrrzxGrkPRep4huj+pHdElPoz8iCTSYvKD5Hg explicit
Enter passphrase for key '/home/praj/.ssh/id_rsa': 
debug1: Authentication succeeded (publickey).
Authenticated to host.bastion.ap-mumbai-1.oci.oraclecloud.com ([192.29.162.226]:22).
debug1: Local connections to LOCALHOST:6443 forwarded to remote address 10.0.0.14:6443
debug1: Local forwarding listening on 127.0.0.1 port 6443.
debug1: channel 0: new [port listener]
debug1: Entering interactive session.
debug1: pledge: network
debug1: Connection to port 6443 forwarding to 10.0.0.14 port 6443 requested.
debug1: channel 1: new [direct-tcpip]
debug1: Connection to port 6443 forwarding to 10.0.0.14 port 6443 requested.
debug1: channel 2: new [direct-tcpip]
debug1: Connection to port 6443 forwarding to 10.0.0.14 port 6443 requested.
debug1: channel 3: new [direct-tcpip]
debug1: Connection to port 6443 forwarding to 10.0.0.14 port 6443 requested.
debug1: channel 4: new [direct-tcpip]
debug1: channel 1: free: direct-tcpip: listening port 6443 for 10.0.0.14 port 6443, connect from 127.0.0.1 port 44054 to 127.0.0.1 port 6443, nchannels 5
debug1: Connection to port 6443 forwarding to 10.0.0.14 port 6443 requested.
debug1: channel 1: new [direct-tcpip]
debug1: channel 2: free: direct-tcpip: listening port 6443 for 10.0.0.14 port 6443, connect from 127.0.0.1 port 44056 to 127.0.0.1 port 6443, nchannels 5
debug1: channel 3: free: direct-tcpip: listening port 6443 for 10.0.0.14 port 6443, connect from 127.0.0.1 port 44058 to 127.0.0.1 port 6443, nchannels 4
debug1: channel 4: free: direct-tcpip: listening port 6443 for 10.0.0.14 port 6443, connect from 127.0.0.1 port 44060 to 127.0.0.1 port 6443, nchannels 3
debug1: channel 1: free: direct-tcpip: listening port 6443 for 10.0.0.14 port 6443, connect from 127.0.0.1 port 44062 to 127.0.0.1 port 6443, nchannels 2
^Cdebug1: channel 0: free: port listener, nchannels 1
Killed by signal 2.

我的集群在两个基于 ARM 的节点（A1 灵活 VM）上运行，默认使用 Oracle Linux 7.9 作为操作系统，Kubernetes 版本为 1.20.8

谁能告诉我问题出在哪里？是否需要任何额外的配置才能连接到 Kubernetes API？

【问题讨论】：

可能的原因是您现在可能在安全列表中有正确的条目以允许从 Bastion 连接到 Kubernetis 节点。请检查您是否有从堡垒（主要是 192.X.X.X 或 172.X.X.X）到 kubernetis VM 的入口通信
我也尝试过，允许从 Bastion 的 IP 连接到 Kube API 端口 (6443)。还值得注意的是，6443 的默认入口规则首先允许 0.0.0.0/0（完全互联网）。但奇怪的是没有奏效。也许是 oci 的一些错误/服务问题，我无法捕捉到。
堡垒和 Kubernetes 集群在同一个 VCN 上。请检查它们之间的路由表中是否有路由。 docs.oracle.com/en-us/iaas/Content/Network/Tasks/…

标签： linux kubernetes ssh oracle-cloud-infrastructure bastion-host

【解决方案1】：

快速提问：

假设您已经为堡垒子网的安全列表设置了正确的入口规则，并且已经创建了会话和端口转发（SSH 隧道），您是否能够通过端口 6443 上的简单 SSH 命令从您的 PC 登录到工作节点？

注意：您需要私钥才能登录工作程序节点，并且工作程序节点应该已使用公钥创建以进行交互式登录。

【讨论】：

【解决方案2】：

我可以通过 SSH 从本地机器连接到私有 K8s 集群，步骤如下：

在集群的同一 VCN 内为堡垒目标子网创建一个单独的公共子网。
为新子网使用集群网络的默认公共路由表。此路由表已经有一个带有互联网网关目标的路由规则，该规则将允许从本地计算机到集群 API 的流量，反之亦然。
将安全列表添加到出口规则为 0.0.0.0/0 的子网作为目标 CIDR，允许所有端口。
确保将子网的端口 CIDR 添加到集群 API 使用的子网的安全列表入口。
创建 OCI 堡垒。请记住将新子网用作目标子网。

【讨论】：