【问题标题】:ETCD 3.4.3 not starting with systemd fileETCD 3.4.3 不以 systemd 文件开头
【发布时间】:2020-02-29 11:23:26
【问题描述】:

我正在尝试运行单节点 ETCD 3.4.3 集群。我以前使用相同的配置文件运行 ETCD 集群,但现在该配置不适用于任何版本的 ETCD。 ETCD 服务器是使用 Vargrant(2.2.6) 生成的 virtualbox(6.0.14r133895) 机器

我为 ETCD 使用以下 systemd 文件

[Unit]
Description=Etcd Server
Documentation=https://github.com/coreos/etcd
After=network.target

[Service]
User=etcd
Type=notify
EnvironmentFile=/etc/default/etcd.conf
ExecStart=/usr/local/bin/etcd

Restart=always
RestartSec=10s
LimitNOFILE=40000
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target

ETCD配置环境文件如下

$ cat /etc/default/etcd.conf
ETCD_NAME=k8s-server-1
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://k8s-server-1:2380
ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
ETCD_ADVERTISE_CLIENT_URLS=https://k8s-server-1:2379,https://127.0.0.1:2379
ETCD_DISCOVERY=https://discovery.etcd.io/78dcc2d35f9a2c1d60f3d457d5a36bfe
ETCD_DISCOVERY_FALLBACK='exit'
ETCD_DATA_DIR=/var/lib/etcd/data
ETCD_WAL_DIR=/var/lib/etcd/wal
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_CLIENT_CERT_AUTH=true
ETCD_TRUSTED_CA_FILE=/etc/tls/k8s/ca/ca.pem
ETCD_CERT_FILE=/etc/tls/k8s/etcd/k8s-server-1-client.pem
ETCD_KEY_FILE=/etc/tls/k8s/etcd/k8s-server-1-client-key.pem
ETCD_PEER_CLIENT_CERT_AUTH=true
ETCD_PEER_TRUSTED_CA_FILE=/etc/tls/k8s/ca/ca.pem
ETCD_PEER_CERT_FILE=/etc/tls/k8s/etcd/k8s-server-1-peer.pem
ETCD_PEER_KEY_FILE=/etc/tls/k8s/etcd/k8s-server-1-peer-key.pem
ETCD_HEARTBEAT_INTERVAL=6000
ETCD_ELECTION_TIMEOUT=30000
ETCD_ENABLE_V2=false
ETCD_LOG_LEVEL=debug
GOMAXPROCS=2

使用上述配置,ETCD 启动失败并出现以下错误

Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=https://k8s-server-1:2379,https://127.0.0.1:2379
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_CERT_FILE=/etc/tls/k8s/etcd/k8s-server-1-client.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_CLIENT_CERT_AUTH=true
Nov 04 05:41:28 k8s-server-1 etcd[15760]: [WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd/data
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/78dcc2d35f9a2c1d60f3d457d5a36bfe
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_DISCOVERY_FALLBACK=exit
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_ELECTION_TIMEOUT=30000
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_ENABLE_V2=false
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_HEARTBEAT_INTERVAL=6000
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=https://k8s-server-1:2380
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_KEY_FILE=/etc/tls/k8s/etcd/k8s-server-1-client-key.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=https://0.0.0.0:2379
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=https://0.0.0.0:2380
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_LOG_LEVEL=debug
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_NAME=k8s-server-1
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_PEER_CERT_FILE=/etc/tls/k8s/etcd/k8s-server-1-peer.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_PEER_CLIENT_CERT_AUTH=true
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_PEER_KEY_FILE=/etc/tls/k8s/etcd/k8s-server-1-peer-key.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_PEER_TRUSTED_CA_FILE=/etc/tls/k8s/ca/ca.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_TRUSTED_CA_FILE=/etc/tls/k8s/ca/ca.pem
Nov 04 05:41:28 k8s-server-1 etcd[15760]: recognized and used environment variable ETCD_WAL_DIR=/var/lib/etcd/wal
Nov 04 05:41:28 k8s-server-1 etcd[15760]: etcd Version: 3.4.3
Nov 04 05:41:28 k8s-server-1 etcd[15760]: Git SHA: 3cf2f69b5
Nov 04 05:41:28 k8s-server-1 etcd[15760]: Go Version: go1.12.12
Nov 04 05:41:28 k8s-server-1 etcd[15760]: Go OS/Arch: linux/amd64
Nov 04 05:41:28 k8s-server-1 etcd[15760]: setting maximum number of CPUs to 2, total number of available CPUs is 2
Nov 04 05:41:28 k8s-server-1 etcd[15760]: the server is already initialized as member before, starting as etcd member...
Nov 04 05:41:28 k8s-server-1 etcd[15760]: peerTLS: cert = /etc/tls/k8s/etcd/k8s-server-1-peer.pem, key = /etc/tls/k8s/etcd/k8s-server-1-peer-key.pem, trusted-ca = /etc/tls/k8s/ca/ca.pem, client-cert-auth = true, crl-file = 
Nov 04 05:41:28 k8s-server-1 etcd[15760]: name = k8s-server-1
Nov 04 05:41:28 k8s-server-1 etcd[15760]: data dir = /var/lib/etcd/data
Nov 04 05:41:28 k8s-server-1 etcd[15760]: member dir = /var/lib/etcd/data/member
Nov 04 05:41:28 k8s-server-1 etcd[15760]: dedicated WAL dir = /var/lib/etcd/wal
Nov 04 05:41:28 k8s-server-1 etcd[15760]: heartbeat = 6000ms
Nov 04 05:41:28 k8s-server-1 etcd[15760]: election = 30000ms
Nov 04 05:41:28 k8s-server-1 etcd[15760]: snapshot count = 100000
Nov 04 05:41:28 k8s-server-1 etcd[15760]: discovery URL= https://discovery.etcd.io/78dcc2d35f9a2c1d60f3d457d5a36bfe
Nov 04 05:41:28 k8s-server-1 etcd[15760]: advertise client URLs = https://127.0.0.1:2379,https://k8s-server-1:2379
Nov 04 05:41:28 k8s-server-1 etcd[15760]: {"level":"info","ts":1572846088.8206353,"caller":"netutil/netutil.go:112","msg":"resolved URL Host","url":"https://k8s-server-1:2380","host":"k8s-server-1:2380","resolved-addr":"127.0.1.1:2380"}
Nov 04 05:41:28 k8s-server-1 etcd[15760]: {"level":"info","ts":1572846088.8212357,"caller":"netutil/netutil.go:112","msg":"resolved URL Host","url":"https://k8s-server-1:2380","host":"k8s-server-1:2380","resolved-addr":"127.0.1.1:2380"}
Nov 04 05:41:33 k8s-server-1 etcd[15760]: failed to join discovery cluster (context deadline exceeded)
Nov 04 05:41:33 k8s-server-1 etcd[15760]: discovery token https://discovery.etcd.io/78dcc2d35f9a2c1d60f3d457d5a36bfe was used, but failed to bootstrap the cluster.
Nov 04 05:41:33 k8s-server-1 etcd[15760]: please generate a new discovery token and try to bootstrap again.
Nov 04 05:41:33 k8s-server-1 systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE
Nov 04 05:41:33 k8s-server-1 systemd[1]: etcd.service: Failed with result 'exit-code'.
Nov 04 05:41:33 k8s-server-1 systemd[1]: Failed to start Etcd Server.
-- Subject: Unit etcd.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
-- 
-- Unit etcd.service has failed.

但是,我可以手动启动 ETCD,如下所示

$ sudo su etcd
etcd@k8s-server-1:~$ etcd --name $ETCD_NAME   \
    --initial-advertise-peer-urls $ETCD_INITIAL_ADVERTISE_PEER_URLS  \
    --advertise-client-urls $ETCD_ADVERTISE_CLIENT_URLS  \
    --client-cert-auth \
    --trusted-ca-file $ETCD_TRUSTED_CA_FILE  \
    --cert-file $ETCD_CERT_FILE  \
    --key-file $ETCD_KEY_FILE  \
    --peer-client-cert-auth  \
    --peer-trusted-ca-file $ETCD_PEER_TRUSTED_CA_FILE  \
    --peer-cert-file $ETCD_PEER_CERT_FILE  \
    --peer-key-file $ETCD_PEER_KEY_FILE  \
    --listen-peer-urls $ETCD_LISTEN_PEER_URLS  \
    --listen-client-urls $ETCD_LISTEN_CLIENT_URLS \
    --initial-cluster-token $ETCD_INITIAL_CLUSTER_TOKEN 

一旦我手动启动 ETCD,systemd 单元文件将继续使用现有配置。谁能告诉我为什么 ETCD 无法从 systemd 文件开始。

【问题讨论】:

  • 您在使用发现服务吗?因为这就是它试图创建集群(和超时)的方式。手动执行时,您只是在宣传同行。好吧,同行。
  • @suren 感谢您指出发现服务。是的,我正在使用发现服务来启动集群。我会尝试增加发现时间。希望这会奏效。
  • 我不认为它会解决这个问题。
  • 有什么建议吗?我认为在这个发现过程中,互联网速度也可能是一个问题。
  • 如果看不到它正在启动什么命令,我会做systemctl status etcd,然后就可以了。将确认发现服务 URL 实际上是有效的。很可能它无法到达那个 URL,所以它超时了。

标签: kubernetes systemd etcd


【解决方案1】:

解决了删除发现服务以创建集群的问题。而是使用特定节点的 IP 创建集群。此外,如果互联网速度不够快,发现机制似乎表现异常。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2020-05-31
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多