使用kubeadm部署k8s集群
环境
| IP地址 | 主机名 | 节点 |
|---|---|---|
| 10.0.0.63 | k8s-master1 | master1 |
| 10.0.0.63 | k8s-master2 | master2 |
| 10.0.0.65 | k8s-node1 | node1 |
| 10.0.0.66 | k8s-node2 | node2 |
1. 简要
kubeadm是官方社区推出的快速部署kubernetes集群工具
部署环境适用于学习和使用k8s相关软件和功能
2. 安装要求
3台纯净centos虚拟机,版本为7.x及以上
机器配置 2核4G以上 x3台
服务器网络互通
禁止swap分区
3. 学习目标
学会使用kubeadm来安装一个集群,便于学习k8s相关知识
4. 环境准备
# 1. 关闭防火墙功能
systemctl stop firewalld
systemctl disable firewalld
# 2.关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config
setenforce 0
# 3. 关闭swap
swapoff -a
#或将命令加入开机启动
echo "swapoff -a" >>/etc/profile
source /etc/profile
# 4. 服务器规划
cat > /etc/hosts << EOF
10.0.0.63 k8s-master1
10.0.0.64 k8s-master2
10.0.0.65 k8s-node1
10.0.0.66 k8s-node2
EOF
#5. 临时主机名配置方法:
hostnamectl set-hostname k8s-master1
bash
#6. 时间同步配置
yum install -y ntpdate
ntpdate time.windows.com
#开启转发
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
#7. 时间同步
echo '*/5 * * * * /usr/sbin/ntpdate -u ntp.api.bz' >>/var/spool/cron/root
systemctl restart crond.service
crontab -l
# 以上可以全部复制粘贴直接运行,但是主机名配置需要重新修改
5. docker安装[所有节点都需要安装]
#源添加
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
wget -P /etc/yum.repos.d/ http://mirrors.aliyun.com/repo/epel-7.repo
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum clean all
yum install -y bash-completion.noarch
# 安装指定版版本
yum -y install docker-ce-18.09.9-3.el7
#也可以查看版本安装
yum list docker-ce --showduplicates | sort -r
#启动docker
systemctl enable docker
systemctl start docker
systemctl status docker
6. docker配置cgroup驱动[所有节点]
rm -f /etc/docker/*
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://ajvcw8qn.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"]
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
拉取flanel镜像:
docker pull lizhenliang/flannel:v0.11.0-amd64
7. 镜像加速[所有节点]
curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
systemctl restart docker
#如果源太多容易出错. 错了就删除一个.bak源试试看
#保留 curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io
这个是阿里云配置的加速
https://cr.console.aliyun.com/cn-hangzhou/instances/mirrors
8.kubernetes源配置[所有节点]
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
9. 安装kubeadm，kubelet和kubectl[所有节点]
yum install -y kubelet-1.18.0 kubeadm-1.18.0 kubectl-1.18.0
systemctl enable kubelet
10. 部署Kubernetes Master [ master 10.0.0.63]
kubeadm init \
--apiserver-advertise-address=10.0.0.63 \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.18.0 \
--service-cidr=10.1.0.0/16 \
--pod-network-cidr=10.244.0.0/16
#成功后加入环境变量:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
初始化后获取到token:
kubeadm join 10.0.0.63:6443 --token 2cdgi6.79j20fhly6xpgfud
--discovery-token-ca-cert-hash sha256:3d847b858ed649244b4110d4d60ffd57f43856f42ca9c22e12ca33946673ccb4
记住token,后面使用
注意:
W0507 00:43:52.681429 3118 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io] [init] Using Kubernetes version: v1.18.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/ error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2 [preflight] If you know what you are doing, you can make a check non-fatal with --ignore-preflight-errors=... To see the stack trace of this error execute with --v=5 or higher
10.1 报错处理
报错1: 需要修改docker驱动为systemd /etc/docker/daemon.json 文件中加入: "exec-opts": ["native.cgroupdriver=systemd"]
报错2: [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2出现该报错,是cpu有限制,将cpu修改为2核4G以上配置即可
报错2: 出现该报错,是cpu有限制,将cpu修改为2核4G以上配置即可
报错3: 加入集群出现报错:
W0507 01:19:49.406337 26642 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR DirAvailable--etc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty
[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
[root@k8s-master2 yum.repos.d]# kubeadm join 10.0.0.63:6443 --token q8bfij.fipmsxdgv8sgcyq4 \
> --discovery-token-ca-cert-hash sha256:26fc15b6e52385074810fdbbd53d1ba23269b39ca2e3ec3bac9376ed807b595c
> --discovery-token-ca-cert-hash sha256:26fc15b6e52385074810fdbbd53d1ba23269b39ca2e3ec3bac9376ed807b595c
W0507 01:20:26.246981 26853 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR DirAvailable--etc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty
[ERROR FileAvailable--etc-kubernetes-kubelet.conf]: /etc/kubernetes/kubelet.conf already exists
[ERROR FileAvailable--etc-kubernetes-pki-ca.crt]: /etc/kubernetes/pki/ca.crt already exists
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
解决办法:
执行: kubeadm reset 重新加入
10.2. kubectl命令工具配置[master]：
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
#获取节点信息
# kubectl get nodes
[root@k8s-master1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 NotReady master 2m59s v1.18.0
k8s-node1 NotReady <none> 86s v1.18.0
k8s-node2 NotReady <none> 85s v1.18.0
#可以获取到其他主机的状态信息,证明集群完毕,另一台k8s-master2 没有加入到集群中,是因为要做多master,这里就不加了.
10.2. 安装网络插件[master]
[直在master上操作]上传kube-flannel.yaml,并执行:
kubectl apply -f kube-flannel.yaml
kubectl get pods -n kube-system
下载地址:
https://www.chenleilei.net/soft/k8s/kube-flannel.yaml
[必须全部运行起来,否则有问题.]
[root@k8s-master1 ~]# kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7ff77c879f-5dq4s 1/1 Running 0 13m
coredns-7ff77c879f-v68pc 1/1 Running 0 13m
etcd-k8s-master1 1/1 Running 0 13m
kube-apiserver-k8s-master1 1/1 Running 0 13m
kube-controller-manager-k8s-master1 1/1 Running 0 13m
kube-flannel-ds-amd64-2ktxw 1/1 Running 0 3m45s
kube-flannel-ds-amd64-fd2cb 1/1 Running 0 3m45s
kube-flannel-ds-amd64-hb2zr 1/1 Running 0 3m45s
kube-proxy-4vt8f 1/1 Running 0 13m
kube-proxy-5nv5t 1/1 Running 0 12m
kube-proxy-9fgzh 1/1 Running 0 12m
kube-scheduler-k8s-master1 1/1 Running 0 13m
[root@k8s-master1 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 14m v1.18.0
k8s-node1 Ready <none> 12m v1.18.0
k8s-node2 Ready <none> 12m v1.18.0
11. 将node1 node2 加入master
node1加入配置
在要加入的节点种执行以下命令来加入:
kubeadm join 10.0.0.63:6443 --token fs0uwh.7yuiawec8tov5igh \
--discovery-token-ca-cert-hash sha256:471442895b5fb77174103553dc13a4b4681203fbff638e055ce244639342701d
#这个配置在安装master的时候有过提示,请注意首先要配置cni网络插件
#加入成功后,master节点检测:
[root@k8s-master1 docker]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master1 Ready master 14m v1.18.0
k8s-node1 Ready <none> 12m v1.18.0
k8s-node2 Ready <none> 12m v1.18.0
12 token创建和查询
默认token会保存24消失,过期后就不可用,如果需要重新建立token,可在master节点使用以下命令重新生成:
kubeadm token create
kubeadm token list
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
结果:
3d847b858ed649244b4110d4d60ffd57f43856f42ca9c22e12ca33946673ccb4
新token加入集群方法:
kubeadm join 10.0.0.63:6443 --discovery-token nuja6n.o3jrhsffiqs9swnu --discovery-token-ca-cert-hash 63bca849e0e01691ae14eab449570284f0c3ddeea590f8da988c07fe2729e924
13. 安装dashboard界面
[root@k8s-master1 ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.1.94.43 <none> 8000/TCP 7m58s
kubernetes-dashboard NodePort 10.1.187.162 <none> 443:30001/TCP 7m58s
13.1 访问测试
10.0.0.63 10.0.0.64 10.0.0.65 集群任意一个角色访问30001端口都可以访问到dashboard页面.
13.2 获取dashboard token, 也就是创建service account并绑定默认cluster-admin管理员集群角色
# kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
# kubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
# kubectl describe secrets -n kubernetes-dashboard $(kubectl -n kubernetes-dashboard get secret | awk '/dashboard-admin/{print $1}')
将复制的token 填写到 上图中的 token选项,并选择token登录
报错处理:
问题1 k8s-node节点加入时报错::
k8s-node节点加入时报错:
W0315 22:16:20.123204 5795 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
处理办法:
echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables
增加后重新加入:
kubeadm join 10.0.0.63:6443 --token 0dr1pw.ejybkufnjpalb8k6 --discovery-token-ca-cert-hash sha256:ca1aa9cb753a26d0185e3df410cad09d8ec4af4d7432d127f503f41bc2b14f2a
这里的token由kubadm服务器生成.
问题2: web页面无法访问处理:
重建dashboard
删除:
kubectl delete -f dashboard.yaml
删除后创建:
kubectl create -f dashboard.yaml
创建账户:
kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
查看密码:
kubectl describe secrets -n kubernetes-dashboard $(kubectl -n kubernetes-dashboard get secret | awk '/dashboard-admin/{print $1}')
重新打开登录即可
问题3: 部署dashboard失败
有可能是网络问题,需要切换一个别的网络,比如vpn,然后重新部署.
YAML附件[请保存为 .yaml 为后缀]
http://www.chenleilei.net/soft/kubeadm快速部署一个Kubernetes集群yaml.zip