【问题标题】:Kubernetes Cluster master/ Worker NodesKubernetes 集群主/工作节点
【发布时间】:2021-06-22 02:16:04
【问题描述】:

我正在尝试创建一个 Kubernetes 集群,这个集群将包含 3 个节点
主节点,我在其中安装和配置 kubeadm 、 kubelete,并在那里安装了我的系统(这是 laravel 开发的 Web 应用程序), 工作节点毫无问题地加入主节点, 我将我的系统部署到 PHP-fpm pod 并创建了服务和水平 Pods Autoscaling 这是我的服务:

PHP             LoadBalancer   10.108.218.232   <pending>     9000:30026/TCP   15h   app=php

这是我的豆荚

NAME                         READY   STATUS    RESTARTS   AGE   IP            NODE                NOMINATED NODE   READINESS GATES
qsinavphp-5b67996888-9clxp   1/1     Running   0          40m   10.244.0.4    taishan             <none>           <none>
qsinavphp-5b67996888-fnv7c   1/1     Running   0          43m   10.244.0.12   kubernetes-master   <none>           <none>
qsinavphp-5b67996888-gbtdw   1/1     Running   0          40m   10.244.0.3    taishan             <none>           <none>
qsinavphp-5b67996888-l6ghh   1/1     Running   0          33m   10.244.0.2    taishan             <none>           <none>
qsinavphp-5b67996888-ndbc8   1/1     Running   0          43m   10.244.0.11   kubernetes-master   <none>           <none>
qsinavphp-5b67996888-qgdbc   1/1     Running   0          43m   10.244.0.10   kubernetes-master   <none>           <none>
qsinavphp-5b67996888-t97qm   1/1     Running   0          43m   10.244.0.13   kubernetes-master   <none>           <none>
qsinavphp-5b67996888-wgrzb   1/1     Running   0          43m   10.244.0.14   kubernetes-master   <none>           <none>

worker nondes 是 taishan,master 是 Kubernetes-master。 这是我的 nginx 配置,它正在向 php 服务发送请求

server {
 listen 80;
  listen 443  ssl;
    server_name k8s.example.com;
    root /var/www/html/Test/project-starter/public;
        ssl_certificate "/var/www/cert/example.cer";
        ssl_certificate_key "/var/www/cert/example.key";

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    index index.php;
    charset utf-8;
 # if ($scheme = http) {
 #   return 301 https://$server_name$request_uri;
 # }
   ssl_protocols TLSv1.2;
      ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES25>
      ssl_prefer_server_ciphers on;

    location / {

try_files $uri $uri/ /index.php?$query_string;

    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ [^/]\.php(/|$) {
         fastcgi_split_path_info  ^(.+\.php)(/.+)$;
         fastcgi_index            index.php;
         fastcgi_pass             10.108.218.232:9000;
         include                  fastcgi_params;
         fastcgi_param   PATH_INFO       $fastcgi_path_info;
         fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
      }

    location ~ /\.(?!well-known).* {
        deny all;
}
}

问题是我在工作节点上有 3 个 Pod,在主节点上有 5 个 Pod,但是没有请求进入工作节点,所有请求都发给主节点, 我的两个节点都处于就绪状态

NAME                STATUS   ROLES                  AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
kubernetes-master   Ready    control-plane,master   15h   v1.20.4   10.14.0.58    <none>        Ubuntu 20.04.1 LTS   5.4.0-70-generic   docker://19.3.8
taishan             Ready    <none>                 79m   v1.20.5   10.14.2.66    <none>        Ubuntu 20.04.1 LTS   5.4.0-42-generic   docker://19.3.8

这是我的 kubectl describe nodes php 结果

Name:                     php
Namespace:                default
Labels:                   tier=backend
Annotations:              <none>
Selector:                 app=php
Type:                     LoadBalancer
IP Families:              <none>
IP:                       10.108.218.232
IPs:                      10.108.218.232
Port:                     <unset>  9000/TCP
TargetPort:               9000/TCP
NodePort:                 <unset>  30026/TCP
Endpoints:                10.244.0.10:9000,10.244.0.11:9000,10.244.0.12:9000 + 7 more...
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type    Reason  Age   From                Message
  ----    ------  ----  ----                -------
  Normal  Type    48m   service-controller  ClusterIP -> LoadBalancer

这是我用来创建部署的 yaml 文件

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: php
  name: qsinavphp
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: php
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: php
    spec:
      containers:
        - name: taishan-php-fpm
          image: starking8b/taishanphp:last
          imagePullPolicy: Never
          ports:
            - containerPort: 9000

          volumeMounts:


            - name: qsinav-nginx-config-volume
              mountPath: /usr/local/etc/php-fpm.d/www.conf
              subPath: www.conf
            - name: qsinav-nginx-config-volume
              mountPath: /usr/local/etc/php/conf.d/docker-php-memlimit.ini
              subPath: php-memory
            - name: qsinav-php-config-volume
              mountPath: /usr/local/etc/php/php.ini-production
              subPath: php.ini
            - name: qsinav-php-config-volume
              mountPath: /usr/local/etc/php/php.ini-development
              subPath: php.ini
            - name: qsinav-php-config-volume
              mountPath: /usr/local/etc/php-fpm.conf
              subPath: php-fpm.conf

            - name: qsinav-www-storage
              mountPath: /var/www/html/Test/qSinav-starter
          resources:
            limits:
              cpu: 4048m

            requests:
              cpu: 4048m



      restartPolicy: Always
      serviceAccountName: ""
      volumes:
        - name: qsinav-www-storage
          persistentVolumeClaim:
            claimName: qsinav-pv-www-claim
        - name: qsinav-nginx-config-volume
          configMap:
            name: qsinav-nginx-config

        - name: qsinav-php-config-volume
          configMap:
            name: qsinav-php-config
 

这是我的服务 yaml 文件

apiVersion: v1
kind: Service
metadata:
  name: php
  labels:
    tier: backend
spec:
  selector:
    app: php

  ports:
    - protocol: TCP
      port: 9000
  type: LoadBalancer

我不确定我的错误在哪里,所以请帮助解决这个问题

【问题讨论】:

  • 因为你的服务没有正常启动。 > LoadBalancer 10.108.218.232 pending状态表示,服务有问题。
  • 感谢回复,我用服务描述更新了问题,但里面没有显示任何错误,请再次检查问题
  • 我不确定,那我该怎么办?请帮帮我
  • 您能发布您的 pod 部署和服务清单吗?
  • @SachithMuhandiram 你好先生,我刚刚更新了问题请看一下,我在最后一个问题中添加了 yaml 文件

标签: kubernetes minikube


【解决方案1】:

实际上问题出在法兰绒网络上,它无法在节点之间建立连接,所以我通过安装现在工作正常的 weave 插件解决了这个问题 通过应用此命令

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

【讨论】:

    【解决方案2】:

    这里我是从基本的baremetal k8安装中添加的

    ##### Creating ssh keys
    
    From master node
    
    `ssh-keygen`
    
    Copy content in `~/.ssh/id_rsa.pub`
    
    Login to other servers and paste this copied part into `~/.ssh/authorized_keys`
    
    Follow these steps in all servers. Master and worker.
    
    `sudo apt-get install python`
    
    `sudo apt install python3-pip`
    
    Adding Ansible 
    
    `sudo apt-add-repository ppa:ansible/ansible`
    
    `sudo apt update`
    
    `sudo apt-get install ansible -y`
    
    [Reference](https://www.techrepublic.com/article/how-to-install-ansible-on-ubuntu-server-18-04/)
    
    ### Install Kubernetes
    
    `sudo apt-get update`
    
    `sudo apt-get install docker.io`
    
    `sudo systemctl enable docker`
    
    `curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add`
    
    `sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"`
    
    `sudo apt-get install kubeadm kubelet kubectl`
    
    `sudo apt-mark hold kubeadm kubelet kubectl`
    
    For more details please [refer](https://phoenixnap.com/kb/install-kubernetes-on-ubuntu)
    
    ### Installing Kubespray
    
    `git clone https://github.com/kubernetes-incubator/kubespray.git`
    
    `cd kubespray`
    
    `sudo pip3 install -r requirements.txt`
    
    `cp -rfp inventory/sample inventory/mycluster`
    
    `declare -a IPS=(10.10.1.3 10.10.1.4 10.10.1.5)` 
    
    Please put your IP addresses here separated with a space.
    
    `CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}`
    
    `ansible-playbook -i inventory/mycluster/hosts.yaml  --become --become-user=root cluster.yml`
    
    For none root user access 
    `ansible-playbook -i inventory/mycluster/hosts.yaml --become --become-user=root cluster.yml --extra-vars "ansible_sudo_pass=password"`
    
    This will take around 15mins to run successfully. If `root` user ssh is not working properly, this will fail. Please check key sharing step again.
    
    [10 Simple stepms](https://dzone.com/articles/kubespray-10-simple-steps-for-installing-a-product)
    
    [Add a node to existing cluster](https://www.serverlab.ca/tutorials/containers/kubernetes/how-to-add-workers-to-kubernetes-clusters/)
    
    [kubelet debug](https://stackoverflow.com/questions/56463783/how-to-start-kubelet-service)
    
    ### Possible Errors
    
    `kubectl get nodes`
    
    > The connection to the server localhost:8080 was refused - did you specify the right host or port?
    
    Perform followings as normal user (none root user)
    
    `mkdir -p $HOME/.kube`
    
    `sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config`
    
    `sudo chown $(id -u):$(id -g) $HOME/.kube/config`
    
    If you are in worker node, you will have to use `scp` to get `/etc/kubernetes/admin.conf` from master node. Master node may have this problem, if so please do these steps locally using normal user.
    
    [Refer](https://www.edureka.co/community/18633/error-saying-connection-server-localhost-refused-specify)
    
    ## Installing MetalLB
    
    kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/namespace.yaml
    kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.3/manifests/metallb.yaml
    # On first install only
    kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
    
    
    [Official Installation guide](https://metallb.universe.tf/installation/)
    
    ### Configuring L2 config
    
    sachith@master:~$ cat << EOF | kubectl create -f -
    apiVersion: v1
    kind: ConfigMap
    metadata:
      namespace: metallb-system
      name: config
    data:
      config: |
        address-pools:
        - name: default
          protocol: layer2
          addresses:
          - 192.168.1.19-192.168.1.29    # Preferred IP range.
    EOF
    

    使用kubectl describe configmap config -n metallb-system验证安装成功

    这将安装两个组件。

    • 控制器:负责分配 IP。
    • 演讲者:促进服务在 LB 中导航。

    【讨论】:

    • 不,我没有使用 minikube,所以,我该怎么办?因为我没有使用它,所以我按照这个文档安装了 kubernetes kubernetes.io/docs/setup/production-environment/tools/kubeadm/…
    • 实际上在公司有 3 台台山服务器,它们现在在本地,我正在尝试在这些服务器之间创建 Kubernetes 集群,所以我按照我发送给你的链接,为了创建其中一台服务器上的主节点,然后我在另一台服务器上应用了 join 命令
    • 我们可以聊聊吗?所以也许你会对我想做的事情有很好的看法
    • 您有什么建议吗?
    • 嗨,兄弟,感谢您的帮助和询问,是的,实际上问题出在法兰绒网络上,它无法在节点之间建立连接,所以我通过安装 weave 插件解决了它现在工作正常
    猜你喜欢
    • 1970-01-01
    • 2019-04-24
    • 2018-02-20
    • 2019-03-22
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2013-04-24
    • 2023-03-27
    相关资源
    最近更新 更多