条件“失败”试图通过“certificates.k8s.io/v1”批准 CSR

Question

我有一个有效的 ValidatingWebhookConfiguration 并且一直在创建|批准带有 certificates.k8s.io/v1beta 的 CSR。

我将 (MicroK8s) 从 1.18 升级到 1.20，并收到警告说 certificates.k8s.io/v1beta 已弃用 1.19+，我想我会尝试（但没有成功）升级到 certificates.k8s.io/v1。

现有（工作）CSR：

apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  usages:
  - digital signature
  - key encipherment
  - server auth

升级 API 产生错误：

missing required field "signerName" in io.k8s.api.certificates.v1.CertificateSigningRequestSpec;

我专门阅读了 CSR 文档中关于 Kubernetes signers 的部分，并且由于我现有的规范使用 server auth，假设我可以使用 kubernetes.io/kubelet-serving，因为这是唯一允许 server auth 的规范。

apiVersion: certificates.k8s.io/v1                      <<--- UPGRADED
kind: CertificateSigningRequest
metadata:
  name: ${SERVICE}.${NAMESPACE}
spec:
  groups:
  - system:authenticated
  request: $(cat ${FILENAME}.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kubelet-serving              <<--- ADDED
  usages:
  - digital signature
  - key encipherment
  - server auth

但是，我在尝试批准 CSR（作为集群管理员）时遇到错误：

kubectl certificate approve ${SERVICE}.${NAMESPACE}
certificatesigningrequest.certificates.k8s.io/${SERVICE}.${NAMESPACE} approved

kubectl get csr ${SERVICE}.${NAMESPACE}
NAME                      SIGNERNAME                      REQUESTOR   CONDITION
${SERVICE}.${NAMESPACE}   kubernetes.io/kubelet-serving   admin       Approved,Failed

注意 Approved 但Failed

而且我拿不到证书（大概是因为Failed）：

kubectl get csr ${SERVICE}.${NAMESPACE} \
--output=jsonpath='{.status.certificate}'

我应该如何使用certificates.k8s.io/v1 API？

更新：2021-01-06

好的，所以我意识到我有更多关于“失败”的信息，这让我有一些东西可以调查......

kubectl get csr/${SERVICE}.${NAMESPACE} \
--output=jsonpath="{.status}" \
| jq .

产量：

{
  "conditions": [
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "This CSR was approved by kubectl certificate approve.",
      "reason": "KubectlApprove",
      "status": "True",
      "type": "Approved"
    },
    {
      "lastTransitionTime": "2021-01-06T18:52:15Z",
      "lastUpdateTime": "2021-01-06T18:52:15Z",
      "message": "subject organization is not system:nodes",
      "reason": "SignerValidationFailure",
      "status": "True",
      "type": "Failed"
    }
  ]
}

更新：2021-01-07

感谢@PjoterS

ubectl describe csr/${SERVICE}.${NAMESPACE}
Name:         eldlund.utopial
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certificates.k8s.io/v1","kind":"CertificateSigningRequest","metadata":{"annotations":{},"name":"eldlund.utopial"},"spec":{"groups":["system:authenticated"],"request":"LS0tLS1C...LS0tLS0K","signerName":"kubernetes.io/kubelet-serving","usages":["digital signature","key encipherment","server auth"]}}

CreationTimestamp:  Thu, 07 Jan 2021 17:03:23 +0000
Requesting User:    admin
Signer:             kubernetes.io/kubelet-serving
Status:             Pending
Subject:
  Common Name:    eldlund.utopial.svc
  Serial Number:  
Subject Alternative Names:
         DNS Names:  eldlund.utopial.svc
                     eldlund.utopial.svc.cluster.local
Events:  <none>

使用 OpenSSL（而不是 Kubernetes）进行签名

我尝试创建 CA crt|key，然后创建服务密钥|CSR 并使用 CA 签署服务 CSR，但 Kubernetes 抱怨：

x509: certificate is not valid for any names, but wanted to match ainsley.utopial.svc

但证书似乎同时包含 CN 和 SAN 条目：

不起作用

openssl x509 -in ${FILENAME}.crt --noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            6f:14:25:8c:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = Validating Webhook CA
        Validity
            Not Before: Jan  7 18:10:50 2021 GMT
            Not After : Feb  6 18:10:50 2021 GMT
        Subject: CN = ainsley.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:ca:56:15:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:ainsley.utopial.svc, DNS:ainsley.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         b2:ec:22:b6:...

注意 CN 是上面的 DNS 名称，但下面是 IP ???

使用v1beta1 恢复到我的工作解决方案并更改服务名称以确保完整性 (loi)，Webhook 成功并且证书似乎与上面显示的证书没有什么不同（不同的服务名称除外）：

有效

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ff:b3:cb:11:...
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = 10.152.183.1
        Validity
            Not Before: Jan  7 18:18:45 2021 GMT
            Not After : Jan  7 18:18:45 2022 GMT
        Subject: CN = loi.utopial.svc
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:d2:cc:c2:...
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Authority Key Identifier: 
                keyid:E7:AE:3A:25:95:D2:F7:5B:C6:EA:50:56:07:E8:25:83:60:88:68:7A

            X509v3 Subject Alternative Name: 
                DNS:loi.utopial.svc, DNS:loi.utopial.svc.cluster.local
    Signature Algorithm: sha256WithRSAEncryption
         48:a1:b2:e2:...

Answer 1

更新切换到cert-manager，一切正常well

我得到它的工作，但我不确定为什么我现在做的是正确的。

openssl 感觉很笨拙（感谢建议）。

环境

DIR=${PWD}/secrets
SERVICE="..."
NAMESPACE="..."

FILENAME="${DIR}/${SERVICE}.${NAMESPACE}"

加拿大

openssl req \
-nodes \
-new \
-x509 \
-keyout ${FILENAME}.ca.key \
-out ${FILENAME}.ca.crt \
-subj "/CN=Validating Webhook CA"

创建（Webhook）服务

需要将服务证书的CN设置为IP

cat ./kubernetes/service.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

ENDPOINT=$(\
  kubectl get service/${SERVICE} \
  --namespace=${NAMESPACE} \
  --output=jsonpath="{.spec.clusterIP}") && echo ${ENDPOINT}

创建企业社会责任

即使我在此处包含 CN 和 alt_names，我也必须复制 SAN 内容（下一步）

echo "[ req ]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext

[ dn ]
commonName = ${ENDPOINT}

[ req_ext ]
subjectAltName = @alt_names

[alt_names]
DNS.1 = ${SERVICE}.${NAMESPACE}.svc
DNS.2 = ${SERVICE}.${NAMESPACE}.svc.cluster.local
" > ${FILENAME}.cfg

openssl req \
-nodes \
-new \
-sha256 \
-newkey rsa:2048 \
-keyout ${FILENAME}.key \
-out ${FILENAME}.csr \
-config ${FILENAME}.cfg

创建 CSR 扩展

不确定为什么我必须复制（或分离）此内容。如果我在 openssl x509 -extfile 中省略了它，则证书不包含 SAN 扩展。

printf "subjectAltName=DNS:${SERVICE}.${NAMESPACE}.svc,DNS:${SERVICE}.${NAMESPACE}.svc.cluster.local" > ${FILENAME}.ext

创建服务证书

我怎样才能使用单个 CSR 而不是 CSR+EXT？

openssl x509 \
-req \
-in ${FILENAME}.csr \
-extfile ${FILENAME}.ext \
-CA ${FILENAME}.ca.crt \
-CAkey ${FILENAME}.ca.key \
-CAcreateserial \
-out ${FILENAME}.crt

创建（Webhook）部署

webhook的底层实现需要服务的crt|key

kubectl create secret tls ${SERVICE} \
--namespace=${NAMESPACE} \
--cert=${FILENAME}.crt \
--key=${FILENAME}.key

cat ./kubernetes/deployment.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

创建网络钩子

获取 CA 证书

CABUNDLE=$(openssl base64 -A <"${FILENAME}.ca.crt")

cat ./kubernetes/webhook.yaml \
| sed "s|SERVICE|${SERVICE}|g" \
| sed "s|NAMESPACE|${NAMESPACE}|g" \
| sed "s|CABUNDLE|${CABUNDLE}|g" \
| kubectl apply --filename=- --namespace=${NAMESPACE}

Answer 2

仅供参考，通过使用类似 kubelet 的请求模拟您的应用 CSR，您可以继续创建 k8s 证书。我有这个问题的详细信息，https://github.com/kubernetes/website/issues/26111，试图让 k8s 文档在这个主题上更有用。