用于运行 ConfigMap 的 CronJob

Question

我正在尝试编写一个 CronJob 来在 Kafka 的 ConfigMap 中执行 shell 脚本。

我的意图是在特定的时间间隔重新分配分区。

但是，我遇到了一些问题。我对它很陌生。任何帮助将不胜感激。

cron-job.yaml

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: partition-cron
spec:
  schedule: "*/10 * * * *"
  startingDeadlineSeconds: 20
  successfulJobsHistoryLimit: 5
  jobTemplate:
    spec:
      completions: 2
      template:
        spec:
          containers:
          - name: partition-reassignment
            image: busybox
            command: ["/configmap/runtimeConfig.sh"]
            volumeMounts:
            - name: configmap
              mountPath: /configmap
          restartPolicy: Never
          volumes:
            - name: configmap
              configMap:
                name: configmap-config

configmap-config.yaml

{{- if .Values.topics -}}
{{- $zk := include "zookeeper.url" . -}}
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: {{ template "kafka.fullname" . }}
    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    heritage: "{{ .Release.Service }}"
    release: "{{ .Release.Name }}"
  name: {{ template "kafka.fullname" . }}-config
data:
  runtimeConfig.sh: |
    #!/bin/bash
    set -e
    cd /usr/bin
    until kafka-configs --zookeeper {{ $zk }} --entity-type topics --describe || (( count++ >= 6 ))
    do
      echo "Waiting for Zookeeper..."
      sleep 20
    done
    until nc -z {{ template "kafka.fullname" . }} 9092 || (( retries++ >= 6 ))
    do
      echo "Waiting for Kafka..."
      sleep 20
    done
    echo "Applying runtime configuration using {{ .Values.image }}:{{ .Values.imageTag }}"
  {{- range $n, $topic := .Values.topics }}
    {{- if and $topic.partitions $topic.replicationFactor $topic.reassignPartitions }}
    cat << EOF > {{ $topic.name }}-increase-replication-factor.json
  {"version":1, "partitions":[
    {{- $partitions := (int $topic.partitions) }}
    {{- $replicas := (int $topic.replicationFactor) }}
    {{- range $i := until $partitions }}
      {"topic":"{{ $topic.name }}","partition":{{ $i }},"replicas":[{{- range $j := until $replicas }}{{ $j }}{{- if ne $j (sub $replicas 1) }},{{- end }}{{- end }}]}{{- if ne $i (sub $partitions 1) }},{{- end }}
    {{- end }}
  ]}
EOF
kafka-reassign-partitions --zookeeper {{ $zk }} --reassignment-json-file {{ $topic.name }}-increase-replication-factor.json --execute
kafka-reassign-partitions --zookeeper {{ $zk }} --reassignment-json-file {{ $topic.name }}-increase-replication-factor.json --verify
  {{- end }}
{{- end -}}

我的目的是定期将 runtimeConfig.sh 脚本作为 cron 作业运行，以便在 Kafka 中重新分配分区。

我不确定我的方法是否正确。

另外，我随机将 image:busybox 放入 cron-job.yaml 文件中。我不确定我应该在里面放什么。

信息部分

$ kubectl get cronjobs
NAME             SCHEDULE       SUSPEND   ACTIVE    LAST SCHEDULE   AGE
partition-cron   */10 * * * *   False     1         5m              12m
$ kubectl get pods
NAME                                               READY     STATUS              RESTARTS   AGE
elegant-hedgehog-metrics-server-58995fcf8b-2vzg6   1/1       Running             0          5d
my-kafka-0                                         1/1       Running             1          12m
my-kafka-1                                         1/1       Running             0          10m
my-kafka-2                                         1/1       Running             0          9m
my-kafka-config-644f815a-pbpl8                     0/1       Completed           0          12m
my-kafka-zookeeper-0                               1/1       Running             0          12m
partition-cron-1548672000-w728w                    0/1       ContainerCreating   0          5m
$ kubectl logs partition-cron-1548672000-w728w
Error from server (BadRequest): container "partition-reassignment" in pod "partition-cron-1548672000-w728w" is waiting to start: ContainerCreating

修改后的 Cron 作业 YAML

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: partition-cron
spec:
  schedule: "*/5 * * * *"
  startingDeadlineSeconds: 20
  successfulJobsHistoryLimit: 5
  jobTemplate:
    spec:
      completions: 1
      template:
        spec:
          containers:
          - name: partition-reassignment
            image: busybox
            command: ["/configmap/runtimeConfig.sh"]
            volumeMounts:
              - name: configmap
                mountPath: /configmap
          restartPolicy: Never
          volumes:
            - name: configmap
              configMap:
                name: {{ template "kafka.fullname" . }}-config

现在，我将 Cron Job pod 的状态设为 ContainerCannotRun。

Answer 1

您已将 ConfigMap 设置为 name: {{ template "kafka.fullname" . }}-config，但在作业中您正在安装 configmap-config。除非您使用 configmap 作为版本名称安装 Helm 图表，否则该 Job 将永远不会启动。

修复它的一种方法是将音量定义为：

          volumes:
            - name: configmap
              configMap:
                name: {{ template "kafka.fullname" . }}-config