【发布时间】：2023-02-15 19:04:33
【问题描述】：
您好，我们在 DR 规划中使用 velero，我们正在研究跨区域备份恢复策略， 我们正在备份工作负载、PV 和 PVC 我们在将备份从 (US-EAST-2) 恢复到第二个区域 (US-West-2) 时遇到问题。
使用以下命令在两个集群上安装都没有任何问题
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.4.0 \
--bucket velerobucket\
--backup-location-config region=us-east-2 \
--snapshot-location-config region=us-east-2 \
--secret-file secret-file
备份创建也没有任何错误
velero backup create zookeeperbkp --include-namespaces zookeeper --snapshot-volumes
从 us-east-2 在 us-west-2 集群上进行恢复时，恢复成功完成，velero 恢复日志中没有任何错误 但是 zookeeper pod 进入 pending 状态
velero restore create --from-backup zookeeperbkp
kubectl get pods -n zookeeper
NAME READY STATUS RESTARTS AGE
zookeeper-0 0/2 Pending 0 3m24s
zookeeper-1 0/2 Pending 0 3m24s
zookeeper-2 0/2 Pending 0 3m24s
在描述了它抱怨的豆荚之后
0/1 nodes are available: 1 node(s) had volume node affinity conflict.
在描述了 PV 之后，它似乎试图在 us-east-2 中创建 PV，标签是 us-east-2 而它应该是 us-west-2（恢复集群）
毕竟，我阅读了更多关于 velero 在跨区域集群中恢复 PV 和 PVC 的限制。人们在 S3 中修改了 velero jsons 的链接 https://github.com/vmware-tanzu/velero/issues/1624
我尝试通过修改 s3 中的 velero 快照 json 文件来做同样的事情
aws s3 cp s3://velerobkpxyz/backups/zookeeper/ ./ --recursive
gunzip zookeeper-volumesnapshots.json.gz
sed -i "s/us-east-2/us-west-2/g" zookeeper-volumesnapshots.json
s3 cp zookeeper-volumesnapshots.json.gz s3://velerobkp/backups/zookeeper/zookeeper-volumesnapshots.json.gz
同样，我对 zookeeper.tar.gz 进行了更改
mkdir zookeeper-temp
tar xzf zookeeper.tar.gz -C zookeeper-temp/
cd zookeeper-temp/
find . -name \*.json -exec sh -c "sed -i 's/us-east-2/us-west-2/g' {}" \;
tar czf ../zookeeper.tar.gz *
aws s3 cp zookeeper.tar.gz s3://velerobkp/backups/zookeeper/
在此之后，描述的备份会为 PV 提供正确的区域名称
velero backup describe zookeeper --details
Name: zookeeper9
Namespace: velero
Labels: velero.io/storage-location=default
Annotations: velero.io/source-cluster-k8s-gitversion=v1.21.5-eks-bc4871b
velero.io/source-cluster-k8s-major-version=1
velero.io/source-cluster-k8s-minor-version=21+
Phase: Completed
Errors: 0
Warnings: 0
Namespaces:
Included: zookeeper
Excluded: <none>
Resources:
Included: *
Excluded: <none>
Cluster-scoped: auto
Label selector: <none>
Storage Location: default
Velero-Native Snapshot PVs: true
TTL: 720h0m0s
Hooks: <none>
Backup Format Version: 1.1.0
Started: 2022-03-30 20:37:53 +0530 IST
Completed: 2022-03-30 20:37:57 +0530 IST
Expiration: 2022-04-29 20:37:53 +0530 IST
Total items to be backed up: 52
Items backed up: 52
Resource List:
apiextensions.k8s.io/v1/CustomResourceDefinition:
- servicemonitors.monitoring.coreos.com
apps/v1/ControllerRevision:
- zookeeper/zookeeper-596cddb599
- zookeeper/zookeeper-5977bdccb6
- zookeeper/zookeeper-5cd569cbf9
- zookeeper/zookeeper-6585c9bc89
- zookeeper/zookeeper-6bf55cfd99
- zookeeper/zookeeper-856646d9f6
- zookeeper/zookeeper-8cdd5f46
- zookeeper/zookeeper-ccf87988c
apps/v1/StatefulSet:
- zookeeper/zookeeper
discovery.k8s.io/v1/EndpointSlice:
- zookeeper/zookeeper-headless-2tnx5
- zookeeper/zookeeper-mzdlc
monitoring.coreos.com/v1/ServiceMonitor:
- zookeeper/zookeeper-exporter
policy/v1/PodDisruptionBudget:
- zookeeper/zookeeper
v1/ConfigMap:
- zookeeper/kube-root-ca.crt
- zookeeper/zookeeper
v1/Endpoints:
- zookeeper/zookeeper
- zookeeper/zookeeper-headless
v1/Namespace:
- zookeeper
v1/PersistentVolume:
- pvc-261b9803-8e55-4880-bb31-b29ca3a6c323
- pvc-89cfd5b9-65da-4fd1-a095-83d21d1d21db
- pvc-9e027e4c-cc9e-11ea-9ce3-061b42a2865e
- pvc-a835d78d-9dfd-41f7-92bd-7f2e752dbeb7
- pvc-c0e454f7-cc9e-11ea-9ce3-061b42a2865e
- pvc-ee6aad46-cc9e-11ea-9ce3-061b42a2865e
v1/PersistentVolumeClaim:
- zookeeper/data-zookeeper-0
- zookeeper/data-zookeeper-1
- zookeeper/data-zookeeper-2
- zookeeper/data-zookeeper-3
- zookeeper/data-zookeeper-4
- zookeeper/data-zookeeper-5
v1/Pod:
- zookeeper/zookeeper-0
- zookeeper/zookeeper-1
- zookeeper/zookeeper-2
- zookeeper/zookeeper-3
- zookeeper/zookeeper-4
- zookeeper/zookeeper-5
v1/Secret:
- zookeeper/default-token-kcl4m
- zookeeper/sh.helm.release.v1.zookeeper.v1
- zookeeper/sh.helm.release.v1.zookeeper.v10
- zookeeper/sh.helm.release.v1.zookeeper.v11
- zookeeper/sh.helm.release.v1.zookeeper.v12
- zookeeper/sh.helm.release.v1.zookeeper.v13
- zookeeper/sh.helm.release.v1.zookeeper.v4
- zookeeper/sh.helm.release.v1.zookeeper.v5
- zookeeper/sh.helm.release.v1.zookeeper.v6
- zookeeper/sh.helm.release.v1.zookeeper.v7
- zookeeper/sh.helm.release.v1.zookeeper.v8
- zookeeper/sh.helm.release.v1.zookeeper.v9
v1/Service:
- zookeeper/zookeeper
- zookeeper/zookeeper-headless
v1/ServiceAccount:
- zookeeper/default
Velero-Native Snapshots:
pvc-9e027e4c-cc9e-11ea-9ce3-061b42a2865e:
Snapshot ID: snap-0f81f2f62e476584a
Type: gp2
Availability Zone: us-west-2b
IOPS: <N/A>
pvc-c0e454f7-cc9e-11ea-9ce3-061b42a2865e:
Snapshot ID: snap-0c689771f3dbfa361
Type: gp2
Availability Zone: us-west-2a
IOPS: <N/A>
pvc-ee6aad46-cc9e-11ea-9ce3-061b42a2865e:
Snapshot ID: snap-068c63f1bb31af3cc
Type: gp2
Availability Zone: us-west-2b
IOPS: <N/A>
pvc-89cfd5b9-65da-4fd1-a095-83d21d1d21db:
Snapshot ID: snap-050e2e51eac92bd74
Type: gp2
Availability Zone: us-west-2a
IOPS: <N/A>
pvc-261b9803-8e55-4880-bb31-b29ca3a6c323:
Snapshot ID: snap-08e45396c99e7aac3
Type: gp2
Availability Zone: us-west-2b
IOPS: <N/A>
pvc-a835d78d-9dfd-41f7-92bd-7f2e752dbeb7:
Snapshot ID: snap-07ad93657b0bdc1a6
Type: gp2
Availability Zone: us-west-2a
IOPS: <N/A>
但是当试图恢复它失败时
velero restore create --from-backup 动物园管理员
velero restore describe zookeeper9-20220331145320
Name: zookeeper9-20220331145320
Namespace: velero
Labels: <none>
Annotations: <none>
Phase: PartiallyFailed (run 'velero restore logs zookeeper9-20220331145320' for more information)
Total items to be restored: 52
Items restored: 52
Started: 2022-03-31 14:53:24 +0530 IST
Completed: 2022-03-31 14:53:36 +0530 IST
Warnings:
Velero: <none>
Cluster: <none>
Namespaces:
zookeeper: could not restore, ConfigMap "kube-root-ca.crt" already exists. Warning: the in-cluster version is different than the backed-up version.
Errors:
Velero: <none>
Cluster: error executing PVAction for persistentvolumes/pvc-261b9803-8e55-4880-bb31-b29ca3a6c323: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2b' does not exist.
status code: 400, request id: 2b5ae55c-dfd5-4c52-8494-105e46bce78b
error executing PVAction for persistentvolumes/pvc-89cfd5b9-65da-4fd1-a095-83d21d1d21db: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2a' does not exist.
status code: 400, request id: ed91b698-d3b9-450f-b7b4-a3869cbae6ae
error executing PVAction for persistentvolumes/pvc-9e027e4c-cc9e-11ea-9ce3-061b42a2865e: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2b' does not exist.
status code: 400, request id: 2b493106-84c6-4210-9663-4d00f47c06de
error executing PVAction for persistentvolumes/pvc-a835d78d-9dfd-41f7-92bd-7f2e752dbeb7: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2a' does not exist.
status code: 400, request id: 387c6c27-6b18-4bc6-9bb8-3ed152cb49d1
error executing PVAction for persistentvolumes/pvc-c0e454f7-cc9e-11ea-9ce3-061b42a2865e: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2a' does not exist.
status code: 400, request id: 7d7d2931-e7d9-4bc5-8cb1-20e3b2849fe2
error executing PVAction for persistentvolumes/pvc-ee6aad46-cc9e-11ea-9ce3-061b42a2865e: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2b' does not exist.
status code: 400, request id: 75648031-97ca-4e2a-a079-8f6618902b2a
Namespaces: <none>
Backup: zookeeper9
Namespaces:
Included: all namespaces found in the backup
Excluded: <none>
Resources:
Included: *
Excluded: nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io
Cluster-scoped: auto
Namespace mappings: <none>
Label selector: <none>
Restore PVs: auto
Preserve Service NodePorts: auto
它抱怨
Cluster: error executing PVAction for persistentvolumes/pvc-261b9803-8e55-4880-bb31-b29ca3a6c323: rpc error: code = Unknown desc = InvalidZone.NotFound: The zone 'us-west-2b' does not exist.
状态码：400，请求ID：2b5ae55c-dfd5-4c52-8494-105e46bce78b
不知道为什么会这样，有什么我错过的吗。
这让我觉得是否还需要对快照采取一些措施，因为备份具有源区域中的快照 ID，而目标区域不可用
【问题讨论】：
标签： amazon-web-services kubernetes amazon-eks velero