5 unstable releases
0.3.0 | May 4, 2023 |
---|---|
0.2.1 | May 3, 2023 |
0.2.0 | May 3, 2023 |
0.1.1 | May 3, 2023 |
0.1.0 | May 3, 2023 |
#1153 in Database interfaces
43KB
874 lines
c8-backup
A CLI tool to automate backup and restore of Camunda Platform 8 deployments.
Current Status
⚠️ Experimental ⚠️
Features:
- List backups
- Create backups
- Restore backups
- Dry runs
Components:
- Zeebe
- Operate
- Tasklist
- Optimize
Deployments:
- Remote Camunda Platform 8 Helm installation (running locally, connecting through the current kubernetes context).
- Local Camunda Platform 8 Helm installation (running as a pod inside the cluster).
Usage
Warning This tool is experimental and not safe for use yet! Running it can result in complete dataloss.
Download a pre-built binary from the latest release or build from sources with
cargo install c8-backup
Currently, this tool is meant to run locally. It will connect to your current kubernetes context and tries to find C8 components there.
Listing backups
The list
command lists recent backups by status and also shows which is the most recent, usable, backup.
This means that the backup is successfully completed by all components.
$ c8-backup list
c8_backup::list::list{}
c8_backup::list::Zeebe{}
0ms INFO c8_backup::list 2 backups Completed: 1683214620, 1683214072, ...
c8_backup::list::Operate{}
0ms INFO c8_backup::list 2 backups Completed: 1683214620, 1683214072, ...
1060ms INFO c8_backup::list The most recent usable backup is 1683214620
1060ms INFO c8_backup::list This backup was created 8 minutes ago at 2023-05-04 15:37:00 UTC
Creating backups
The create
command starts without any additional confirmation and will take the following steps:
- Take a backup of Operate
- Pause Zeebe exporting
- Take a backup of exported Zeebe records
- Take a backup of Zeebe
- Resume Zeebe exporting
Resuming exporting is crucial and this command tries to resume exporting if any error occurs while taking a backup but you should manually confirm that exporting resumed, for example by following the log output.
$ c8-backup create
c8_backup::create::create{}
c8_backup::create::try_backup{backup_id=1683214620}
c8_backup::create::backup_operate{backup_id=1683214620}
c8_backup::operate::take_backup{backup_id=1683214620}
267ms INFO c8_backup::create Started backup
521ms INFO c8_backup::create Checking again in 5 seconds, state is InProgress
5870ms INFO c8_backup::create Checking again in 5 seconds, state is InProgress
11233ms INFO c8_backup::create Checking again in 5 seconds, state is Incomplete
16598ms INFO c8_backup::create Backup completed
c8_backup::zeebe::pause_exporting{}
c8_backup::create::backup_zeebe_export{backup_id=1683214620}
c8_backup::elasticsearch::take_snapshot{req=SnapshotRequest { indices: "zeebe-record*", feature_states: ["none"] }, name="camunda_zeebe_records_1683214620"}
c8_backup::create::backup_zeebe{backup_id=1683214620}
c8_backup::zeebe::take_backup{backup_id=1683214620}
280ms INFO c8_backup::create Started backup
831ms INFO c8_backup::create Checking again in 5 seconds, state is InProgress
6180ms INFO c8_backup::create Backup completed
c8_backup::zeebe::resume_exporting{}```
### Restoring backups
The `restore` commands starts without any additional confirmation and will take the following steps:
1. Find the latest backup from Zeebe and Operate
2. Stop Zeebe and Operate
3. Delete **all** indices from Elasticsearch
4. Restore Elasticsearch indices based on backups
5. Delete all Zeebe data
6. Restore Zeebe data based on backup
7. Start Zeebe and Operate
```shell
$ c8-backup restore
c8_backup::restore::restore{}
c8_backup::restore::find_newest_backup{}
1573ms INFO c8_backup::restore Using backup 1683214620
c8_backup::restore::shutdown_apps{}
331ms INFO c8_backup::restore Shut down ccs23-dev-zeebe-gateway
380ms INFO c8_backup::restore Shut down ccs23-dev-operate
431ms INFO c8_backup::restore Shut down ccs23-dev-zeebe
c8_backup::restore::delete_indices{}
c8_backup::elasticsearch::get_all_indices{}
434ms INFO c8_backup::elasticsearch Deleted index operate-flownode-instance-8.2.0_
695ms INFO c8_backup::elasticsearch Deleted index operate-decision-instance-8.2.0_
966ms INFO c8_backup::elasticsearch Deleted index operate-list-view-8.1.0_
1263ms INFO c8_backup::elasticsearch Deleted index operate-import-position-8.2.0_
1587ms INFO c8_backup::elasticsearch Deleted index operate-user-1.2.0_
1907ms INFO c8_backup::elasticsearch Deleted index zeebe-record_deployment-distribution_8.2.3_2023-05-04
2223ms INFO c8_backup::elasticsearch Deleted index operate-operation-8.2.0_
2509ms INFO c8_backup::elasticsearch Deleted index zeebe-record_deployment_8.2.3_2023-05-04
2809ms INFO c8_backup::elasticsearch Deleted index operate-batch-operation-1.0.0_
3077ms INFO c8_backup::elasticsearch Deleted index operate-process-8.1.8_
3352ms INFO c8_backup::elasticsearch Deleted index operate-web-session-1.1.0_
3634ms INFO c8_backup::elasticsearch Deleted index operate-incident-8.2.0_
3938ms INFO c8_backup::elasticsearch Deleted index zeebe-record_variable_8.2.3_2023-05-04
4216ms INFO c8_backup::elasticsearch Deleted index zeebe-record_job_8.2.3_2023-05-04
4472ms INFO c8_backup::elasticsearch Deleted index operate-variable-8.2.0_
4796ms INFO c8_backup::elasticsearch Deleted index operate-migration-steps-repository-1.1.0_
5069ms INFO c8_backup::elasticsearch Deleted index operate-decision-1.0.0_
5324ms INFO c8_backup::elasticsearch Deleted index operate-event-8.1.0_
5624ms INFO c8_backup::elasticsearch Deleted index zeebe-record_process-instance-creation_8.2.3_2023-05-04
5939ms INFO c8_backup::elasticsearch Deleted index zeebe-record_process_8.2.3_2023-05-04
6234ms INFO c8_backup::elasticsearch Deleted index operate-metric-1.0.0_
6526ms INFO c8_backup::elasticsearch Deleted index operate-sequence-flow-8.2.0_
6844ms INFO c8_backup::elasticsearch Deleted index zeebe-record_process-instance_8.2.3_2023-05-04
7164ms INFO c8_backup::elasticsearch Deleted index operate-decision-requirements-1.0.0_
c8_backup::restore::restore_indices{backup=Backup { id: 1683214620, snapshots: ["camunda_zeebe_records_1683214620", "camunda_operate_1683214620_8.2.3_part_1_of_6", "camunda_operate_1683214620_8.2.3_part_2_of_6", "camunda_operate_1683214620_8.2.3_part_3_of_6", "camunda_operate_1683214620_8.2.3_part_4_of_6", "camunda_operate_1683214620_8.2.3_part_5_of_6", "camunda_operate_1683214620_8.2.3_part_6_of_6"] }}
12203ms INFO c8_backup::restore Restored snapshot camunda_zeebe_records_1683214620
14050ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_1_of_6
18322ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_2_of_6
18925ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_3_of_6
23336ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_4_of_6
23991ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_5_of_6
26099ms INFO c8_backup::restore Restored snapshot camunda_operate_1683214620_8.2.3_part_6_of_6
c8_backup::restore::delete_zeebe_data{backup=Backup { id: 1683214620, snapshots: ["camunda_zeebe_records_1683214620", "camunda_operate_1683214620_8.2.3_part_1_of_6", "camunda_operate_1683214620_8.2.3_part_2_of_6", "camunda_operate_1683214620_8.2.3_part_3_of_6", "camunda_operate_1683214620_8.2.3_part_4_of_6", "camunda_operate_1683214620_8.2.3_part_5_of_6", "camunda_operate_1683214620_8.2.3_part_6_of_6"] }}
303ms INFO c8_backup::restore Deleting data of data-ccs23-dev-zeebe-0
332ms INFO c8_backup::restore Deleting data of data-ccs23-dev-zeebe-1
360ms INFO c8_backup::restore Deleting data of data-ccs23-dev-zeebe-2
17284ms INFO c8_backup::restore Deleted data of data-ccs23-dev-zeebe-0
17385ms INFO c8_backup::restore Deleted data of data-ccs23-dev-zeebe-1
17482ms INFO c8_backup::restore Deleted data of data-ccs23-dev-zeebe-2
c8_backup::restore::restore_zeebe_data{backup=Backup { id: 1683214620, snapshots: ["camunda_zeebe_records_1683214620", "camunda_operate_1683214620_8.2.3_part_1_of_6", "camunda_operate_1683214620_8.2.3_part_2_of_6", "camunda_operate_1683214620_8.2.3_part_3_of_6", "camunda_operate_1683214620_8.2.3_part_4_of_6", "camunda_operate_1683214620_8.2.3_part_5_of_6", "camunda_operate_1683214620_8.2.3_part_6_of_6"] }}
97ms INFO c8_backup::restore Restoring data of data-ccs23-dev-zeebe-0
131ms INFO c8_backup::restore Restoring data of data-ccs23-dev-zeebe-1
161ms INFO c8_backup::restore Restoring data of data-ccs23-dev-zeebe-2
26025ms INFO c8_backup::restore Restored data of data-ccs23-dev-zeebe-0
26117ms INFO c8_backup::restore Restored data of data-ccs23-dev-zeebe-1
26210ms INFO c8_backup::restore Restored data of data-ccs23-dev-zeebe-2
c8_backup::restore::start_apps{}
0ms INFO c8_backup::restore Starting apps
29ms INFO c8_backup::restore Started ccs23-dev-zeebe-gateway
59ms INFO c8_backup::restore Started ccs23-dev-operate
90ms INFO c8_backup::restore Started ccs23-dev-zeebe```
Dependencies
~74MB
~1M SLoC