1 unstable release
0.1.2 | Oct 5, 2024 |
---|
#439 in Configuration
28KB
495 lines
YABE (YAml Base Extractor) - Multi-layer YAML organizer
The idea comes from the need to manage huge amount of YAML files in a GitOps environment. Especially when using ArgoCD multi-source apps with some common values and some overrides. The tool helps to compute the common base configuration among multiple YAML files and generate differences for each file, reducing the duplication of configuration values. It also provides the ability to sort YAML content based on user-defined configuration.
Features
- Compute diffs: Detect differences between YAML files.
- Merge YAML files: Combine YAML files with a base YAML, either from an existing file or dynamically computed.
- Quorum-based diffing: Extract common base YAML based on a quorum percentage.
- Sort YAML content: Sort keys in YAML files based on user-defined configuration.
- Helm Values Integration: Merge input YAML files with Helm values files.
- In-place modification or output to new files.
Usage
Usage: yabe [OPTIONS] <INPUT_FILES>...
Arguments:
<INPUT_FILES>... Input YAML files
Options:
-r, --read-base <READ_BASE> (Optional) Read-only base for values deduplication
-b, --base <WRITE_BASE> (Optional) Common values of all input files, if not provided, will be computed
-i, --in-place Modify the original input files with diffs
-o, --out <OUT_FOLDER> Output folder for diff files [default: ./out]
--debug Enable debug logging
-q, --quorum <QUORUM> Quorum percentage (0-100) [default: 51]
--base-out-path <BASE_OUT_PATH> (Optional) Base file output path [default: ./base.yaml]
--sort-config-path <SORT_CONFIG_PATH> (Optional) Sort configuration file path [default: ./sort-config.yaml], if not provided, will not sort
-h, --help Print help
-V, --version Print version
Basic Usage
Run the tool with the YAML override files:
./yabe file1.yaml file2.yaml file3.yaml
This will compute the differences among the override files and generate:
- base.yaml: The common base configuration.
- file1_diff.yaml, file2_diff.yaml, file3_diff.yaml: The differences for each file.
Inplace Modification
Use the -i or --inplace flag to modify the original override files with their differences:
./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml
Enable Debug Logging
Use the --debug flag to enable detailed debug logging:
./yabe --debug -r helm_values.yaml file1.yaml file2.yaml file3.yaml
Examples
Sample Input Files
helm_values.yaml
settings:
theme: dark
notifications: true
advanced:
mode: auto
level: 5
file1.yaml
settings:
theme: dark
notifications: true
advanced:
mode: auto
level: 5
file2.yaml
settings:
theme: light
notifications: true
advanced:
mode: manual
level: 5
file3.yaml
settings:
theme: dark
notifications: false
advanced:
mode: auto
level: 7
Running the Tool
./yabe -r helm_values.yaml file1.yaml file2.yaml file3.yaml
Expected Output
base.yaml
settings:
advanced:
level: 5
file1_diff.yaml (Empty file or not generated since there are no differences)
file2_diff.yaml
settings:
theme: light
advanced:
mode: manual
file3_diff.yaml
settings:
notifications: false
advanced:
level: 7
Inplace Modification Example
Running with the -i flag:
./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml
Testing
The project includes a suite of tests to verify functionality. To run the tests:
cargo test
Ensure all tests pass to verify that the tool is functioning correctly.
Project Structure
- src/
- lib.rs: The library module containing core functionality.
- main.rs: The main executable entry point.
- diff.rs: Functions for computing diffs and common bases.
- deep_equal.rs: Utility function for deep comparison of YAML values.
- sorter.rs: Functions for sorting YAML content.
- tests/
- test_deep_equal.rs: Tests for the deep_equal function.
- test_diff.rs: Tests for compute_diff and diff_and_common_multiple functions.
- test_common.rs: Common tests for the project.
- test_sorter.rs: Tests for the sorter functions.
- Cargo.toml: Project configuration file.
- sort-config.yaml: Configuration file for sorting YAML content.
Dependencies
~4MB
~70K SLoC