Add CLI flags for every config option and add config options for many action flags (#303). #1040

witten · 2025-03-22T20:37:10Z

witten commented

2025-03-22 20:37:10 +00:00

Implements #303. This PR:

Adds CLI flags for (nearly) every existing configuration option. The few exceptions are certain options that are intentionally omitted because there are already corresponding flags on individual actions (especially after the following point).
Adds configuration options for certain existing action flags (create --progress, repo-create --encryption, etc.).

The intent is to: 1. Replace --override with a CLI flag syntax that makes overriding settings more natural and granular (now you can target individual list items), and 2. Make more things actually configurable so the user doesn't need to put as much on the command-line and therefore wrap borgmatic with a wrapper script.

Still to do

do an organizational pass on the new code
get unit/integration tests passing
get end-to-end tests passing
add/update unit/integration tests for changes in this PR
add/update end-to-end tests for changes in this PR
audit for missing test coverage
documentation
fix broken --list, --progress, etc. overrides.
manual testing

Not part of this, but obvious follow-on work: #793.

Implements #303. This PR: 1. Adds CLI flags for (nearly) every existing configuration option. The few exceptions are certain options that are intentionally omitted because there are already corresponding flags on individual actions (especially after the following point). 2. Adds configuration options for certain existing action flags (`create --progress`, `repo-create --encryption`, etc.). The intent is to: 1. Replace `--override` with a CLI flag syntax that makes overriding settings more natural and granular (now you can target individual list items), and 2. Make more things actually configurable so the user doesn't need to put as much on the command-line and therefore wrap borgmatic with a wrapper script. ### Still to do * [x] do an organizational pass on the new code * [x] get unit/integration tests passing * [x] get end-to-end tests passing * [x] add/update unit/integration tests for changes in this PR * [x] add/update end-to-end tests for changes in this PR * [x] audit for missing test coverage * [x] documentation * [x] fix broken `--list`, `--progress`, etc. overrides. * [x] manual testing Not part of this, but obvious follow-on work: #793.

witten added 24 commits 2025-03-22 20:37:11 +00:00

Initial work on command-line flags for all configuration (#303 ). 92279d3c71

Merge branch 'main' into config-command-line. e1ff51ff1e

Expand docstrings (#303 ). 05900c188f

Support setting whole lists and dicts from the command-line (#303 ). eca78fbc2c

Code formatting (#303 ). 87b9ad5aea

Add feature to NEWS (#303 ). 7b14e8c7f2

Merge branch 'main' into config-command-line d75c8609c5

Factor out schema type parsing (#303 ). 903308864c

Add an encryption option to repositories (#303 ). 93e7da823c

UX nicety to make default-false boolean options into valueless CLI flags (#303 ). 711f5fa6cb

Add additional options under "repositories:" for parity with repo-create #303 . 63b0c69794

Add "progress" option to configuration (#303 ). 1097a6576f

Use schema defaults instead of a flag name whitelist to make valueless boolean flags (#303 ). d02d31f445

Add "stats" option to configuration (#303 ). 3e21cdb579

Add "list" option to configuration, corresponding to "--list" (#303 ). ed6022d4a9

In configuration option descriptions, remove mention of corresponding CLI flags because it looks dumb on the command-line help (#303 ). 3119c924b4

Merge branch 'main' into config-command-line. 89c98de122

Don't auto-add CLI flags for configuration options that already have per-action CLI flags (#303 ). 7d989f727d

Exclude --match-archives from global flags since it already exists on several actions (#303 ). cbfc0bead1

Merge branch 'main' into config-command-line f3cc3b1b65

Merge branch 'main' into config-command-line 5c11052b8c

Merge branch 'main' of ssh://projects.torsion.org:3022/borgmatic-collective/borgmatic into config-command-line 6f1c77bc7d

Merge branch 'main' into config-command-line. 5246a10b99

Add "compact_threshold" option, overridden by "compact --threshold" flag (#303 ). 976fb8f343

witten changed title from ~~Add CLI flags for every config option and add config options for many action flags (#303).~~ to WIP: Add CLI flags for every config option and add config options for many action flags (#303).

2025-03-22 20:37:16 +00:00

witten added 2 commits 2025-03-22 21:01:56 +00:00

Fix list examples in generated configuration. cc14421460

Code formatting (#303 ). f8eda92379

witten added 1 commit 2025-03-22 21:04:03 +00:00

Bold "not yet released" in docs to prevent confusion (#303 ). dc9da3832d

witten added 1 commit 2025-03-23 05:52:40 +00:00

Organizational refactoring (#303 ). f222bf2c1a

witten added 1 commit 2025-03-23 18:25:13 +00:00

Factor out schema type comparion in config generation and get several tests passing (#303 ). 57721937a3

witten added 1 commit 2025-03-23 18:27:06 +00:00

Fix missing argument descriptions (#303 ). ba75958a2f

witten added 2 commits 2025-03-24 00:00:18 +00:00

Add Bash completion for completing flags like "--foo[3].bar". 9f7c71265e

Get existing unit/integration tests passing (#303 ). 423627e67b

witten added 1 commit 2025-03-24 01:46:09 +00:00

Get existing end-to-end tests passing (#303 ). 16a1121649

witten added 2 commits 2025-03-26 05:40:03 +00:00

Clarify NEWS items (#303 ). 65b1d8e8b2

Merge branch 'main' into config-command-line. 5a0430b9c8

witten added 2 commits 2025-03-27 18:08:04 +00:00

Merge branch 'main' into config-command-line. 4c6674e0ad

Add tests for adding array element arguments and fix the code under test (#303 ). 50beb334dc

witten added 1 commit 2025-03-27 19:43:27 +00:00

Set the action type when cloning an argument for a list index flag (#303 ). 79bf641668

witten added 1 commit 2025-03-27 23:49:33 +00:00

Add tests for CLI arguments from schema logic (#303 ). b4c558d013

witten added 1 commit 2025-03-28 04:02:13 +00:00

More automated tests (#303 ). 5bf2f546b9

witten added 1 commit 2025-03-28 18:38:09 +00:00

Add additional tests for complete coverage (#303 ). 975a6e4540

witten added 1 commit 2025-03-28 20:47:53 +00:00

Add an end-to-end test for command-line flags of configuration options (#303 ). 1c27e0dadc

witten added 1 commit 2025-03-29 06:11:47 +00:00

Adding some missing tests and fixing related flag vs. config logic (#303 ). 668f767bfc

witten added 1 commit 2025-03-29 21:27:10 +00:00

Add last couple of missing tests after audit (#303 ). f6929f8891

witten added 2 commits 2025-03-29 22:25:03 +00:00

Merge branch 'main' into config-command-line. c7feb16ab5

Add "list_details" config option support to new "recreate" action (#303 ). 8101e5c56f

witten added 2 commits 2025-03-30 02:03:38 +00:00

Merge branch 'main' of ssh://projects.torsion.org:3022/borgmatic-collective/borgmatic into config-command-line cffb8e88da

Merge branch 'main' into config-command-line. 3e05eeb4de

witten added 1 commit 2025-03-30 02:55:11 +00:00

Merge branch 'main' into config-command-line. 92de539bf9

witten added 1 commit 2025-03-30 05:00:19 +00:00

Add documentation for "native" command-line overrides without --override (#303 ). fd8c11eb0a

witten added 1 commit 2025-03-30 05:52:52 +00:00

Merge branch 'main' into config-command-line. de4d7af507

witten added 1 commit 2025-03-31 18:34:50 +00:00

Fix broken list_details, progress, and statistics options (#303 ). cf477bdc1c

witten added 1 commit 2025-03-31 20:33:54 +00:00

Fix bootstrap --progress flag (#303 ). 10fb02c40a

witten added 1 commit 2025-03-31 22:27:20 +00:00

Fix new "repositories:" sub-options ("append_only", "make_parent_directories", etc.) (#303 ). f166111b9b

witten added 1 commit 2025-04-01 00:13:32 +00:00

Merge branch 'main' into config-command-line. 5490a83d77

witten added 1 commit 2025-04-01 19:10:02 +00:00

Merge branch 'main' into config-command-line. af97b95e2b

witten added 2 commits 2025-04-02 02:45:34 +00:00

Fix "progress" option with the "transfer" action (#303 ). e96db2e100

Fix for the example not showing up in generated config for empty YAML objects (#303 ). 017cbae4f9

witten added 1 commit 2025-04-02 04:06:35 +00:00

Expose propertyless YAML objects from configuration (e.g. "constants") as command-line flags (#303 ). affe7cdc1b

witten added 1 commit 2025-04-02 06:06:01 +00:00

Fix use of dashed command-line flags like "--repositories[2].append-only" generated from configuration (#303 ). 4065c5d0f7

witten added 1 commit 2025-04-02 16:38:28 +00:00

Fix incorrect matching of non-zero array index flags with dashed names (#303 ). 364200c65a

witten added 1 commit 2025-04-02 16:55:42 +00:00

Merge branch 'main' into config-command-line. 9301ab13cc

witten added 1 commit 2025-04-03 00:08:43 +00:00

For boolean configuration options, add separate "--foo" and "--no-foo" CLI flags (#303 ). bbf6f27715

witten added 1 commit 2025-04-03 06:16:18 +00:00

Make a CLI flag for any config option that's a list of scalars (#303 ). d2c3ed26a9

witten added 1 commit 2025-04-03 16:25:19 +00:00

Add a TL;DR to NEWS since 2.0.0 is such a huge release and ain't nobody got time for reading a huge changelog. d0a5aa63be

witten added 1 commit 2025-04-03 18:28:58 +00:00

Fix setting of "--checks" on the command-line (#303 ). 9407f24674

witten added 1 commit 2025-04-03 18:47:38 +00:00

Merge branch 'main' into config-command-line. 3f25f3f0ff

witten added 1 commit 2025-04-03 23:22:12 +00:00

If a boolean option name already starts with "no_", don't add a "--no-no-..." CLI flag (#303 ). 3eabda45f2

witten added 1 commit 2025-04-03 23:38:54 +00:00

Add a documentation note about a limitation: You can't pass flags as values to flags (#303 ). 9ea55d9aa3

witten changed title from ~~WIP: Add CLI flags for every config option and add config options for many action flags (#303).~~ to Add CLI flags for every config option and add config options for many action flags (#303).

2025-04-03 23:48:23 +00:00

witten merged commit 929d343214 into main

2025-04-03 23:48:49 +00:00

witten referenced this issue from a commit

2025-04-03 23:48:51 +00:00

Add CLI flags for every config option and add config options for many action flags (#303).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#1040