Set borgmatic command-line flag defaults in config file #303
Loading…
x
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What I'm trying to do and why
Specify
--encryption
type inconfig.yaml
.Steps to reproduce (if a bug)
Actual behavior (if a bug)
Expected behavior (if a bug)
The borg init command would be passed the
--encryption
option fromextra_borg_options
.Other notes / implementation ideas
It would be really helpful if all borgmatic command-line options (excluding
--config
obviously) could be specified viaconfig.yaml
.As
borgmatic
with no arguments is meant to 'just work' this would allow otherborg
commands to similarly 'just work' and I could capture my output preferences as well. As it is, I need to use a script to wrap borgmatic.To go with this, it would also be useful if
--config
could be read from the environment (e.g.${BORGMATIC_CONFIG}
).Environment
borgmatic version: 1.5.2
borgmatic installation method: pip
Borg version: 1.1.9
Python version: 3.7.3
operating system and version: Debian GNU/Linux 10 (buster)
extra_borg_args not parsed early enough to permit --encryption type for init command to be provided in configto extra_borg_args not parsed so encryption type cannot be provided in configSorry, just remembered that borgmatic doesn't look at
extra_borg_options
at all. However the problem is roughly the same. I.e. Borgmatic configuration file can't handle all the configuration options needed by Borg.Neat idea! Could you say some more about your use case? Is it that you're running
borgmatic init
manually on occasion, and you don't want to have to type/remember all the necessary command-line arguments every time? Or are you running it in an automated fashion, and it's a pain to pass in the arguments?And is it the encryption mode flag in particular? Or are there others that you find yourself wanting to have defaults?
Also, for the
BORGMATIC_CONFIG
environment variable idea, what's your use case? Are you invoking borgmatic from somewhere (e.g. a Docker container) where passing in environment variables is more convenient than command-line flags? Thanks.Yes, that's what led me to this. More generally it's to achieve DRYness which would help with configuration management. With the
ruamel.yaml
extension to include other files I'm aiming for a 'single generic file with all my settings'. Then some simple deployment for the machine-specific settings (i.e. hooks and repos). For me, things like verbosity and encryption type are not machine-specific therefore should live in the config file.I feel it should be all options. What's irritating is that argparse for Borgmatic is erroring (because I guess that option has been set to required) but certainly in the case of encryption, I would expect it to just pass the arguments through to Borg (adding in extra arguments like the repo of course, but otherwise just combining). The way it behaves at the moment, it could work but argparse's configuration stops it (I am guessing).
Yep. Pretty much. But generally, being able to alter the configuration prior to invocation is the motivation. Could also be during development/testing from an IDE configuration.
Fair enough. I think I'll re-title the ticket in terms of this ask.
Could be most. But there are probably flags like
--repository
or--archive
that only make sense at the command-line.I think I'm missing something here. Borg requires the
--encryption
flag as well. So what would it accomplish for borgmatic not to require it?And to be clear, you're thinking that
BORGMATIC_CONFIG
would be a path to config file? Or the actual config YAML contents?extra_borg_args not parsed so encryption type cannot be provided in configto Set borgmatic command-line flag defaults in config fileThanks. That's a better title.
Absolutely. I should have worded that better. I suppose what I meant was "all backup related options". As you point out, restore related options generally don't make sense in a configuration file. Although maybe there is a use-case for extracting a random sample and comparing the contents but that might be better served using
consistency.checks: extract
or could probably be done with hooks -- either way its certainly beyond the scope of this idea.It does, but if the
argparse
configuration was more permissive, I might have been able to pass it through by instorage.extra_borg_options.init
. Not important though if (backup related) options can be included in config.Just the path to the config file.
That's exactly my reason as well: I like the idea of borgmatic providing a config combining most/all aspects of the backup, interacting with the repo etc. very much. That includes how to create the repo in the first step, even if it would never be required latter again. Though, it would be "forever" documented in the config file, which is part of some GIT or SVN repo very likely, that e.g. encryption mode
keyfile
was used and at the same time make it easier to create a repo without needing to provide command line arguments manually.Besides that,
init
might not be a one-shot task especially for new users as well. When I started with borg and borgmatic and tested things, I recreated some repos for various different reasons. Of course there's the shell history most likely making this easy as well, but that history is not forever and command simply become shorter when aspects like encryption mode are in the config file already.I'm scoping this ticket down a bit for a couple of reasons: 1. I think it'll be more likely to get more traction that way, and 2. there are practical limitations to what borgmatic command-line flags can be in the configuration file. For instance, it'd be tough to put logging and verbosity flags in configuration, because logging and verbosity necessarily take effect before configuration is even loaded. (Although see #793.) And as discussed above, there are some flags like
--repository
or--repair
that would typically only ever be used on the command-line.So here's my proposal on existing command-line flags to add configuration options for. The idea is that if such an option is set in the configuration file, it would take effect unless overridden on the command-line.
rcreate
(a.k.a.init
)--encryption
--append-only
--storage-quota
--make-parent-dirs
transfer
--progress
create
--progress
--stats
--list
prune
--stats
--list
compact
--progress
--threshold
(could also apply to when borgmatic runs compact automatically)check
--progress
extract
--progress
export-tar
--list
rlist
andlist
--short
--format
This is just a proposal. Feel free to add/subtract supported flags as appropriate. For instance, an argument could be made for supporting
--prefix
or--match-archives
. Same or--first
and--last
. But it might make sense to start small and expand the supported flags from there.Where to put the new options in the configuration file is left as exercise to the implementer. And in some cases where the same flag exists across multiple actions, there could possibly be a single option that controls them.
I'm new to borgmatic and was confused about this, too.
One of the main reasons why I found borgmatic so attractive is that I can write things in a config file and get rid of having to create and maintain my own wrapper scripts where I call Borg with lines and lines of options and flags. A command like
borgmatic create --verbosity 1 --list --progress --stats
is kind of the opposite of the elegance that I aim to achieve, and almost forces me to write a wrapper script again, but this time around borgmatic.So, in addition to the per-command arguments listed above, I'd like to suggest global arguments, too:
--verbosity
(-v
), together with--syslog-verbosity
,--log-file-verbosity
and--monitoring-verbosity
.YAML could actually provide a way to set all of these at once, or at least in a less verbose fashion:
--log-file
Thanks for weighing in. Part of the reason verbosity is only on the command-line now is that verbosity settings apply before configuration is even loaded! However see #793 for some ideas on how both logging and verbosity setup can potentially be stored in a borgmatic configuration file despite that current limitation.
Especially when running as a systemd service, that would be very useful. Because it is hard to customize what comes with the distro already.
Are there particular command-line options that would be useful to have in configuration for you?
Specifically the logging options like
--verbosity
. Because on Fedora systemd would record the output, no need for syslog.Makes sense, thanks!