YAML !include with relative path #532
Labels
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Milestone
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: borgmatic-collective/borgmatic#532
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What I'm trying to do and why
I'd like to be able to use the YAML include file and merging features with relative path names, something like this:
It seems that any non-absolute filename is taken as relative to the current working directory, rather than relative to the file in which it sits.
I guess I want this because I'm keeping configs for several machines on other machines, so that it's easy to maintain and restore repos on behalf of other hosts. Essentially I'm doing this by lifting local-machine configs into
borgmatic.d
and leaving the other-machine configs in a subdir.Also to further complicate matters, I'm using NixOS and installing the files into /etc as symlinks to the nix store. I'd like it to work either from the installed default location (
/etc/borgmatic.d
) or from a draft "dotfiles" location (borgmatic -c ~/dotfiles/sys/borgmatic/HOST.yaml
).Steps to reproduce (if a bug)
where
host1.yaml
is like:and
no-checks.yaml
is:Actual behavior (if a bug)
Expected behavior (if a bug)
Which is what happens now when I change the include to a path relative to CWD:
Other notes / implementation ideas
I can definitely see this is a niche use case. But if it's relatively easy to say what the CWD should be when loading
./something.yaml
that would be sweet. All the examples I'm finding in documentation or the issues forum use absolute paths.(If this becomes a WONTFIX, a line in the docs saying "must use absolute paths" would be appropriate.)
Thankss for work on borgmatic... at first I was writing, then generating my own hacky borg scripts, but this will end up being cleaner and more reliable!
Environment
borgmatic version: 1.5.18
borgmatic installation method: NixOS, nixpkgs
Borg version: 1.1.17
Python version: 3.9.12
Database version (if applicable): [version here]
operating system and version: NixOS 21.11, MacOS
First off, I appreciate the detailed ticket write-up. I think a feature like this could work in borgmatic. However, if borgmatic simply changed includes from being relative-from-working-directory to relative-from-config, that would be a breaking change for anyone relying on relative includes today. There have been breaking changes in the past (see version 1.6.0 for an example). But one way to do this without breaking anything would change the behavior to probe: So when doing a relative include, first probe for relative-from-working-directory. If not found, try relative-from-config next. Or I suppose the other way around could work too.
Please let me know your thoughts!
I think a probe and fallback should be fine. Or even a list of directories conveyed in a PATH-style environment variable would be usable. Might need to be a little cautious when it comes to includes-within-includes, or symlinks.
I wouldn't want to overengineer it... after all we're just trying to unify a few fragments of config to keep settings consistent. In other words, don't go nuts to highlight some obscure corners... it's not all that much harder to use some preprocessor to generate configs.
If one wants to argue for one behavior over the other, with "principle of least surprise", I'd say most of those preprocessors (cpp, m4, makefile include) are relative to the file in which the include is found. It does get confusing when the same file might be reached through different symlinks. I don't know offhand if those tools ‘realpath’ it, or what. Anyway, thanks for considering.
This is implemented in master now!
It does a probe and fallback of: 1. the current working directory, and 2. the directory containing the file doing the including. Not otherwise configurable, for now. It does do the
realpath
equivalent on the second path.Feel free to give it a shot or wait for the next release. And thanks for suggesting this!
Released in borgmatic 1.6.1!