remote_path should (also) be per-repository

gaaf commented

2025-11-20 10:22:02 +00:00

What I'd like to do and why

Hi,

I have multiple repositories configured, but one of them needs a remote_path setting. The other repository breaks with the remote_path for the first, eliminating the possibility to use the global setting.

Isn't remote_path very remote-specific? I really wonder why it is (only) a global option.

I know about the possibility to use multiple config files, but I'd rather avoid that as it contains lots of options that are not remote-specific.

Other notes / implementation ideas

No response

### What I'd like to do and why Hi, I have multiple repositories configured, but one of them needs a `remote_path` setting. The other repository breaks with the `remote_path` for the first, eliminating the possibility to use the global setting. Isn't `remote_path` very remote-specific? I really wonder why it is (only) a global option. I know about the possibility to use multiple config files, but I'd rather avoid that as it contains lots of options that are not remote-specific. ### Other notes / implementation ideas _No response_

👍 1

witten commented

2025-11-20 18:38:40 +00:00

Owner

The approach that borgmatic takes to repository-specific options is indeed separate configuration files. The way I'd recommend doing that to avoid duplication is via includes. So you'd put all your shared configuration options in a common configuration file. And then you can make one configuration file for each repository, performing an include to pull in the common options so they're not repeated—and then overriding any options (like remote_path) that vary per repository.

Hope that helps.

The approach that borgmatic takes to repository-specific options is indeed separate configuration files. The way I'd recommend doing that to avoid duplication is [via includes](https://torsion.org/borgmatic/reference/configuration/includes/). So you'd put all your shared configuration options in a common configuration file. And then you can make one configuration file for each repository, performing an include to pull in the common options so they're not repeated—and then [overriding](https://torsion.org/borgmatic/reference/configuration/includes/#include-merging) any options (like `remote_path`) that vary per repository. Hope that helps.

witten added the

question / support

label

2025-11-20 18:38:49 +00:00

witten commented

2025-11-20 18:42:52 +00:00

Owner

And to answer your question a little more directly: The reason that remote_path isn't an option directly under repositories: is that one could argue almost any borgmatic option is (or can be) repository-specific, in which case you'd end up with pretty much the entire configuration file repeated under repositories:. That just seems pretty unwieldy to me from a schema perspective.

And to answer your question a little more directly: The reason that `remote_path` isn't an option directly under `repositories:` is that one could argue almost *any* borgmatic option is (or can be) repository-specific, in which case you'd end up with pretty much the entire configuration file repeated under `repositories:`. That just seems pretty unwieldy to me from a schema perspective.

gaaf commented

2025-11-24 17:55:02 +00:00

Author

Well, remote_path is something that could be different on every remote and might not be under the control of the local admin. That can not be said about most other options, those are mostly just choices or preferences. For example, one of borg(matic)'s funders (Hetzner) requires remote_path if one does not want to use a very ancient borg version.

About repeating the options under repositories: Most schema definition formats allow references to objects that can be included on multiple locations. The schema/configuration could be setup so that most (all?) options can be overridden under a repository. That way one does not have the hassle of having to configure multiple backups that still perform the necessary pre- and post actions only once and in the correct order and without overlapping.

Well, `remote_path` is something that could be different on every remote and might not be under the control of the local admin. That can not be said about most other options, those are mostly just choices or preferences. For example, one of borg(matic)'s funders (Hetzner) requires `remote_path` if one does not want to use a very ancient borg version. About repeating the options under repositories: Most schema definition formats allow references to objects that can be included on multiple locations. The schema/configuration could be setup so that most (all?) options can be overridden under a repository. That way one does not have the hassle of having to configure multiple backups that still perform the necessary pre- and post actions only once and in the correct order and without overlapping.

witten commented

2025-11-26 23:16:08 +00:00

Owner

Well, remote_path is something that could be different on every remote and might not be under the control of the local admin. That can not be said about most other options, those are mostly just choices or preferences. For example, one of borg(matic)'s funders (Hetzner) requires remote_path if one does not want to use a very ancient borg version.

While I agree that there are many options that represent client-side preferences, there are several other options that I could see being repository-specific, depending on the situation:

encryption_passphrase (e.g. if each repo has a separate passphrase for security purposes)
encryption_passcommand
checkpoint_interval (e.g. if one repo is less reliable than another, depending on where it's hosted)
compression (e.g. depending on differing repository server resources)
upload_rate_limit (same)
upload_buffer_size
retries (for less reliable servers)
retry_wait
lock_wait
etc etc.

About repeating the options under repositories: Most schema definition formats allow references to objects that can be included on multiple locations. The schema/configuration could be setup so that most (all?) options can be overridden under a repository. That way one does not have the hassle of having to configure multiple backups that still perform the necessary pre- and post actions only once and in the correct order and without overlapping.

I'm honestly not wild about this either. Let's say we can use a JSON Schema $ref to avoid schema repetition under repositories. Not all options make sense under there (examples: verbosity, log_file, log_file_format, monitoring_verbosity, etc.). But let's say we ignore those and just support all options under each repository. Then borgmatic would have to effectively merge the "global" options into the repository-specific options, and every corner of borgmatic would have to be updated to pull values out of the repository-specific option rather than global scope. Does that support deep merging of complex values? Are there ways to omit merging of certain values? What about merging of lists?

There are all features that are already implemented by the existing includes mechanism, and I don't see the value in reinventing the wheel when all of this already is available to users.

Now, having said all that—and in the interests of actually addressing the spirit of your ask if not the actual request—here are a couple of related ideas:

Is your main problem with includes that you want to do everything all in one configuration file rather than two or three? If so, maybe borgmatic supporting a multi-document YAML file would address that concern? However I'm not sure how you'd do includes in such a scenario.. Maybe by named references or something.
Perhaps if remote_path is really an outlier like you're making the case for here, it should be moved outright from global scope to repository scope? That might address your issue without opening up the whole can of worms related to every configuration option. However I can see that being a slippery slope for other options listed above like encryption_passphrase, checkpoint_interval, etc. I just don't know where we draw the line. And the current includes-based approach means we don't have to draw the line. Every option can be repository-specific if it's thrown in its own config file.

> Well, `remote_path` is something that could be different on every remote and might not be under the control of the local admin. That can not be said about most other options, those are mostly just choices or preferences. For example, one of borg(matic)'s funders (Hetzner) requires `remote_path` if one does not want to use a very ancient borg version. While I agree that there are many options that represent client-side preferences, there are several other options that I could see being repository-specific, depending on the situation: * `encryption_passphrase` (e.g. if each repo has a separate passphrase for security purposes) * `encryption_passcommand` * `checkpoint_interval` (e.g. if one repo is less reliable than another, depending on where it's hosted) * `compression` (e.g. depending on differing repository server resources) * `upload_rate_limit` (same) * `upload_buffer_size` * `retries` (for less reliable servers) * `retry_wait` * `lock_wait` * etc etc. > About repeating the options under repositories: Most schema definition formats allow references to objects that can be included on multiple locations. The schema/configuration could be setup so that most (all?) options can be overridden under a repository. That way one does not have the hassle of having to configure multiple backups that still perform the necessary pre- and post actions only once and in the correct order and without overlapping. I'm honestly not wild about this either. Let's say we can use a JSON Schema `$ref` to avoid schema repetition under `repositories`. Not all options make sense under there (examples: `verbosity`, `log_file`, `log_file_format`, `monitoring_verbosity`, etc.). But let's say we ignore those and just support all options under each repository. Then borgmatic would have to effectively merge the "global" options into the repository-specific options, and every corner of borgmatic would have to be updated to pull values out of the repository-specific option rather than global scope. Does that support deep merging of complex values? Are there ways to omit merging of certain values? What about merging of lists? There are all features that are already implemented by the existing includes mechanism, and I don't see the value in reinventing the wheel when all of this already is available to users. Now, having said all that—and in the interests of actually addressing the spirit of your ask if not the actual request—here are a couple of related ideas: * Is your main problem with includes that you want to do everything all in one configuration file rather than two or three? If so, maybe borgmatic supporting a [multi-document YAML file](https://www.yaml.info/learn/document.html) would address that concern? However I'm not sure how you'd do includes in such a scenario.. Maybe by named references or something. * Perhaps if `remote_path` is really an outlier like you're making the case for here, it should be *moved* outright from global scope to repository scope? That might address your issue without opening up the whole can of worms related to every configuration option. However I can see that being a slippery slope for other options listed above like `encryption_passphrase`, `checkpoint_interval`, etc. I just don't know where we draw the line. And the current includes-based approach means we don't have to draw the line. Every option can be repository-specific if it's thrown in its own config file.

witten added the

waiting for response

label

2025-12-21 18:26:26 +00:00

dbrgn commented

2025-12-26 18:22:50 +00:00

I ran into the same issue when trying to set up a Hetzner StorageBox.

When using the suggested approach with includes and separate config files, that means that I can't just run "borgmatic" once for all backup targets, but need to call it multiple times (once per repo), right?

I ran into the same issue when trying to set up a Hetzner StorageBox. When using the suggested approach with includes and separate config files, that means that I can't just run "borgmatic" once for all backup targets, but need to call it multiple times (once per repo), right?

witten commented

2025-12-26 19:46:19 +00:00

Owner

When using the suggested approach with includes and separate config files, that means that I can't just run "borgmatic" once for all backup targets, but need to call it multiple times (once per repo), right?

You can indeed just run borgmatic once for all backup targets if you put each config file in /etc/borgmatic.d or in another directory you specify. For more information, see: https://torsion.org/borgmatic/how-to/make-per-application-backups/

> When using the suggested approach with includes and separate config files, that means that I can't just run "borgmatic" once for all backup targets, but need to call it multiple times (once per repo), right? You can indeed just run `borgmatic` once for all backup targets if you put each config file in `/etc/borgmatic.d` or in another directory you specify. For more information, see: https://torsion.org/borgmatic/how-to/make-per-application-backups/

👍 1

gaaf commented

2026-01-26 15:50:43 +00:00

Author

Is your main problem with includes that you want to do everything all in one configuration file rather than two or three?

Preferably, but not necessarily. I just want to run a single instance of borgmatic, so there's no multiple backups running at the same time. And I need the the before_everything hook to run exactly once per backup (once for all repositories, not once per repository).

> Is your main problem with includes that you want to do everything all in one configuration file rather than two or three? Preferably, but not necessarily. I just want to run a single instance of borgmatic, so there's no multiple backups running at the same time. And I need the the `before_everything` hook to run exactly once per backup (once for all repositories, not once per repository).

witten commented

2026-01-26 23:12:22 +00:00

Owner

Multiple configuration files run with a single instance of borgmatic if you either put them in /etc/borgmatic.d or pass them in with --config on the command line. And the backups for each run serially instead of at the same time. Also, before_everything hooks collected from all borgmatic configuration files run once before all configuration files (prior to all actions), but only if there is a create action.

Anyway, it sounds like the work here is to move remote_path be under individual repositories.

Multiple configuration files run with a single instance of borgmatic if you either put them in `/etc/borgmatic.d` or pass them in with `--config` on the command line. And the backups for each run serially instead of at the same time. Also, `before_everything` hooks collected from all borgmatic configuration files run once before all configuration files (prior to all actions), but only if there is a create action. Anyway, it sounds like the work here is to move `remote_path` be under individual repositories.

witten removed the

waiting for response

label

2026-01-26 23:12:30 +00:00

Rows
Columns

remote_path should (also) be per-repository #1185

What I'd like to do and why

Other notes / implementation ideas