ability to set retention by repository #608

Closed
opened 2022-10-29 07:16:05 +00:00 by eoli3n · 3 comments

What I'm trying to do and why

I have two repositories, one in borgbase, that I pay for, and one at home with free storage.
I would like to be able to store a minimum data in borgbase to keep the cheapest plan, and to store a lot of backups in my homeserver as main backup server.

The ability to set the retention by repository will let us adapt the amount of data per storage capacity.

The main problem to add this feature without breaking compatibility, is that the yaml structure doesn't have any id for repositories.

If you can refactor in a major version

location:
  repositories:
    - name: nas-backup
      remote: ssh://root@nas-backup/data/zfs/backups/work
    - name: borgbase
      remote: ssh://user@borgbase/azerty

# global
retention:
  prefix: '{hostname}-'
  keep_daily: 30
  
# specific
retention_nas-backup:
  prefix: '{hostname}-'
  keep_daily: 7
  keep_weekly: 4
  keep_monthly: 6
 
retention_borgbase:
  prefix: '{hostname}-'
  keep_daily: 7

Something like this could work with a minor update

location:
  repositories:
    - ssh://root@nas-backup/data/zfs/backups/work
    - ssh://user@borgbase/azerty

# global
retention:
  prefix: '{hostname}-'
  keep_daily: 30
  
  per_repository:
    - remote: ssh://root@nas-backup/data/zfs/backups/work
      keep_daily: 7
      keep_weekly: 4
      keep_monthly: 6
    - remote: ssh://user@borgbase/azerty
      keep_daily: 7
#### What I'm trying to do and why I have two repositories, one in borgbase, that I pay for, and one at home with free storage. I would like to be able to store a minimum data in borgbase to keep the cheapest plan, and to store a lot of backups in my homeserver as main backup server. The ability to set the retention by repository will let us adapt the amount of data per storage capacity. The main problem to add this feature without breaking compatibility, is that the yaml structure doesn't have any id for repositories. ### If you can refactor in a major version ```yaml location: repositories: - name: nas-backup remote: ssh://root@nas-backup/data/zfs/backups/work - name: borgbase remote: ssh://user@borgbase/azerty # global retention: prefix: '{hostname}-' keep_daily: 30 # specific retention_nas-backup: prefix: '{hostname}-' keep_daily: 7 keep_weekly: 4 keep_monthly: 6 retention_borgbase: prefix: '{hostname}-' keep_daily: 7 ``` ### Something like this could work with a minor update ```yaml location: repositories: - ssh://root@nas-backup/data/zfs/backups/work - ssh://user@borgbase/azerty # global retention: prefix: '{hostname}-' keep_daily: 30 per_repository: - remote: ssh://root@nas-backup/data/zfs/backups/work keep_daily: 7 keep_weekly: 4 keep_monthly: 6 - remote: ssh://user@borgbase/azerty keep_daily: 7 ```

You can accomplish this with "deep merge" includes. For example, put only repository and retention info in .yaml files in /etc/borgmatic.d, with everything else in "config.common":

# Local borg backup configuration.
# Local repository location and special retention is set here.
# All common configuration is set in config.common

<<: !include config.common

location:
    repositories:
        - /var/local/backups/local.borg

retention:
    keep_within: 3H
You can accomplish this with "deep merge" includes. For example, put only repository and retention info in .yaml files in /etc/borgmatic.d, with everything else in "config.common": ``` # Local borg backup configuration. # Local repository location and special retention is set here. # All common configuration is set in config.common <<: !include config.common location: repositories: - /var/local/backups/local.borg retention: keep_within: 3H ```
Owner

I'm going to close this one "won't fix" for now. The recommended way to vary configuration options by repository is by putting those repositories into separate borgmatic configuration files. And then you can always use includes as @rettops mentions to cut down on duplication if you'd like.

The rationale is that it's not feasible to make every option in a borgmatic configuration file repository-specific—that'd either require supporting repository-specific variants of each option as per your example or nesting an arbitrary number of options under each repository definition. I feel like a separate configuration file is much cleaner than either approach.

Of course, please feel free to comment if you disagree or have any other feedback on this.

I'm going to close this one "won't fix" for now. The recommended way to vary configuration options by repository is by putting those repositories into [separate borgmatic configuration files](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/). And then you can always use includes as @rettops mentions to cut down on duplication if you'd like. The rationale is that it's not feasible to make every option in a borgmatic configuration file repository-specific—that'd either require supporting repository-specific variants of each option as per your example or nesting an arbitrary number of options under each repository definition. I feel like a separate configuration file is much cleaner than either approach. Of course, please feel free to comment if you disagree or have any other feedback on this.
Author

I just did modify my config to use include, and it worked like a charm, thanks !

I just did modify my config to use include, and it worked like a charm, thanks !
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#608
No description provided.