--keep-exclude-tags and other borg flags? #234

New Issue

anarcat · 2019-11-01T22:24:58Z

anarcat commented

2019-11-01 22:24:58 +00:00

What I'm trying to do and why

I'm trying to convert my silly shell scripts into nice little borgmatic configs.

Steps to reproduce (if a bug)

I have this shell script:


export BORG_RELOCATED_REPO_ACCESS_IS_OK=yes
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes

BORG_OPTS='--exclude-caches --keep-exclude-tags --one-file-system'
BORG_OPTS="$BORG_OPTS --exclude-if-present .nobackup"
BORG_OPTS="$BORG_OPTS --verbose --stats"

borg create $BORG_OPTS \
     -e "*/[Cc]ache/*" \
     -e "*.vmdk" \
     -e '/home/*/.cache/' \
     -e '*/.Trash-*/' \
     -e '*/.bitcoin/blocks/' \
     -e '*/build-area/*' \
     -e "/var/cache/*" \
     -e "/tmp/*" \
     -e "/var/tmp/*" \
     -e /srv/chroot \
     -e "/var/log/*" \
     "$ARCHIVE" / /boot /usr /var /home

I could find most flags, but not --keep-exclude-tags. I can't figure out how to pass arbitrary flags down to borg.

Other notes / implementation ideas

Maybe there could be an "extra flags" setting of some sort to just shove stuff down into borg?

Environment

borgmatic version: 1.2.11-1 (note that borgmatic --version gives an error)

borgmatic installation method: debian package

Borg version: 1.1.9

Python version: N/A

operating system and version: debian stable 10 (buster)

#### What I'm trying to do and why I'm trying to convert my silly shell scripts into nice little borgmatic configs. #### Steps to reproduce (if a bug) I have this shell script: ``` export BORG_RELOCATED_REPO_ACCESS_IS_OK=yes export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes BORG_OPTS='--exclude-caches --keep-exclude-tags --one-file-system' BORG_OPTS="$BORG_OPTS --exclude-if-present .nobackup" BORG_OPTS="$BORG_OPTS --verbose --stats" borg create $BORG_OPTS \ -e "*/[Cc]ache/*" \ -e "*.vmdk" \ -e '/home/*/.cache/' \ -e '*/.Trash-*/' \ -e '*/.bitcoin/blocks/' \ -e '*/build-area/*' \ -e "/var/cache/*" \ -e "/tmp/*" \ -e "/var/tmp/*" \ -e /srv/chroot \ -e "/var/log/*" \ "$ARCHIVE" / /boot /usr /var /home ``` I could find most flags, but not `--keep-exclude-tags`. I can't figure out how to pass arbitrary flags down to borg. #### Other notes / implementation ideas Maybe there could be an "extra flags" setting of some sort to just shove stuff down into borg? #### Environment **borgmatic version:** 1.2.11-1 (note that borgmatic --version gives an error) **borgmatic installation method:** debian package **Borg version:** 1.1.9 **Python version:** N/A **operating system and version:** debian stable 10 (buster)

witten referenced this issue from a commit

2019-11-01 22:52:40 +00:00

Support for Borg --keep-exclude-tags and --exclude-nodump options (#234).

witten commented

2019-11-01 22:58:46 +00:00

I implemented support for --keep-exclude-tags via a new borgmatic location section option called keep_exclude_tags. You should be able to upgrade to borgmatic 1.4.4 and set the option to true. Thanks for suggesting it!

As for a way to pass arbitrary flags to Borg.. I've instead been opting to add support to borgmatic for new or missing flags on-demand. That way borgmatic ultimately gets native support for everything Borg can do. Although I realize that can be somewhat obnoxious in the interim.

Please feel free to file tickets for any other missing options, and we can always revisit this approach if need be!

I implemented support for `--keep-exclude-tags` via a new borgmatic `location` section option called `keep_exclude_tags`. You should be able to upgrade to borgmatic 1.4.4 and set the option to `true`. Thanks for suggesting it! As for a way to pass arbitrary flags to Borg.. I've instead been opting to add support to borgmatic for new or missing flags on-demand. That way borgmatic ultimately gets native support for everything Borg can do. Although I realize that can be somewhat obnoxious in the interim. Please feel free to file tickets for any other missing options, and we can always revisit this approach if need be!

witten closed this issue

2019-11-01 22:58:46 +00:00

witten commented

2019-11-01 23:01:42 +00:00

borgmatic version: 1.2.11-1 (note that borgmatic --version gives an error)

That's probably because borgmatic --version was added in 1.2.14. Time to upgrade! 😄

> borgmatic version: 1.2.11-1 (note that borgmatic --version gives an error) That's probably because `borgmatic --version` was added in 1.2.14. Time to upgrade! :smile:

anarcat commented

2019-11-01 23:05:34 +00:00

it's great that borgmatic supports so many of borg's option, but this is bound to create needless complexity and synchronization issue.

it also makes it needlessly hard to keep both programs in sync: for example now I would need to uprgade borgmatic (to a version not yet packaged in debian, let alone backports) to benefit from this (great!) new functionality. if there would be support for arbitrary flags, i wouldn't need to upgrade at all!

inversely, if I want to try out a new borg feature without support from borgmatic, it's not possbile directly from borgmatic unless I get it implemented there first...

i would encourage you to reconsider, but i understand the reasons why you're going this way. :)

it would be nice to have a comparative table to list which features are supported and which aren't, if you keep going that way...

thanks for the extremely fast response!

it's great that borgmatic supports so many of borg's option, but this is bound to create needless complexity and synchronization issue. it also makes it needlessly hard to keep both programs in sync: for example now I would need to uprgade borgmatic (to a version not yet packaged in debian, let alone backports) to benefit from this (great!) new functionality. if there would be support for arbitrary flags, i wouldn't need to upgrade at all! inversely, if I want to try out a new borg feature without support from borgmatic, it's not possbile directly from borgmatic unless I get it implemented there first... i would encourage you to reconsider, but i understand the reasons why you're going this way. :) it would be nice to have a comparative table to list which features are supported and which aren't, if you keep going that way... thanks for the extremely fast response!

witten commented

2019-11-01 23:42:49 +00:00

I'd be willing to entertain support for arbitrary flags if you file a ticket for it, especially if this is something several users are interested in.. I just haven't seen a super strong need so far. But then again, I'm the one in the position to add borgmatic support for Borg flags as soon as I need them. 😃 I realize that doesn't apply to everyone!

There actually is a script included with borgmatic which tries to probe for flags that are not yet supported, but it's written in hacked-together shell and therefore doesn't work super reliably.

As for the topic of upgrading: I unfortunately don't have the resources to support all old versions of borgmatic. For instance, 1.2.11 is nearly a year old, and there have been 39 releases since then, some of them quite significant! I totally understand why you might be using Debian stable, and the version of borgmatic included with it.. but I encourage you to find a way to upgrade! Example: Switching to installing directly with pip.

If there are things I can do on my end to make using newer versions easier (keeping in mind limited resources), please let me know!

I'd be willing to entertain support for arbitrary flags if you file a ticket for it, especially if this is something several users are interested in.. I just haven't seen a super strong need so far. But then again, I'm the one in the position to add borgmatic support for Borg flags as soon as I need them. :smiley: I realize that doesn't apply to everyone! There actually is a script included with borgmatic which tries to probe for flags that are not yet supported, but it's written in hacked-together shell and therefore doesn't work super reliably. As for the topic of upgrading: I unfortunately don't have the resources to support all old versions of borgmatic. For instance, 1.2.11 is nearly a year old, and there have been 39 releases since then, some of them quite significant! I totally understand why you might be using Debian stable, and the version of borgmatic included with it.. but I encourage you to find a way to upgrade! Example: Switching to installing directly with pip. If there are things I can do on my end to make using newer versions easier (keeping in mind limited resources), please let me know!

anarcat commented

2019-11-02 00:20:44 +00:00

there are ways to run newer versions of stuff in debian with having to revert back to pip and so on. the normal way to do so is through "backports", but first the newer release needs to trickle down into unstable, testing and then stable-backports. since not even 1.4.0 has hit unstable yet, this might take a while. :)

but i'm not in a rush.

thanks!

there are ways to run newer versions of stuff in debian with having to revert back to pip and so on. the normal way to do so is through "backports", but first the newer release needs to trickle down into unstable, testing and *then* stable-backports. since not even 1.4.0 has hit unstable yet, this might take a while. :) but i'm not in a rush. thanks!

Sign in to join this conversation.