borgmatic-collective/borgmatic
11
70
Fork 40
Code Issues 51 Pull Requests 6 Actions 1 Packages Releases 152 Activity

Support for Borg 2 #557

New Issue
Closed
opened 2022-06-30 16:50:29 +00:00 by witten · 8 comments
witten commented 2022-06-30 16:50:29 +00:00
Owner
Copy Link

Borg 2 is currently in beta, and has a number of breaking changes that borgmatic will need to support:

  • borg transfer to transition 1.x repositories to the new 2.0 repo format.
  • Specify repository with new -r flag instead of :: syntax:
    • prune
    • compact
    • create
    • check
    • extract
    • export-tar
    • mount
    • umount
    • restore
    • list
    • info
    • borg
  • Replace use of --prefix with --glob-archives, e.g. --glob-archives 'myprefix*'.
    • prune
    • check
    • info
    • rlist
    • list
  • Specify archive with new positional parameter for most Borg sub-commands instead of :: syntax.
  • Specify archive with --glob-archives flag with a glob matching multiple archives for certain Borg sub-commands, e.g. borg delete --glob-archives 'myarchive*' or borg recreate --glob-archives 'myarchive*'.
  • scp style repository syntax is gone in favor of ssh://.
  • Repository sub-commands have been split off into new commands:
    • init -> rcreate
    • list -> rlist
    • info -> rinfo
  • Other Borg sub-commands may have changed as well. Audit and update the following borgmatic actions:
    • prune
    • compact
    • create
    • check
    • extract
    • export-tar
    • mount
    • umount
    • restore
    • list
    • info
    • borg
  • borg rcreate --other-repo flag and any additional new flags.
  • borg upgrade has been removed. (Wasn't wrapped by borgmatic.)
  • borg compact --cleanup-commits flag has been removed. (Going to leave it in place. If a user uses it with borgmatic and Borg 2, they'll get the same error they'll get when they use --cleanup-commits with Borg 2 directly.)
  • Replace --remote-ratelimit with --upload-ratelimit. Rename remote_rate_limit configuration option to upload_rate_limit as well.
  • Replace --numeric-owner with --numeric-ids. Rename numeric_owner configuration option to numeric_ids as well.
  • Replace --nobsdflags with --noflags. Rename bsd_flags configuration option to `flags as well.
  • Replace --noatime (the default now) with --atime. Fortunately, the configuration option is already atime.
  • Audit documentation for any changes needed that haven't already been made.

Full changelog: https://borgbackup.readthedocs.io/en/2.0.0b1/changes.html#version-2-0-0b1-2022-08-08

Discussion tickets: https://github.com/borgbackup/borg/discussions/6952 and https://github.com/borgbackup/borg/discussions/6799.

It's unclear at this point if the right approach is a breaking change release (e.g. borgmatic 2) that only supports Borg 2 ... or somehow make one version of borgmatic that supports both Borg 1.x and 2 simultaneously.

Borg 2 is currently in beta, and has a number of breaking changes that borgmatic will need to support: * [x] `borg transfer` to transition 1.x repositories to the new 2.0 repo format. * [x] Specify repository with new `-r` flag instead of `::` syntax: * [x] `prune` * [x] `compact` * [x] `create` * [x] `check` * [x] `extract` * [x] `export-tar` * [x] `mount` * [x] `umount` * [x] `restore` * [x] `list` * [x] `info` * [x] `borg` * [x] Replace use of `--prefix` with `--glob-archives`, e.g. `--glob-archives 'myprefix*'`. * [x] `prune` * [x] `check` * [x] `info` * [x] `rlist` * [x] `list` * [x] Specify archive with new positional parameter for most Borg sub-commands instead of `::` syntax. * [x] Specify archive with `--glob-archives` flag with a glob matching multiple archives for certain Borg sub-commands, e.g. `borg delete --glob-archives 'myarchive*'` or `borg recreate --glob-archives 'myarchive*'`. * [x] scp style repository syntax is gone in favor of `ssh://`. * [x] Repository sub-commands have been split off into new commands: * [x] `init` -> `rcreate` * [x] `list` -> `rlist` * [x] `info` -> `rinfo` * [x] Other Borg sub-commands may have changed as well. Audit and update the following borgmatic actions: * [x] `prune` * [x] `compact` * [x] `create` * [x] `check` * [x] `extract` * [x] `export-tar` * [x] `mount` * [x] `umount` * [x] `restore` * [x] `list` * [x] `info` * [x] `borg` * [x] `borg rcreate --other-repo` flag and any additional new flags. * [x] `borg upgrade` has been removed. (Wasn't wrapped by borgmatic.) * [x] `borg compact --cleanup-commits` flag has been removed. (Going to leave it in place. If a user uses it with borgmatic and Borg 2, they'll get the same error they'll get when they use `--cleanup-commits` with Borg 2 directly.) * [x] Replace `--remote-ratelimit` with `--upload-ratelimit`. Rename `remote_rate_limit` configuration option to `upload_rate_limit` as well. * [x] Replace `--numeric-owner` with `--numeric-ids`. Rename `numeric_owner` configuration option to `numeric_ids` as well. * [x] Replace `--nobsdflags` with `--noflags`. Rename `bsd_flags` configuration option to `flags as well. * [x] Replace `--noatime` (the default now) with `--atime`. Fortunately, the configuration option is already `atime`. * [x] Audit documentation for any changes needed that haven't already been made. Full changelog: https://borgbackup.readthedocs.io/en/2.0.0b1/changes.html#version-2-0-0b1-2022-08-08 Discussion tickets: https://github.com/borgbackup/borg/discussions/6952 and https://github.com/borgbackup/borg/discussions/6799. It's unclear at this point if the right approach is a breaking change release (e.g. borgmatic 2) that *only* supports Borg 2 ... or somehow make one version of borgmatic that supports both Borg 1.x and 2 simultaneously.
m3nu commented 2022-06-30 17:13:37 +00:00
Copy Link

It's unclear at this point if the right approach is a breaking change release (e.g. borgmatic 2) that only supports Borg 2 ... or somehow make one version of borgmatic that supports both Borg 1.x and 2 simultaneously.

Also unclear on this. Best to wait for the actual changes. For now the new arguments aren't finalized. Were just discussing on what to do with archive names today. They don't add a whole lot of value currently.

> It's unclear at this point if the right approach is a breaking change release (e.g. borgmatic 2) that only supports Borg 2 ... or somehow make one version of borgmatic that supports both Borg 1.x and 2 simultaneously. Also unclear on this. Best to wait for the actual changes. For now the new arguments aren't finalized. Were just discussing on what to do with archive names today. They don't add a whole lot of value currently.
witten commented 2022-06-30 17:22:16 +00:00
Author
Owner
Copy Link

Okay, good to know! Given how much borgmatic does its own thing with repository/archive syntax, it wouldn't be impossible to map that to whatever Borg settles on.

Okay, good to know! Given how much borgmatic does its own thing with repository/archive syntax, it wouldn't be impossible to map that to whatever Borg settles on.
witten commented 2022-08-12 21:57:28 +00:00
Author
Owner
Copy Link

With Borg 2 in beta, I began work on this in a branch. My current hypothesis is that borgmatic can support both Borg 1 and 2 simultaneously with the same version of code. Whether that approach remains feasible depends on how things go with the rest of the implementation. If it turns out that it's too much of a hassle, then I'll instead make a breaking change release that only supports Borg 2.

With Borg 2 in beta, I began work on this in [a branch](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/borg-2). My current hypothesis is that borgmatic can support both Borg 1 and 2 simultaneously with the same version of code. Whether that approach remains feasible depends on how things go with the rest of the implementation. If it turns out that it's too much of a hassle, then I'll instead make a breaking change release that only supports Borg 2.
m3nu commented 2022-08-13 05:26:14 +00:00
Copy Link

Great list! Adjusted it a bit for use with Vorta: https://github.com/borgbase/vorta/issues/1366

How do you plan on getting the Borg version at the very start? Vorta runs borg version and then we have a dict of available features. https://github.com/borgbase/vorta/blob/master/src/vorta/borg/_compatibility.py

Great list! Adjusted it a bit for use with Vorta: https://github.com/borgbase/vorta/issues/1366 How do you plan on getting the Borg version at the very start? Vorta runs `borg version` and then we have a dict of available features. https://github.com/borgbase/vorta/blob/master/src/vorta/borg/_compatibility.py
witten commented 2022-08-13 05:40:58 +00:00
Author
Owner
Copy Link

Yeah, borgmatic is already running borg version in a similar manner on start with feature lookup inspired by that Vorta code! See: https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/borg-2/borgmatic/borg/feature.py

Unrelated edit: I neglected to properly tag the following changesets with this ticket number:

Yeah, borgmatic is already running `borg version` in a similar manner on start with feature lookup inspired by that Vorta code! See: https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/borg-2/borgmatic/borg/feature.py Unrelated edit: I neglected to properly tag the following changesets with this ticket number: * cc04bf57dfedb26bfa663d512bfd97dc92962a91 * 89d201c8fffed6f01e11fc7851202e09a26e41b4
witten commented 2022-08-19 06:21:37 +00:00
Author
Owner
Copy Link

Merged to master! This will be part of the next release. Documentation on transferring repositories is here: https://torsion.org/borgmatic/docs/how-to/upgrade/#upgrading-borg

borgmatic changelog (such as it is) is here: https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/master/NEWS

Merged to master! This will be part of the next release. Documentation on transferring repositories is here: https://torsion.org/borgmatic/docs/how-to/upgrade/#upgrading-borg borgmatic changelog (such as it is) is here: https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/master/NEWS
witten commented 2022-08-22 04:59:32 +00:00
Author
Owner
Copy Link

Just released in borgmatic 1.7.0!

Just released in borgmatic 1.7.0!
m3nu commented 2022-08-22 06:17:07 +00:00
Copy Link

Congrats! Need to look at that commit how you ended up supporting 2 versions.

Congrats! Need to look at that commit how you ended up supporting 2 versions.
👍 1
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
witten
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#557
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?