Borg 2 date-based matching flags for archive selection #659

New Issue

witten · 2023-03-23T21:48:42Z

witten commented

2023-03-23 21:48:42 +00:00

What I'm trying to do and why

Borg 2.0beta5 recently added support for date-based archive matching flags on several sub-commands: --newer/--older/--newest/--oldest. See for instance the borg rlist documentation.

As part of this ticket, implement these flags on relevant borgmatic actions and plumb them through to the appropriate borgmatic/borg/*.py code so they make their way to Borg.

I personally find the descriptions of these flags in Borg's --help a little confusing, so consider coming up with simpler phrasing for documenting the borgmatic versions of these flags.

These flags can be Borg 2 only and documented as such. E.g.: "(Borg 2.x+ only)" in the description of each flag. There's no need to enforce that they only run with Borg 2 IMO.

#### What I'm trying to do and why Borg 2.0beta5 recently added support for date-based archive matching flags on several sub-commands: `--newer`/`--older`/`--newest`/`--oldest`. See for instance the [borg rlist documentation](https://borgbackup.readthedocs.io/en/2.0.0b5/usage/rlist.html). As part of this ticket, implement these flags on relevant borgmatic actions and plumb them through to the appropriate `borgmatic/borg/*.py` code so they make their way to Borg. I personally find the descriptions of these flags in Borg's `--help` a little confusing, so consider coming up with simpler phrasing for documenting the borgmatic versions of these flags. These flags can be Borg 2 only and documented as such. E.g.: "(Borg 2.x+ only)" in the description of each flag. There's no need to enforce that they only run with Borg 2 IMO.

witten added the

good first issue

label 2023-03-23 21:48:42 +00:00

witten referenced this issue

2023-03-23 21:51:31 +00:00

Allow restoring to other database server #326

jetchirag commented

2023-03-24 04:10:42 +00:00

I'll check this one.

👍 1

jetchirag commented

2023-03-24 08:05:37 +00:00

It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions?

There's no need to enforce that they only run with Borg 2 IMO.

Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298 (comment).

rlist_group.add_argument(
        '--oldest', metavar='TIMESPAN', help='X'
    )

It seems `--first` and `--last` flags are not implemented for some actions like `check` and `prune`. Should I add these timespan flags to such actions? > There's no need to enforce that they only run with Borg 2 IMO. Also, `add_argument()` in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying `borgmatic/borg/rlist.py` according to https://projects.torsion.org/borgmatic-collective/borgmatic/issues/298#issuecomment-5702. ``` rlist_group.add_argument( '--oldest', metavar='TIMESPAN', help='X' ) ```

witten commented

2023-03-24 20:51:31 +00:00

It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions?

You're talking about adding --first and --last flags to relevant borgmatic actions as part of this ticket? Sure, if you want to add those while you're adding the other flags, that'd be great!

Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298.

Yeah, it looks like borgmatic/borg/rlist.py calls flags.make_flags_from_arguments(), which should handle passing the newly added flags to Borg. But that may or may not be the case for other borgmatic actions.

> It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions? You're talking about adding `--first` and `--last` flags to relevant borgmatic actions as part of this ticket? Sure, if you want to add those while you're adding the other flags, that'd be great! > Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298. Yeah, it looks like `borgmatic/borg/rlist.py` calls `flags.make_flags_from_arguments()`, which should handle passing the newly added flags to Borg. But that may or may not be the case for other borgmatic actions.

jetchirag commented

2023-03-25 20:18:32 +00:00

Is there any specific reason make_flags_from_arguments() isn't used on every action? I'm working on the PR and have left the borg files untouched which are used this function and passed all arguments in others to make full_command

+ (flags.make_flags('first', mount_arguments.first) if mount_arguments.first else ())
+ (flags.make_flags('last', mount_arguments.last) if mount_arguments.last else ())
+ (flags.make_flags('newest', mount_arguments.newest) if mount_arguments.newest else ())
+ (flags.make_flags('oldest', mount_arguments.oldest) if mount_arguments.oldest else ())
+ (flags.make_flags('older', mount_arguments.older) if mount_arguments.older else ())
+ (flags.make_flags('newer', mount_arguments.newer) if mount_arguments.newer else ())

Is there any specific reason `make_flags_from_arguments()` isn't used on every action? I'm working on the PR and have left the borg files untouched which are used this function and passed all arguments in others to make `full_command` ``` + (flags.make_flags('first', mount_arguments.first) if mount_arguments.first else ()) + (flags.make_flags('last', mount_arguments.last) if mount_arguments.last else ()) + (flags.make_flags('newest', mount_arguments.newest) if mount_arguments.newest else ()) + (flags.make_flags('oldest', mount_arguments.oldest) if mount_arguments.oldest else ()) + (flags.make_flags('older', mount_arguments.older) if mount_arguments.older else ()) + (flags.make_flags('newer', mount_arguments.newer) if mount_arguments.newer else ()) ```

jetchirag referenced a pull request that will close this issue

2023-03-25 20:36:13 +00:00

Added TIMESPAN flags to match archive in various commands (Borg2 feature) #661

witten commented

2023-03-25 21:07:48 +00:00

Is there any specific reason make_flags_from_arguments() isn't used on every action?

It could probably be used in more places than it is, but make_flags_from_arguments() doesn't currently handle:

Flag logic that needs to vary based on the Borg version present (1.x vs. 2.x). The create action has a lot of this for instance.
Converting integers to strings when constructing flags. (Although this could be added.)
Non-arguments! Many flags that borgmatic passes to Borg have values coming from borgmatic configuration options or logging state rather than command-line arguments.
Probably other stuff I'm forgetting.

Having said all that though, feel free to use make_flags_from_arguments() wherever it makes sense / saves you typing.

> Is there any specific reason make_flags_from_arguments() isn't used on every action? It could probably be used in more places than it is, but `make_flags_from_arguments()` doesn't currently handle: * Flag logic that needs to vary based on the Borg version present (1.x vs. 2.x). The `create` action has a lot of this for instance. * Converting integers to strings when constructing flags. (Although this could be added.) * Non-arguments! Many flags that borgmatic passes to Borg have values coming from borgmatic configuration options or logging state rather than command-line arguments. * Probably other stuff I'm forgetting. Having said all that though, feel free to use `make_flags_from_arguments()` wherever it makes sense / saves you typing.

witten closed this issue

2023-05-23 21:26:18 +00:00

witten referenced this issue from a commit

2023-05-23 21:26:18 +00:00

Add Borg 2 date-based matching flags for archive selection (#659).

witten referenced this issue from a commit

2023-05-23 21:30:34 +00:00

Add date-based matching flags to NEWS (#659).

witten referenced this issue from a commit

2023-05-23 21:49:27 +00:00

Add integration test for mount action (#659).

witten referenced this issue from a commit