Borg 2 date-based matching flags for archive selection #659

New Issue

Closed

opened 2023-03-23 21:48:42 +00:00 by witten · 7 comments

witten commented 2023-03-23 21:48:42 +00:00

Owner

Copy Link

What I'm trying to do and why

Borg 2.0beta5 recently added support for date-based archive matching flags on several sub-commands: --newer/--older/--newest/--oldest. See for instance the borg rlist documentation.

As part of this ticket, implement these flags on relevant borgmatic actions and plumb them through to the appropriate borgmatic/borg/*.py code so they make their way to Borg.

I personally find the descriptions of these flags in Borg's --help a little confusing, so consider coming up with simpler phrasing for documenting the borgmatic versions of these flags.

These flags can be Borg 2 only and documented as such. E.g.: "(Borg 2.x+ only)" in the description of each flag. There's no need to enforce that they only run with Borg 2 IMO.

#### What I'm trying to do and why Borg 2.0beta5 recently added support for date-based archive matching flags on several sub-commands: `--newer`/`--older`/`--newest`/`--oldest`. See for instance the [borg rlist documentation](https://borgbackup.readthedocs.io/en/2.0.0b5/usage/rlist.html). As part of this ticket, implement these flags on relevant borgmatic actions and plumb them through to the appropriate `borgmatic/borg/*.py` code so they make their way to Borg. I personally find the descriptions of these flags in Borg's `--help` a little confusing, so consider coming up with simpler phrasing for documenting the borgmatic versions of these flags. These flags can be Borg 2 only and documented as such. E.g.: "(Borg 2.x+ only)" in the description of each flag. There's no need to enforce that they only run with Borg 2 IMO.

witten added the good first issue label 2023-03-23 21:48:42 +00:00

witten referenced this issue 2023-03-23 21:51:31 +00:00

Allow restoring to other database server #326

jetchirag commented 2023-03-24 04:10:42 +00:00

Contributor

Copy Link

I'll check this one.

I'll check this one.

👍 1

jetchirag commented 2023-03-24 08:05:37 +00:00

Contributor

Copy Link

It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions?

There's no need to enforce that they only run with Borg 2 IMO.

Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298 (comment).

rlist_group.add_argument(
        '--oldest', metavar='TIMESPAN', help='X'
    )

It seems `--first` and `--last` flags are not implemented for some actions like `check` and `prune`. Should I add these timespan flags to such actions? > There's no need to enforce that they only run with Borg 2 IMO. Also, `add_argument()` in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying `borgmatic/borg/rlist.py` according to https://projects.torsion.org/borgmatic-collective/borgmatic/issues/298#issuecomment-5702. ``` rlist_group.add_argument( '--oldest', metavar='TIMESPAN', help='X' ) ```

witten commented 2023-03-24 20:51:31 +00:00

Author

Owner

Copy Link

It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions?

You're talking about adding --first and --last flags to relevant borgmatic actions as part of this ticket? Sure, if you want to add those while you're adding the other flags, that'd be great!

Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298.

Yeah, it looks like borgmatic/borg/rlist.py calls flags.make_flags_from_arguments(), which should handle passing the newly added flags to Borg. But that may or may not be the case for other borgmatic actions.

> It seems --first and --last flags are not implemented for some actions like check and prune. Should I add these timespan flags to such actions? You're talking about adding `--first` and `--last` flags to relevant borgmatic actions as part of this ticket? Sure, if you want to add those while you're adding the other flags, that'd be great! > Also, add_argument() in arguments.py is passing the required flags correctly (checked rlist) so unless we want to enforce version there doesn't seem to need of modifying borgmatic/borg/rlist.py according to #298. Yeah, it looks like `borgmatic/borg/rlist.py` calls `flags.make_flags_from_arguments()`, which should handle passing the newly added flags to Borg. But that may or may not be the case for other borgmatic actions.

jetchirag commented 2023-03-25 20:18:32 +00:00

Contributor

Copy Link

Is there any specific reason make_flags_from_arguments() isn't used on every action? I'm working on the PR and have left the borg files untouched which are used this function and passed all arguments in others to make full_command

+ (flags.make_flags('first', mount_arguments.first) if mount_arguments.first else ())
+ (flags.make_flags('last', mount_arguments.last) if mount_arguments.last else ())
+ (flags.make_flags('newest', mount_arguments.newest) if mount_arguments.newest else ())
+ (flags.make_flags('oldest', mount_arguments.oldest) if mount_arguments.oldest else ())
+ (flags.make_flags('older', mount_arguments.older) if mount_arguments.older else ())
+ (flags.make_flags('newer', mount_arguments.newer) if mount_arguments.newer else ())

Is there any specific reason `make_flags_from_arguments()` isn't used on every action? I'm working on the PR and have left the borg files untouched which are used this function and passed all arguments in others to make `full_command` ``` + (flags.make_flags('first', mount_arguments.first) if mount_arguments.first else ()) + (flags.make_flags('last', mount_arguments.last) if mount_arguments.last else ()) + (flags.make_flags('newest', mount_arguments.newest) if mount_arguments.newest else ()) + (flags.make_flags('oldest', mount_arguments.oldest) if mount_arguments.oldest else ()) + (flags.make_flags('older', mount_arguments.older) if mount_arguments.older else ()) + (flags.make_flags('newer', mount_arguments.newer) if mount_arguments.newer else ()) ```

jetchirag referenced a pull request that will close this issue 2023-03-25 20:36:13 +00:00

Added TIMESPAN flags to match archive in various commands (Borg2 feature) #661

witten commented 2023-03-25 21:07:48 +00:00

Author

Owner

Copy Link

Is there any specific reason make_flags_from_arguments() isn't used on every action?

It could probably be used in more places than it is, but make_flags_from_arguments() doesn't currently handle:

• Flag logic that needs to vary based on the Borg version present (1.x vs. 2.x). The create action has a lot of this for instance.
• Converting integers to strings when constructing flags. (Although this could be added.)
• Non-arguments! Many flags that borgmatic passes to Borg have values coming from borgmatic configuration options or logging state rather than command-line arguments.
• Probably other stuff I'm forgetting.

Having said all that though, feel free to use make_flags_from_arguments() wherever it makes sense / saves you typing.

> Is there any specific reason make_flags_from_arguments() isn't used on every action? It could probably be used in more places than it is, but `make_flags_from_arguments()` doesn't currently handle: * Flag logic that needs to vary based on the Borg version present (1.x vs. 2.x). The `create` action has a lot of this for instance. * Converting integers to strings when constructing flags. (Although this could be added.) * Non-arguments! Many flags that borgmatic passes to Borg have values coming from borgmatic configuration options or logging state rather than command-line arguments. * Probably other stuff I'm forgetting. Having said all that though, feel free to use `make_flags_from_arguments()` wherever it makes sense / saves you typing.

witten closed this issue 2023-05-23 21:26:18 +00:00

witten referenced this issue from a commit 2023-05-23 21:26:18 +00:00

Add Borg 2 date-based matching flags for archive selection (#659).

witten referenced this issue from a commit 2023-05-23 21:30:34 +00:00

Add date-based matching flags to NEWS (#659).

witten referenced this issue from a commit 2023-05-23 21:49:27 +00:00

Add integration test for mount action (#659).

witten referenced this issue from a commit 2023-05-23 23:57:12 +00:00

Fix incorrect argument ordering (#659).

witten commented 2023-06-04 05:29:35 +00:00

Author

Owner

Copy Link

Just released as part of borgmatic 1.7.14!

Just released as part of borgmatic 1.7.14!

jetchirag commented 2023-06-04 07:08:39 +00:00

Contributor

Copy Link

Thank you for bearing with me and your continuous help! :)

Thank you for bearing with me and your continuous help! :)

👍 1

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

command-hook-steps

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

No Label good first issue

Milestone

No items

No Milestone

Assignees

Clear assignees

witten

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#659