borgmatic list --successful omits certain non-checkpoint archives #224

New Issue

Closed

opened 2019-10-16 16:49:59 +00:00 by witten · 2 comments

witten commented 2019-10-16 16:49:59 +00:00

Owner

Copy Link

What I'm trying to do and why

Filter out checkpoint archives with borgmatic list --successful so as to only report on completed backups (e.g. for monitoring purposes).

Steps to reproduce / actual behavior

The existing implementation of --successful relies on a hacky glob match which doesn't work as expected. It excludes short archive names and those ending with a "t", even if they're not checkpoint archives.

See the discussion here: https://github.com/borgbackup/borg/issues/4047

Expected behavior

The --successful flag should only filter out checkpoint archives (those whose name ends with ".checkpoint", without excluding any other archives.

Other notes / implementation ideas

As noted in the linked ticket, the only possible approach right now is applying a checkpoint filter after Borg applies its own filters. This is a huge PITA because 1. Without the --json flag, borgmatic doesn't even consume Borg's output, and 2. Even with the --json flag, there's not a convenient place in the code right now to filter the output due to the way the results are yielded.

Environment

borgmatic version: 1.3.25

borgmatic installation method: pip install --user

Borg version: 1.1.10

Python version: 3.7.4

operating system and version: Manjaro Linux stable

#### What I'm trying to do and why Filter out checkpoint archives with `borgmatic list --successful` so as to only report on completed backups (e.g. for monitoring purposes). #### Steps to reproduce / actual behavior The existing implementation of `--successful` relies on a hacky glob match which doesn't work as expected. It excludes short archive names and those ending with a "t", even if they're not checkpoint archives. See the discussion here: https://github.com/borgbackup/borg/issues/4047 #### Expected behavior The `--successful` flag should only filter out checkpoint archives (those whose name ends with ".checkpoint", without excluding any other archives. #### Other notes / implementation ideas As noted in the linked ticket, the only possible approach right now is applying a checkpoint filter *after* Borg applies its own filters. This is a huge PITA because 1. Without the `--json` flag, borgmatic doesn't even consume Borg's output, and 2. Even with the `--json` flag, there's not a convenient place in the code right now to filter the output due to the way the results are `yield`ed. #### Environment **borgmatic version:** 1.3.25 **borgmatic installation method:** `pip install --user` **Borg version:** 1.1.10 **Python version:** 3.7.4 **operating system and version:** Manjaro Linux stable

witten added the

bug

label 2019-10-16 16:58:57 +00:00

witten commented 2019-10-16 17:28:19 +00:00

Author

Owner

Copy Link

Well, not a great fix, but perhaps better than the original implementation:
c12c47cace

Here are the new docs for this:

borgmatic list includes support for a --successful flag that only lists
successful (non-checkpoint) backups. This flag works via a basic heuristic: It
assumes that non-checkpoint archive names end with a digit (e.g. from a
timestamp), while checkpoint archive names do not. This means that if you're
using custom archive names that do not end in a digit, the --successful flag
will not work as expected.

Well, not a great fix, but perhaps better than the original implementation: https://projects.torsion.org/witten/borgmatic/commit/c12c47caceb93bc1ee6a41e298887b0cb8de759b Here are the new docs for this: > `borgmatic list` includes support for a `--successful` flag that only lists successful (non-checkpoint) backups. This flag works via a basic heuristic: It assumes that non-checkpoint archive names end with a digit (e.g. from a timestamp), while checkpoint archive names do not. This means that if you're using custom archive names that do not end in a digit, the `--successful` flag will not work as expected.

witten commented 2019-10-16 17:32:33 +00:00

Author

Owner

Copy Link

Fix released in 1.3.26.

Fix released in 1.3.26.

witten closed this issue 2019-10-16 17:32:33 +00:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#224

No description provided.