borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 56
Code Issues 64 Pull requests 5 Releases 161 Packages 1 Activity Actions 1

check_last option is ignored when data consistency checks are enabled, even though data implies archives #611

New issue
Closed
opened 2022-11-15 01:36:07 +00:00 by 2rs2ts · 7 comments
2rs2ts commented 2022-11-15 01:36:07 +00:00
Copy link

What I'm trying to do and why

I want to only check the last 14 archives during my consistency checks, to save time. It is exceedingly rare that I will need to dip into archives older than that, so I'm willing to accept the risk of not checking them. However, the check_last option is being ignored in my case because I don't specify archives checks, but because I do data checks, I actually do perform the archive checks, resulting in my runs checking all the archives every time.

Steps to reproduce (if a bug)

This is the relevant part of my actual config that is experiencing this problem:

consistency:
    checks:
        - name: repository
          frequency: 1 weeks
        - name: data
          frequency: 1 weeks
        - name: extract
          frequency: 1 weeks
    check_last: 14

Actual behavior (if a bug)

Log result:

[2022-11-06 08:08:28,976] INFO: Ignoring check_last option, as "archives" is not in consistency checks

And then later on, we can see more than 14 archives are checked (I'll just show the last 2 lines, since the total numbers make it obvious it's more than 14):

[2022-11-06 20:10:45,353] INFO: Analyzing archive 2022-11-06T03:00:11 (26/26)
[2022-11-06 20:10:46,343] INFO: Archive consistency check complete, no problems found.

Expected behavior (if a bug)

I expected that it would not log something like this, and would only check the last 14 archives.

Other notes / implementation ideas

The reason for my expectation is that I read, from the sample config output, the following comment above the check names:

# Note that "data" implies "archives".

As well as the following comment above the check_last option:

# Applies only to the "archives" check.

I'm new to both borg and borgmatic, so I don't know for sure if this is a bug or just an unintuitive/poorly documented interface. I am not sure whether I should add archives to the consistency checks or not, since I thought that would be redundant with data. If this is not a bug, then I'd appreciate some advice on what I should change about my config, as well as an update to the documentation to help prevent others from similar confusion.

Environment

borgmatic version: 1.7.4

borgmatic installation method: homebrew

Borg version: 1.2.2

Python version: 3.6.7

Database version (if applicable): n/a

operating system and version: macOS 12.6
(using Automator and a calendar event to run borgmatic nightly.)

#### What I'm trying to do and why I want to only check the last 14 archives during my consistency checks, to save time. It is exceedingly rare that I will need to dip into archives older than that, so I'm willing to accept the risk of not checking them. However, the `check_last` option is being ignored in my case because I don't specify `archives` checks, but because I do `data` checks, I actually do perform the archive checks, resulting in my runs checking all the archives every time. #### Steps to reproduce (if a bug) This is the relevant part of my actual config that is experiencing this problem: ```yaml consistency: checks: - name: repository frequency: 1 weeks - name: data frequency: 1 weeks - name: extract frequency: 1 weeks check_last: 14 ``` #### Actual behavior (if a bug) Log result: ``` [2022-11-06 08:08:28,976] INFO: Ignoring check_last option, as "archives" is not in consistency checks ``` And then later on, we can see more than 14 archives are checked (I'll just show the last 2 lines, since the total numbers make it obvious it's more than 14): ``` [2022-11-06 20:10:45,353] INFO: Analyzing archive 2022-11-06T03:00:11 (26/26) [2022-11-06 20:10:46,343] INFO: Archive consistency check complete, no problems found. ``` #### Expected behavior (if a bug) I expected that it would not log something like this, and would only check the last 14 archives. #### Other notes / implementation ideas The reason for my expectation is that I read, from the sample config output, the following comment above the check names: ``` # Note that "data" implies "archives". ``` As well as the following comment above the `check_last` option: ``` # Applies only to the "archives" check. ``` I'm new to both borg and borgmatic, so I don't know for sure if this is a bug or just an unintuitive/poorly documented interface. I am not sure whether I should add `archives` to the consistency checks or not, since I thought that would be redundant with `data`. If this is not a bug, then I'd appreciate some advice on what I should change about my config, as well as an update to the documentation to help prevent others from similar confusion. #### Environment **borgmatic version:** 1.7.4 **borgmatic installation method:** homebrew **Borg version:** 1.2.2 **Python version:** 3.6.7 **Database version (if applicable):** n/a **operating system and version:** macOS 12.6 (using Automator and a calendar event to run borgmatic nightly.)
witten commented 2022-11-15 03:46:11 +00:00
Owner
Copy link

Thanks for taking the time to file this and explain the problem. What you've outlined here seems like a reasonable user expectation. Looking at the code, it should be a pretty easy change to make check_last apply when a data consistency check is present (and archives is only implied).

Thanks for taking the time to file this and explain the problem. What you've outlined here seems like a reasonable user expectation. Looking at the code, it should be a pretty easy change to make `check_last` apply when a `data` consistency check is present (and `archives` is only implied).
witten commented 2022-11-17 18:21:18 +00:00
Owner
Copy link

This has been fixed in master and will be part of the next release. I also fixed the consistency prefix option to work when only a data consistency check is present. Thanks again for filing this!

This has been fixed in master and will be part of the next release. I also fixed the consistency `prefix` option to work when only a `data` consistency check is present. Thanks again for filing this!
❤️ 1
witten commented 2022-11-27 17:05:54 +00:00
Owner
Copy link

Just released in borgmatic 1.7.5!

Just released in borgmatic 1.7.5!
2rs2ts commented 2022-12-02 03:42:14 +00:00
Author
Copy link

I upgraded yesterday and ran my backups; I no longer see this message:

INFO: Ignoring check_last option, as "archives" is not in consistency checks

However, it's still analyzing more archives than it should be. I still have check_last set to 14, but it analyzed 16 archives. Is this a new bug, or the same one, just not finished?

I upgraded yesterday and ran my backups; I no longer see this message: ``` INFO: Ignoring check_last option, as "archives" is not in consistency checks ``` However, it's still analyzing more archives than it should be. I still have `check_last` set to 14, but it analyzed 16 archives. Is this a new bug, or the same one, just not finished?
witten commented 2022-12-02 06:15:54 +00:00
Owner
Copy link

Would it be possible to run borgmatic with --verbosity 2 and look for the actual borg check command that's invoked? Does that include the --last flag? Thanks.

Would it be possible to run borgmatic with `--verbosity 2` and look for the actual `borg check` command that's invoked? Does that include the `--last` flag? Thanks.
2rs2ts commented 2022-12-02 06:28:04 +00:00
Author
Copy link

My apologies, I had 1.7.4 on the machine that experienced this, but thought I had 1.7.5. When you released 1.7.5, the homebrew formula hadn't yet updated, and I just did brew upgrade without paying attention to the actual version number. And I simply missed the INFO message when reading the logs, precisely because since the day I upgraded, I also changed my automation to use --verbosity 2 and that message occurred way earlier than I was expecting. Please disregard my above message. I've definitely got the right version now. If the problem recurs, I'll report back with the borg check command. Sorry again about that!

My apologies, I had 1.7.4 on the machine that experienced this, but thought I had 1.7.5. When you released 1.7.5, the homebrew formula hadn't yet updated, and I just did `brew upgrade` without paying attention to the actual version number. And I simply missed the INFO message when reading the logs, precisely because since the day I upgraded, I also changed my automation to use `--verbosity 2` and that message occurred way earlier than I was expecting. Please disregard my above message. I've definitely got the right version now. If the problem recurs, I'll report back with the `borg check` command. Sorry again about that!
witten commented 2022-12-02 06:29:51 +00:00
Owner
Copy link

Phew, no worries!

Phew, no worries!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
bootstrap-multiple-latest-archives
parallel-borgmatic
command-hook-steps
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#611
No description provided.