check_last option is ignored when data consistency checks are enabled, even though data implies archives #611

Closed
opened 5 months ago by 2rs2ts · 7 comments

What I'm trying to do and why

I want to only check the last 14 archives during my consistency checks, to save time. It is exceedingly rare that I will need to dip into archives older than that, so I'm willing to accept the risk of not checking them. However, the check_last option is being ignored in my case because I don't specify archives checks, but because I do data checks, I actually do perform the archive checks, resulting in my runs checking all the archives every time.

Steps to reproduce (if a bug)

This is the relevant part of my actual config that is experiencing this problem:

consistency:
    checks:
        - name: repository
          frequency: 1 weeks
        - name: data
          frequency: 1 weeks
        - name: extract
          frequency: 1 weeks
    check_last: 14

Actual behavior (if a bug)

Log result:

[2022-11-06 08:08:28,976] INFO: Ignoring check_last option, as "archives" is not in consistency checks

And then later on, we can see more than 14 archives are checked (I'll just show the last 2 lines, since the total numbers make it obvious it's more than 14):

[2022-11-06 20:10:45,353] INFO: Analyzing archive 2022-11-06T03:00:11 (26/26)
[2022-11-06 20:10:46,343] INFO: Archive consistency check complete, no problems found.

Expected behavior (if a bug)

I expected that it would not log something like this, and would only check the last 14 archives.

Other notes / implementation ideas

The reason for my expectation is that I read, from the sample config output, the following comment above the check names:

# Note that "data" implies "archives".

As well as the following comment above the check_last option:

# Applies only to the "archives" check. 

I'm new to both borg and borgmatic, so I don't know for sure if this is a bug or just an unintuitive/poorly documented interface. I am not sure whether I should add archives to the consistency checks or not, since I thought that would be redundant with data. If this is not a bug, then I'd appreciate some advice on what I should change about my config, as well as an update to the documentation to help prevent others from similar confusion.

Environment

borgmatic version: 1.7.4

borgmatic installation method: homebrew

Borg version: 1.2.2

Python version: 3.6.7

Database version (if applicable): n/a

operating system and version: macOS 12.6
(using Automator and a calendar event to run borgmatic nightly.)

#### What I'm trying to do and why I want to only check the last 14 archives during my consistency checks, to save time. It is exceedingly rare that I will need to dip into archives older than that, so I'm willing to accept the risk of not checking them. However, the `check_last` option is being ignored in my case because I don't specify `archives` checks, but because I do `data` checks, I actually do perform the archive checks, resulting in my runs checking all the archives every time. #### Steps to reproduce (if a bug) This is the relevant part of my actual config that is experiencing this problem: ```yaml consistency: checks: - name: repository frequency: 1 weeks - name: data frequency: 1 weeks - name: extract frequency: 1 weeks check_last: 14 ``` #### Actual behavior (if a bug) Log result: ``` [2022-11-06 08:08:28,976] INFO: Ignoring check_last option, as "archives" is not in consistency checks ``` And then later on, we can see more than 14 archives are checked (I'll just show the last 2 lines, since the total numbers make it obvious it's more than 14): ``` [2022-11-06 20:10:45,353] INFO: Analyzing archive 2022-11-06T03:00:11 (26/26) [2022-11-06 20:10:46,343] INFO: Archive consistency check complete, no problems found. ``` #### Expected behavior (if a bug) I expected that it would not log something like this, and would only check the last 14 archives. #### Other notes / implementation ideas The reason for my expectation is that I read, from the sample config output, the following comment above the check names: ``` # Note that "data" implies "archives". ``` As well as the following comment above the `check_last` option: ``` # Applies only to the "archives" check. ``` I'm new to both borg and borgmatic, so I don't know for sure if this is a bug or just an unintuitive/poorly documented interface. I am not sure whether I should add `archives` to the consistency checks or not, since I thought that would be redundant with `data`. If this is not a bug, then I'd appreciate some advice on what I should change about my config, as well as an update to the documentation to help prevent others from similar confusion. #### Environment **borgmatic version:** 1.7.4 **borgmatic installation method:** homebrew **Borg version:** 1.2.2 **Python version:** 3.6.7 **Database version (if applicable):** n/a **operating system and version:** macOS 12.6 (using Automator and a calendar event to run borgmatic nightly.)
Owner

Thanks for taking the time to file this and explain the problem. What you've outlined here seems like a reasonable user expectation. Looking at the code, it should be a pretty easy change to make check_last apply when a data consistency check is present (and archives is only implied).

Thanks for taking the time to file this and explain the problem. What you've outlined here seems like a reasonable user expectation. Looking at the code, it should be a pretty easy change to make `check_last` apply when a `data` consistency check is present (and `archives` is only implied).
witten added the
bug
label 5 months ago
Owner

This has been fixed in master and will be part of the next release. I also fixed the consistency prefix option to work when only a data consistency check is present. Thanks again for filing this!

This has been fixed in master and will be part of the next release. I also fixed the consistency `prefix` option to work when only a `data` consistency check is present. Thanks again for filing this!
witten closed this issue 4 months ago
Owner

Just released in borgmatic 1.7.5!

Just released in borgmatic 1.7.5!
Poster

I upgraded yesterday and ran my backups; I no longer see this message:

INFO: Ignoring check_last option, as "archives" is not in consistency checks

However, it's still analyzing more archives than it should be. I still have check_last set to 14, but it analyzed 16 archives. Is this a new bug, or the same one, just not finished?

I upgraded yesterday and ran my backups; I no longer see this message: ``` INFO: Ignoring check_last option, as "archives" is not in consistency checks ``` However, it's still analyzing more archives than it should be. I still have `check_last` set to 14, but it analyzed 16 archives. Is this a new bug, or the same one, just not finished?
Owner

Would it be possible to run borgmatic with --verbosity 2 and look for the actual borg check command that's invoked? Does that include the --last flag? Thanks.

Would it be possible to run borgmatic with `--verbosity 2` and look for the actual `borg check` command that's invoked? Does that include the `--last` flag? Thanks.
Poster

My apologies, I had 1.7.4 on the machine that experienced this, but thought I had 1.7.5. When you released 1.7.5, the homebrew formula hadn't yet updated, and I just did brew upgrade without paying attention to the actual version number. And I simply missed the INFO message when reading the logs, precisely because since the day I upgraded, I also changed my automation to use --verbosity 2 and that message occurred way earlier than I was expecting. Please disregard my above message. I've definitely got the right version now. If the problem recurs, I'll report back with the borg check command. Sorry again about that!

My apologies, I had 1.7.4 on the machine that experienced this, but thought I had 1.7.5. When you released 1.7.5, the homebrew formula hadn't yet updated, and I just did `brew upgrade` without paying attention to the actual version number. And I simply missed the INFO message when reading the logs, precisely because since the day I upgraded, I also changed my automation to use `--verbosity 2` and that message occurred way earlier than I was expecting. Please disregard my above message. I've definitely got the right version now. If the problem recurs, I'll report back with the `borg check` command. Sorry again about that!
Owner

Phew, no worries!

Phew, no worries!
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#611
Loading…
There is no content yet.