borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
14
58
Fork 41
Code Issues 58 Pull Requests 2 Actions Packages Releases 122 Activity

Overriding check_last when running borgmatic check --only archives #534

New Issue
Closed
opened 2022-05-16 06:34:23 +00:00 by trisager · 3 comments
trisager commented 2022-05-16 06:34:23 +00:00
Copy Link

What I'm trying to do and why

I am trying to run a manual consistency check of all archives in two repositories.

The config files have check_last: 3 in the consistency sections. The manual consistency check is intended to be run on demand, and should check all archives, not just the three latest.

Steps to reproduce (if a bug)

Create a config file with check_last: 3.

Run borgmatic check --only archives

Actual behavior (if a bug)

The check only verifies consistency of the last three arcives;

# borgmatic check --only archives -v 2

Ensuring legacy configuration is upgraded
borg --version --debug --show-rc
/etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function initialize_monitor
/etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/borgbase.yaml: Pinging Healthchecks start
/etc/borgmatic.d/borgbase.yaml: Using Healthchecks ping URL https://redacted/start
/etc/borgmatic.d/borgbase.yaml: No commands to run for pre-check hook
redacted:repo: Running consistency checks
borg check --archives-only --last 3 --prefix {hostname}- --debug --show-rc redacted:repo
using builtin fallback logging configuration
38 self tests completed in 0.07 seconds
SSH command line: ['ssh', 'redacted', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'repo'
Remote: Resolved repository path to '/srv/repos/redacted/repo'
Starting archive consistency check...
Remote: Verified integrity of /srv/repos/redacted/repo/index.112
TAM-verified manifest
Analyzing archive redacted (1/3)
Analyzing archive redacted (2/3)
Analyzing archive redacted (3/3)
RepositoryCache: current items 17, size 741.21 kB / 2.15 GB, 19 hits, 17 misses, 0 slow misses (+0.0s), 0 evictions, 0 ENOSPC hit

Orphaned objects check skipped (needs all archives checked).

Archive consistency check complete, no problems found.
RemoteRepository: 1.50 kB bytes sent, 989.61 kB bytes received, 31 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/borgbase.yaml: No commands to run for post-check hook
/etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/borgbase.yaml: Pinging Healthchecks finish
/etc/borgmatic.d/borgbase.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted
/etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function destroy_monitor
borg --version --debug --show-rc
/etc/borgmatic.d/oba.yaml: Calling healthchecks hook function initialize_monitor
/etc/borgmatic.d/oba.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/oba.yaml: Pinging Healthchecks start
/etc/borgmatic.d/oba.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted/start
/etc/borgmatic.d/oba.yaml: No commands to run for pre-check hook
redacted: Running consistency checks
borg check --archives-only --last 3 --prefix {hostname}- --debug --show-rc redacted:backups
using builtin fallback logging configuration
38 self tests completed in 0.08 seconds
SSH command line: ['ssh', 'redacted', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: 38 self tests completed in 0.11 seconds
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'backups'
Remote: Resolved repository path to 'redacted'
Starting archive consistency check...
Remote: Verified integrity of redacted/index.423
TAM-verified manifest
Analyzing archive redacted (1/3)
Analyzing archive redacted (2/3)
Analyzing archive redacted (3/3)
RepositoryCache: current items 16, size 908.02 kB / 2.15 GB, 20 hits, 16 misses, 0 slow misses (+0.0s), 0 evictions, 0 ENOSPC hit

Orphaned objects check skipped (needs all archives checked).

Archive consistency check complete, no problems found.
RemoteRepository: 1.46 kB bytes sent, 1.17 MB bytes received, 30 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/oba.yaml: No commands to run for post-check hook
/etc/borgmatic.d/oba.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/oba.yaml: Pinging Healthchecks finish
/etc/borgmatic.d/oba.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted
/etc/borgmatic.d/oba.yaml: Calling healthchecks hook function destroy_monitor

summary:
/etc/borgmatic.d/borgbase.yaml: Successfully ran configuration file
/etc/borgmatic.d/oba.yaml: Successfully ran configuration file

Expected behavior (if a bug)

The check should verify the integrity of all archives, disregarding the check_last: 3 line in the config file

Other notes / implementation ideas

Add to documentation if this is intentional (or won't be fixed).

The documentation could describe how to run this using an arbitrary borg command.

Environment

borgmatic version: 1.6.0

borgmatic installation method: sudo pip3 install --user --upgrade borgmatic

Borg version: 1.2.0

Python version: 3.8.10

Database version (if applicable): Ver 15.1 Distrib 10.5.15-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

operating system and version: Description: Ubuntu 20.04.4 LTS

#### What I'm trying to do and why I am trying to run a manual consistency check of all archives in two repositories. The config files have `check_last: 3` in the consistency sections. The manual consistency check is intended to be run on demand, and should check all archives, not just the three latest. #### Steps to reproduce (if a bug) Create a config file with `check_last: 3`. Run `borgmatic check --only archives` #### Actual behavior (if a bug) The check only verifies consistency of the last three arcives; ``` # borgmatic check --only archives -v 2 Ensuring legacy configuration is upgraded borg --version --debug --show-rc /etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function initialize_monitor /etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/borgbase.yaml: Pinging Healthchecks start /etc/borgmatic.d/borgbase.yaml: Using Healthchecks ping URL https://redacted/start /etc/borgmatic.d/borgbase.yaml: No commands to run for pre-check hook redacted:repo: Running consistency checks borg check --archives-only --last 3 --prefix {hostname}- --debug --show-rc redacted:repo using builtin fallback logging configuration 38 self tests completed in 0.07 seconds SSH command line: ['ssh', 'redacted', 'borg', 'serve', '--debug'] Remote: using builtin fallback logging configuration Remote: borg selftest disabled via BORG_SELFTEST env variable Remote: using builtin fallback logging configuration Remote: Initialized logging system for JSON-based protocol Remote: Resolving repository path b'repo' Remote: Resolved repository path to '/srv/repos/redacted/repo' Starting archive consistency check... Remote: Verified integrity of /srv/repos/redacted/repo/index.112 TAM-verified manifest Analyzing archive redacted (1/3) Analyzing archive redacted (2/3) Analyzing archive redacted (3/3) RepositoryCache: current items 17, size 741.21 kB / 2.15 GB, 19 hits, 17 misses, 0 slow misses (+0.0s), 0 evictions, 0 ENOSPC hit Orphaned objects check skipped (needs all archives checked). Archive consistency check complete, no problems found. RemoteRepository: 1.50 kB bytes sent, 989.61 kB bytes received, 31 messages sent terminating with success status, rc 0 /etc/borgmatic.d/borgbase.yaml: No commands to run for post-check hook /etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/borgbase.yaml: Pinging Healthchecks finish /etc/borgmatic.d/borgbase.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted /etc/borgmatic.d/borgbase.yaml: Calling healthchecks hook function destroy_monitor borg --version --debug --show-rc /etc/borgmatic.d/oba.yaml: Calling healthchecks hook function initialize_monitor /etc/borgmatic.d/oba.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/oba.yaml: Pinging Healthchecks start /etc/borgmatic.d/oba.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted/start /etc/borgmatic.d/oba.yaml: No commands to run for pre-check hook redacted: Running consistency checks borg check --archives-only --last 3 --prefix {hostname}- --debug --show-rc redacted:backups using builtin fallback logging configuration 38 self tests completed in 0.08 seconds SSH command line: ['ssh', 'redacted', 'borg', 'serve', '--debug'] Remote: using builtin fallback logging configuration Remote: 38 self tests completed in 0.11 seconds Remote: using builtin fallback logging configuration Remote: Initialized logging system for JSON-based protocol Remote: Resolving repository path b'backups' Remote: Resolved repository path to 'redacted' Starting archive consistency check... Remote: Verified integrity of redacted/index.423 TAM-verified manifest Analyzing archive redacted (1/3) Analyzing archive redacted (2/3) Analyzing archive redacted (3/3) RepositoryCache: current items 16, size 908.02 kB / 2.15 GB, 20 hits, 16 misses, 0 slow misses (+0.0s), 0 evictions, 0 ENOSPC hit Orphaned objects check skipped (needs all archives checked). Archive consistency check complete, no problems found. RemoteRepository: 1.46 kB bytes sent, 1.17 MB bytes received, 30 messages sent terminating with success status, rc 0 /etc/borgmatic.d/oba.yaml: No commands to run for post-check hook /etc/borgmatic.d/oba.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/oba.yaml: Pinging Healthchecks finish /etc/borgmatic.d/oba.yaml: Using Healthchecks ping URL https://hc-ping.com/redacted /etc/borgmatic.d/oba.yaml: Calling healthchecks hook function destroy_monitor summary: /etc/borgmatic.d/borgbase.yaml: Successfully ran configuration file /etc/borgmatic.d/oba.yaml: Successfully ran configuration file ``` #### Expected behavior (if a bug) The check should verify the integrity of all archives, disregarding the `check_last: 3` line in the config file #### Other notes / implementation ideas Add to documentation if this is intentional (or won't be fixed). The documentation could describe how to run this using an arbitrary borg command. #### Environment **borgmatic version:** 1.6.0 **borgmatic installation method:** `sudo pip3 install --user --upgrade borgmatic` **Borg version:** 1.2.0 **Python version:** 3.8.10 **Database version (if applicable):** Ver 15.1 Distrib 10.5.15-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 **operating system and version:** Description: Ubuntu 20.04.4 LTS
witten commented 2022-05-16 16:19:57 +00:00
Owner
Copy Link

Thanks for filing the ticket! The behavior you're seeing is actually by design. borgmatic runs your requested action (e.g. check) with all the configured options in borgmatic's configuration file so that you don't have to re-specify them. However, in this case, if you really want to override the check_last option, there is a way to do that! Try this:

borgmatic check --only archives --override consistency.check_last=0

The --override flag allows you to override arbitrary configuration file options on the command-line.

The documentation could describe how to run this using an arbitrary borg command.

It does! See this how-to and let me know if it meets your needs.

Thanks for filing the ticket! The behavior you're seeing is actually by design. borgmatic runs your requested action (e.g. `check`) with all the configured options in borgmatic's configuration file so that you don't have to re-specify them. However, in this case, if you really want to override the `check_last` option, there is a way to do that! Try this: ``` borgmatic check --only archives --override consistency.check_last=0 ``` The [`--override` flag](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#configuration-overrides) allows you to override arbitrary configuration file options on the command-line. > The documentation could describe how to run this using an arbitrary borg command. It does! [See this how-to](https://torsion.org/borgmatic/docs/how-to/run-arbitrary-borg-commands/) and let me know if it meets your needs.
trisager commented 2022-05-17 04:40:53 +00:00
Author
Copy Link

Thanks! I had missed the --override flag, perfect.

Thanks! I had missed the `--override` flag, perfect.
witten commented 2022-05-17 04:42:33 +00:00
Owner
Copy Link

Glad to hear it might work for you! If it turns out it doesn't for whatever reason, please let me know.

Glad to hear it might work for you! If it turns out it doesn't for whatever reason, please let me know.
witten added the
question / support
 label 2022-05-17 04:42:40 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#534
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?