borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
14
57
Fork 41
Code Issues 53 Pull Requests 2 Actions Packages Releases 122 Activity

soft failure via findmnt not working #631

New Issue
Closed
opened 2023-01-25 09:18:42 +00:00 by hpaantee · 5 comments
hpaantee commented 2023-01-25 09:18:42 +00:00
Copy Link

What I'm trying to do and why

I have multiple borgmatic configurations (yaml files). Some of them are run weekly, while some are for removable hard drives. To not throw errors when my running borgmatic, there is the before_backup hook with the findmnt command as explained in https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/.

Steps to reproduce (if a bug)

I would expect to not receive an email, as the soft failure should not trigger them. The relevant section of my configurations looks like this:

hooks:
    before_backup:
        - findmnt /media/daniel/wd_book > /dev/null || exit 75
    on_error:
        - /etc/borgmatic/email_on_error.sh "{configuration_filename}" "{repository}" "{error}" "{output}"

Actual behavior (if a bug)

I get an email alert every week for every removable hard drive which borgmatic can not backup to. Running the relevant configuration with verbosity level 2:

/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release
  warnings.warn(
/usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
Ensuring legacy configuration is upgraded
/etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-everything hook
borg --version --debug --show-rc
/etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-actions hook
/etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-prune hook
/media/daniel/externe/Backups/eldin: Pruning archives
borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin
using builtin fallback logging configuration
38 self tests completed in 0.15 seconds
Repository /media/daniel/externe/Backups/eldin does not exist.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open
    st = os.stat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin'
During handling of the above exception, another exception occurred:
/media/daniel/externe/Backups/eldin: Error running actions for repository
Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2.
/etc/borgmatic.d/eldin_externe.yaml: Variable '{hostname}' is not supported in on-error hook
/etc/borgmatic.d/eldin_externe.yaml: Running command for on-error hook
/etc/borgmatic/email_on_error.sh "/etc/borgmatic.d/eldin_externe.yaml" "/media/daniel/externe/Backups/eldin" "Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2." "using builtin fallback logging configuration
38 self tests completed in 0.15 seconds
Repository /media/daniel/externe/Backups/eldin does not exist.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open
    st = os.stat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin'
During handling of the above exception, another exception occurred:"
/etc/borgmatic.d/eldin_externe.yaml: Error running configuration file
/etc/borgmatic.d/eldin_externe.yaml: No commands to run for post-everything hook

summary:
/etc/borgmatic.d/eldin_externe.yaml: Error running configuration file
/media/daniel/externe/Backups/eldin: Error running actions for repository
using builtin fallback logging configuration
38 self tests completed in 0.15 seconds
Repository /media/daniel/externe/Backups/eldin does not exist.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open
    st = os.stat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin'
During handling of the above exception, another exception occurred:
Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2.

Expected behavior (if a bug)

I should not get any email alert, the soft failure hook should prevent that.

Environment

borgmatic version: 1.7.5

borgmatic installation method: python pip

Borg version: 1.2

Python version: 3.10.6

operating system and version: Ubuntu 22.04

#### What I'm trying to do and why I have multiple borgmatic configurations (yaml files). Some of them are run weekly, while some are for removable hard drives. To not throw errors when my running borgmatic, there is the `before_backup` hook with the `findmnt` command as explained in https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/. #### Steps to reproduce (if a bug) I would expect to not receive an email, as the soft failure should not trigger them. The relevant section of my configurations looks like this: ``` hooks: before_backup: - findmnt /media/daniel/wd_book > /dev/null || exit 75 on_error: - /etc/borgmatic/email_on_error.sh "{configuration_filename}" "{repository}" "{error}" "{output}" ``` #### Actual behavior (if a bug) I get an email alert every week for every removable hard drive which borgmatic can not backup to. Running the relevant configuration with verbosity level 2: ``` /usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release warnings.warn( /usr/lib/python3/dist-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release warnings.warn( Ensuring legacy configuration is upgraded /etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-everything hook borg --version --debug --show-rc /etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-actions hook /etc/borgmatic.d/eldin_externe.yaml: No commands to run for pre-prune hook /media/daniel/externe/Backups/eldin: Pruning archives borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin using builtin fallback logging configuration 38 self tests completed in 0.15 seconds Repository /media/daniel/externe/Backups/eldin does not exist. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open st = os.stat(path) FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin' During handling of the above exception, another exception occurred: /media/daniel/externe/Backups/eldin: Error running actions for repository Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2. /etc/borgmatic.d/eldin_externe.yaml: Variable '{hostname}' is not supported in on-error hook /etc/borgmatic.d/eldin_externe.yaml: Running command for on-error hook /etc/borgmatic/email_on_error.sh "/etc/borgmatic.d/eldin_externe.yaml" "/media/daniel/externe/Backups/eldin" "Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2." "using builtin fallback logging configuration 38 self tests completed in 0.15 seconds Repository /media/daniel/externe/Backups/eldin does not exist. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open st = os.stat(path) FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin' During handling of the above exception, another exception occurred:" /etc/borgmatic.d/eldin_externe.yaml: Error running configuration file /etc/borgmatic.d/eldin_externe.yaml: No commands to run for post-everything hook summary: /etc/borgmatic.d/eldin_externe.yaml: Error running configuration file /media/daniel/externe/Backups/eldin: Error running actions for repository using builtin fallback logging configuration 38 self tests completed in 0.15 seconds Repository /media/daniel/externe/Backups/eldin does not exist. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/borg/repository.py", line 425, in open st = os.stat(path) FileNotFoundError: [Errno 2] No such file or directory: '/media/daniel/externe/Backups/eldin' During handling of the above exception, another exception occurred: Command 'borg prune --keep-within 7d --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 1 --glob-archives {hostname}-* --debug --show-rc /media/daniel/externe/Backups/eldin' returned non-zero exit status 2. ``` #### Expected behavior (if a bug) I should not get any email alert, the soft failure hook should prevent that. #### Environment **borgmatic version:** 1.7.5 **borgmatic installation method:** python pip **Borg version:** 1.2 **Python version:** 3.10.6 **operating system and version:** Ubuntu 22.04
witten commented 2023-01-25 18:10:59 +00:00
Owner
Copy Link

Based on the content of the error message, it looks like Borg itself is raising an error, specifically about:

No such file or directory: '/media/daniel/externe/Backups/eldin'

However, your before_backup findmnt command is running on an apparently different mount point: /media/daniel/wd_book. So is it possible that Borg is just being given the wrong path? Or that the findmnt command path needs tweaking?

Based on the content of the error message, it looks like Borg itself is raising an error, specifically about: ``` No such file or directory: '/media/daniel/externe/Backups/eldin' ``` However, your `before_backup` `findmnt` command is running on an apparently different mount point: `/media/daniel/wd_book`. So is it possible that Borg is just being given the wrong path? Or that the `findmnt` command path needs tweaking?
hpaantee commented 2023-01-25 20:28:25 +00:00
Author
Copy Link

Ah sorry, that was my mistake. I copied the wrong configuration file. The respective one is actually using
- findmnt /media/daniel/externe > /dev/null || exit 75
So the findmnt command and the log have the same mount point.

From the log is seems to not run the before_backup hook, as it starts pruning. I tried using the before_everything hook, but that also does not seem to work, it complains that it exited with error code 75. All in red font.

Ah sorry, that was my mistake. I copied the wrong configuration file. The respective one is actually using `- findmnt /media/daniel/externe > /dev/null || exit 75` So the `findmnt` command and the log have the same mount point. From the log is seems to not run the `before_backup` hook, as it starts pruning. I tried using the `before_everything` hook, but that also does not seem to work, it complains that it exited with error code 75. All in red font.
witten commented 2023-01-25 20:52:00 +00:00
Owner
Copy Link

Ah, good find, that's it! The before_backup hook only runs for the create action. So you could use the before_prune hook to run for the prune action. Or if you want this to run for all actions (which probably makes sense for this case), try before_actions. (Documentation here. I'll also add this to the removable drive docs so it's easier to find.)

before_everything didn't work for you because it doesn't support soft failures. (This is documented in the caveats and details.)

Ah, good find, that's it! The `before_backup` hook only runs for the `create` action. So you could use the `before_prune` hook to run for the prune action. Or if you want this to run for _all_ actions (which probably makes sense for this case), try `before_actions`. ([Documentation here](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/). I'll also add this to the [removable drive](https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/) docs so it's easier to find.) `before_everything` didn't work for you because it doesn't support soft failures. (This is documented in the [caveats and details](https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/#caveats-and-details).)
hpaantee commented 2023-01-26 07:06:47 +00:00
Author
Copy Link

Ah perfect, with the before_actions hook it works as expected now. Thanks a lot!

Ah perfect, with the `before_actions` hook it works as expected now. Thanks a lot!
witten commented 2023-01-26 07:07:28 +00:00
Owner
Copy Link

Glad to hear that did it!

Glad to hear that did it!
witten added the
question / support
 label 2023-01-26 07:07:36 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#631
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?