On_error hook does not execute when "borg" is part of the command-string in before_backup and exit code is 1 #256

New Issue

NielsH · 2019-11-27T22:39:53Z

NielsH commented

2019-11-27 22:39:53 +00:00

What I'm trying to do and why

Trigger on_error hook when an error occurs.

Steps to reproduce (if a bug)

Run borgmatic with a config file that includes:

hooks:
  before_backup:
    - '/usr/local/sbin/before'
  on_error:
     - 'echo triggering on_error

Contents of /usr/local/sbin/before

#!/usr/bin/env bash

exit 1

With this config, run borgmatic -v 1
Get error (as expected)

/etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for pre-backup hook
/etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for on-error hook
triggering on_error

summary:
/etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Error running configuration file
/etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Error running pre-backup hook
Command '/usr/local/sbin/before' returned non-zero exit status 1

Need some help? https://torsion.org/borgmatic/#issue

Now change as this:
mv /usr/local/sbin/before /usr/local/sbin/borg-before

hooks:
  before_backup:
    - '/usr/local/sbin/borg-before'
  on_error:
     - 'echo triggering on_error

Output:

/etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for pre-backup hook
ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b: Pruning archives
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
Deleted data:                    0 B                  0 B                  0 B
All archives:              144.80 GB             82.34 GB              3.60 GB
                       Unique chunks         Total chunks
Chunk index:                   99738              2618484
------------------------------------------------------------------------------
ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b: Creating archive
Creating archive at "ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}"
.....<rest of backup output without errors>.....

Interesting thing to note is: if I do exit 2 instead of exit 1, the on_error is triggered correctly in both cases, regardles of whether the filename is before or borg-before

Environment

borgmatic version: 1.4.10

Use sudo borgmatic --version or sudo pip show borgmatic | grep ^Version

borgmatic installation method: pip

Borg version: 1.1.10

Use sudo borg --version

Python version: 3.5.3

Use python3 --version

operating system and version: Debian 9

#### What I'm trying to do and why Trigger on_error hook when an error occurs. #### Steps to reproduce (if a bug) Run borgmatic with a config file that includes: ``` hooks: before_backup: - '/usr/local/sbin/before' on_error: - 'echo triggering on_error ``` Contents of `/usr/local/sbin/before` ``` #!/usr/bin/env bash exit 1 ``` With this config, run `borgmatic -v 1` Get error (as expected) ``` /etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for pre-backup hook /etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for on-error hook triggering on_error summary: /etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Error running configuration file /etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Error running pre-backup hook Command '/usr/local/sbin/before' returned non-zero exit status 1 Need some help? https://torsion.org/borgmatic/#issue ``` Now change as this: `mv /usr/local/sbin/before /usr/local/sbin/borg-before` ``` hooks: before_backup: - '/usr/local/sbin/borg-before' on_error: - 'echo triggering on_error ``` Output: ``` /etc/borgmatic.d/backup02-3c923097-f49e-4454-855c-db8dc1635a5b.yaml: Running command for pre-backup hook ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b: Pruning archives ------------------------------------------------------------------------------ Original size Compressed size Deduplicated size Deleted data: 0 B 0 B 0 B All archives: 144.80 GB 82.34 GB 3.60 GB Unique chunks Total chunks Chunk index: 99738 2618484 ------------------------------------------------------------------------------ ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b: Creating archive Creating archive at "ssh://borg@<ip>:22/./3c923097-f49e-4454-855c-db8dc1635a5b::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}" .....<rest of backup output without errors>..... ``` Interesting thing to note is: if I do `exit 2` instead of `exit 1`, the on_error is triggered correctly in both cases, regardles of whether the filename is `before` or `borg-before` #### Environment **borgmatic version:** 1.4.10 Use `sudo borgmatic --version` or `sudo pip show borgmatic | grep ^Version` **borgmatic installation method:** pip **Borg version:** 1.1.10 Use `sudo borg --version` **Python version:** 3.5.3 Use `python3 --version` **operating system and version:** Debian 9

NielsH commented

2019-11-27 22:57:38 +00:00

This is probably related to: https://projects.torsion.org/witten/borgmatic/src/branch/master/borgmatic/execute.py#L17 ?

witten commented

2019-11-28 06:56:56 +00:00

Yes, it is due to that line! Here's what's going on: Because Borg considers exit code 2 as an error and exit code 1 as a warning, borgmatic treats any command it runs containing the string "borg" that exits with a value of 1 as just a warning.

And now you've discovered an edge case in which this heuristic doesn't work as intended.. In the before_backup hook, when you're typically not running Borg itself. Thanks for filing the detailed ticket.. I'll have to think of whether there's a better way to handle this.

Yes, it is due to that line! Here's what's going on: Because Borg considers exit code 2 as an error and exit code 1 as a warning, borgmatic treats any command it runs containing the string "borg" that exits with a value of 1 as just a warning. And now you've discovered an edge case in which this heuristic doesn't work as intended.. In the `before_backup` hook, when you're typically not running Borg itself. Thanks for filing the detailed ticket.. I'll have to think of whether there's a better way to handle this.

witten added the

bug

label 2019-11-28 06:57:03 +00:00

NielsH commented

2019-11-29 22:47:30 +00:00

Thanks for your response!
Would it not be possible to match on the base command name with ^borg$ ?
Or exclude user-defined hooks from the exit code check?

Thanks for your response! Would it not be possible to match on the base command name with `^borg$` ? Or exclude user-defined hooks from the exit code check?

witten commented

2019-11-30 22:55:51 +00:00

Matching on ^borg$ wouldn't work in all cases, because there are hosting providers where the Borg binary includes a version, e.g. borg1. I think your idea of excluding user-defined hooks from the exit code check is probably the way to go.

Matching on `^borg$` wouldn't work in all cases, because there are hosting providers where the Borg binary includes a version, e.g. `borg1`. I think your idea of excluding user-defined hooks from the exit code check is probably the way to go.

witten referenced this issue from a commit

2019-12-01 00:56:13 +00:00

Fix for "before_backup" hook not triggering an error when the command contains "borg" and has an exit code of 1 (#256).

witten commented

2019-12-01 00:59:50 +00:00

Fixed in borgmatic 1.4.16, just released. Thanks again for reporting this!

witten closed this issue

2019-12-01 00:59:51 +00:00

NielsH commented

2019-12-01 16:02:33 +00:00

Thank you for the fix!

👍 1