Make borgbackup context available in on_error hook #174

New Issue

Closed

opened 2019-05-14 11:43:43 +00:00 by decentral1se · 6 comments

decentral1se commented 2019-05-14 11:43:43 +00:00

Contributor

Copy Link

So, here is my current on_error definition:

on_error:
  - cat failed.txt | msmtp -a default infra@foo.com

Now, I run backups on multiple archives and I include and re-use this for all of them. The issue is, when an error happens on different archives, I get the same mail without context of which archive failed. I could not include and re-use this hook but then I would have to dupicate my hooks definition.

So, I am wondering how to pass some information like archive name, repository, time of archiving, etc. into the on_error context so that I can have it available to use for my emailing.

This could be considered feature creep. However, I already experience a lot of duplication in the borgmatic configurations (because of running multiple redundant remote repositories and different applications groups) and I think it is something borgmatic could improve on.

Thoughts?

So, here is my current `on_error` definition: ```yaml on_error: - cat failed.txt | msmtp -a default infra@foo.com ``` Now, I run backups on multiple archives and I include and re-use this for all of them. The issue is, when an error happens on different archives, I get the same mail without context of which archive failed. I could not include and re-use this hook but then I would have to dupicate my hooks definition. So, I am wondering how to pass some information like archive name, repository, time of archiving, etc. into the `on_error` context so that I can have it available to use for my emailing. This could be considered feature creep. However, I already experience a lot of duplication in the `borgmatic` configurations (because of running multiple redundant remote repositories and different applications groups) and I think it is something `borgmatic` could improve on. Thoughts?

witten commented 2019-05-14 18:09:09 +00:00

Owner

Copy Link

This does sound like feature creep, but also reasonable feature creep. (Almost like a variant of Zawinski's Law except for sending email.)

In the particular location in the code that the on_error hook is executed now, its "context" doesn't have everything you're looking for. It has: configuration filename, the loaded config, and any command-line arguments. In theory, with some refactoring, it could have access to more context. It's also possible that some of that information could ride on the exception object itself.

It's probably worth thinking about exactly what you'd want to see in an ideal email, because that might drive what context gets plumbed through.

I will say that it may be worth asking what the high-level feature / use case is that you're trying to solve for. For instance, is it more broadly "I want borgmatic error notifications, so I find out when my backups are busted and can fix them"? I say that because starting from the high-level use case may lead to a different solution than the (relatively) more surgical change you mention here.

Related issue (old): witten/borgmatic#39

Also related: witten/borgmatic#126

This does sound like feature creep, but also reasonable feature creep. (Almost like a variant of [Zawinski's Law](http://catb.org/jargon/html/Z/Zawinskis-Law.html) except for sending email.) In the particular location in the code that the `on_error` hook is executed now, its "context" doesn't have everything you're looking for. It has: configuration filename, the loaded config, and any command-line arguments. In theory, with some refactoring, it could have access to more context. It's also possible that some of that information could ride on the exception object itself. It's probably worth thinking about exactly what you'd want to see in an ideal email, because that might drive what context gets plumbed through. I will say that it may be worth asking what the high-level feature / use case is that you're trying to solve for. For instance, is it more broadly "I want borgmatic error notifications, so I find out when my backups are busted and can fix them"? I say that because starting from the high-level use case may lead to a different solution than the (relatively) more surgical change you mention here. Related issue (old): https://projects.torsion.org/witten/borgmatic/issues/39 Also related: https://projects.torsion.org/witten/borgmatic/issues/126

decentral1se commented 2019-05-14 18:54:16 +00:00

Author

Contributor

Copy Link

Righto! Let's then bring this discussion to witten/borgmatic#126.

Righto! Let's then bring this discussion to https://projects.torsion.org/witten/borgmatic/issues/126.

decentral1se closed this issue 2019-05-14 18:54:16 +00:00

witten commented 2019-10-01 19:07:30 +00:00

Owner

Copy Link

So, I lied. I think it may be worth implementing this even with the more holistic solution described in #126. My thinking is that there are a couple of different levels of backup alerting/monitoring one might want:

• A backup just failed. Send me an alert about it so I can look at the problem and fix it! In other words, this ticket.
• I don't trust the backup process to monitor itself. Also check that backups are running in general via some separate mechanism. This is more #126 territory.

So, I lied. I think it may be worth implementing this *even with* the more holistic solution described in #126. My thinking is that there are a couple of different levels of backup alerting/monitoring one might want: * A backup just failed. Send me an alert about it so I can look at the problem and fix it! In other words, this ticket. * I don't trust the backup process to monitor itself. Also check that backups are running in general via some separate mechanism. This is more #126 territory.

witten reopened this issue 2019-10-01 19:07:30 +00:00

witten referenced this issue from a commit 2019-10-01 19:24:53 +00:00

More detailed error alerting via runtime context available in "on_error" hook (#174).

witten commented 2019-10-01 19:27:48 +00:00

Owner

Copy Link

Okay, I went ahead and implemented this, released in borgmatic 1.3.23. There are now several context variables available in the on_error hook, although perhaps not all the ones you're looking for. We can always iterate. I'll post a link to the new docs when they finish deploying.

Okay, I went ahead and implemented this, released in borgmatic 1.3.23. There are now several context variables available in the `on_error` hook, although perhaps not all the ones you're looking for. We can always iterate. I'll post a link to the new docs when they finish deploying.

witten closed this issue 2019-10-01 19:27:48 +00:00

witten commented 2019-10-01 19:38:27 +00:00

Owner

Copy Link

The docs that were promised: https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/#error-alerting

Let me know how this works out for you.

The docs that were promised: https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/#error-alerting Let me know how this works out for you.

decentral1se commented 2019-10-01 20:00:43 +00:00

Author

Contributor

Copy Link

Excellent, thank you, I will take a look at this and integrate it into my setup.

Excellent, thank you, I will take a look at this and integrate it into my setup.

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#174

No description provided.