How to limit output to healthchecks.io? #277

Closed
opened 2020-01-04 21:21:37 +00:00 by aardbol · 18 comments

I'm using the healthchecks.io hook and the included systemd unit and timer for automated backups. What I noticed is that borgmatic also sends a body to hc.io which includes a list of file names and locations that were backed up and a summary in the end. I don't want those file names to be sent, but the summary is very interesting.

How can I limit the body contents to the summary? The standard setting in the .service file is --syslog-verbosity 1.

I'm using the healthchecks.io hook and the included systemd unit and timer for automated backups. What I noticed is that borgmatic also sends a body to hc.io which includes a list of file names and locations that were backed up and a summary in the end. I don't want those file names to be sent, but the summary is very interesting. How can I limit the body contents to the summary? The standard setting in the .service file is --syslog-verbosity 1.
Owner

There currently isn't a dedicated Healthchecks verbosity setting. Right now, it takes the maximum of any other borgmatic verbosity levels (--verbosity, --syslog-verbosity, etc.). Sounds like a dedicated Healtchecks verbosity setting would be a useful addition for your use case?

There currently isn't a dedicated Healthchecks verbosity setting. Right now, it takes the maximum of any other borgmatic verbosity levels (--verbosity, --syslog-verbosity, etc.). Sounds like a dedicated Healtchecks verbosity setting would be a useful addition for your use case?
Author

Actually I don't think that making a specific exception for hc.io is the right approach. Borgmatic should just have a summary option like borg has with "borg create -s", so that I can see the information about the repo size, the checking and pruning in the body of every hc.io log. That way I can replace --syslog-verbosity with the summary argument in the systemd unit file

Aside from that I think --syslog-verbosity isn't doing it's job probably because no logs are appended to the journal, nor is a separate log file created in /var/log

Actually I don't think that making a specific exception for hc.io is the right approach. Borgmatic should just have a summary option like borg has with "borg create -s", so that I can see the information about the repo size, the checking and pruning in the body of every hc.io log. That way I can replace --syslog-verbosity with the summary argument in the systemd unit file Aside from that I think --syslog-verbosity isn't doing it's job probably because no logs are appended to the journal, nor is a separate log file created in /var/log
Author

I just checked a second log and it seems like regardless of the verbosity setting, borgmatic is always sending the full data to hc.io. So an option to limit it to just the summary would be very interesting

I just checked a second log and it seems like regardless of the verbosity setting, borgmatic is always sending the full data to hc.io. So an option to limit it to just the summary would be very interesting
Owner

borgmatic create already has a -s/--stats flag, just like Borg's. However, it doesn't influence borgmatic's verbosity, so you can still tweak verbosity up or down independently.

Aside from that I think --syslog-verbosity isn’t doing it’s job probably because no logs are appended to the journal, nor is a separate log file created in /var/log

This sounds like a separate issue, perhaps. Could you provide more info here, or ideally in a new ticket? Are you perhaps running borgmatic interactively from the command-line? That disables syslog logging, with the idea that: 1. You have an interactive console where you can see output directly, so you don't need to go spelunking in logs for it, and 2. If you're running it interactively, you probably don't want your batch backup logs mucked up with all the output of you running borgmatic list, borgmatic extract, etc.

If you have a use case for syslog logging when running borgmatic interactively (say, testing!), I could see adding a way to override this default behavior.

I just checked a second log and it seems like regardless of the verbosity setting, borgmatic is always sending the full data to hc.io. So an option to limit it to just the summary would be very interesting

Okay, that to me suggests a separate --healthchecks-verbosity flag or perhaps a config file option.

`borgmatic create` already has a `-s`/`--stats` flag, just like Borg's. However, it doesn't influence borgmatic's verbosity, so you can still tweak verbosity up or down independently. > Aside from that I think --syslog-verbosity isn’t doing it’s job probably because no logs are appended to the journal, nor is a separate log file created in /var/log This sounds like a separate issue, perhaps. Could you provide more info here, or ideally in a new ticket? Are you perhaps running borgmatic interactively from the command-line? That disables syslog logging, with the idea that: 1. You have an interactive console where you can see output directly, so you don't need to go spelunking in logs for it, and 2. If you're running it interactively, you probably don't want your batch backup logs mucked up with all the output of you running `borgmatic list`, `borgmatic extract`, etc. If you have a use case for syslog logging when running borgmatic interactively (say, testing!), I could see adding a way to override this default behavior. > I just checked a second log and it seems like regardless of the verbosity setting, borgmatic is always sending the full data to hc.io. So an option to limit it to just the summary would be very interesting Okay, that to me suggests a separate `--healthchecks-verbosity` flag or perhaps a config file option.
Author

Perhaps a generic --hooks-verbosity would be more interesting?

As for the --syslog-verbosity, it's set by the systemd unit file that's provided by borgmatic. But it doesn't seem to log anything in the journal.

Perhaps a generic `--hooks-verbosity` would be more interesting? As for the `--syslog-verbosity`, it's set by the systemd unit file that's provided by borgmatic. But it doesn't seem to log anything in the journal.
Owner

As for the --syslog-verbosity, it’s set by the systemd unit file that’s provided by borgmatic. But it doesn’t seem to log anything in the journal.

A separate ticket for this would be helpful, along with any details you have about your systemd unit and associated configuration. Not sure why it wouldn't log at all.

> As for the --syslog-verbosity, it’s set by the systemd unit file that’s provided by borgmatic. But it doesn’t seem to log anything in the journal. A separate ticket for this would be helpful, along with any details you have about your systemd unit and associated configuration. Not sure why it wouldn't log at all.
Owner

Perhaps a generic --hooks-verbosity would be more interesting?

Yeah, that could work pretty well!

> Perhaps a generic --hooks-verbosity would be more interesting? Yeah, that could work pretty well!
Owner

Related ticket: #282.

Related ticket: #282.
Owner

Aside from that I think --syslog-verbosity isn’t doing it’s job probably because no logs are appended to the journal, nor is a separate log file created in /var/log

Are you perhaps using --log-file as well? Doing so disables syslog logging:

  --syslog-verbosity {-1,0,1,2}
                        Log verbose progress to syslog (from only errors to very verbose: -1, 0, 1, or
                        2). Ignored when console is interactive or --log-file is given
> Aside from that I think --syslog-verbosity isn’t doing it’s job probably because no logs are appended to the journal, nor is a separate log file created in /var/log Are you perhaps using `--log-file` as well? Doing so disables syslog logging: ```bash --syslog-verbosity {-1,0,1,2} Log verbose progress to syslog (from only errors to very verbose: -1, 0, 1, or 2). Ignored when console is interactive or --log-file is given ```
Owner

This is implemented in master now in the form of a new --monitoring-verbosity flag. Additionally, as of witten/borgmatic#283, detailed file listings and stats are disabled by default (everywhere, not just for the Healthchecks integration). You can opt back in with --files and --stats, respectively.

Please feel free to open another ticket for the syslog issue though.

This is implemented in master now in the form of a new `--monitoring-verbosity` flag. Additionally, as of https://projects.torsion.org/witten/borgmatic/pulls/283, detailed file listings and stats are disabled by default (everywhere, not just for the Healthchecks integration). You can opt back in with `--files` and `--stats`, respectively. Please feel free to open another ticket for the syslog issue though.
Author

Okay great thanks! I will be able to test it out as soon as you make another release, because my installation is linked to the Arch repositories and they are linked to the actual Git releases

Okay great thanks! I will be able to test it out as soon as you make another release, because my installation is linked to the Arch repositories and they are linked to the actual Git releases
Owner

Released in borgmatic 1.5.0! Hopefully it'll be packaged in Arch soon.

Released in borgmatic 1.5.0! Hopefully it'll be packaged in Arch soon.
Author

Hi

I'd like to get back to this. It seems like this change doesn't send anything useful anymore to healthchecks with level 1. Can you tell me which level also sends the borg stats in the end? That would be useful to be able to see in the HC logs

Hi I'd like to get back to this. It seems like this change doesn't send anything useful anymore to healthchecks with level 1. Can you tell me which level also sends the borg stats in the end? That would be useful to be able to see in the HC logs
Owner

I believe that all you should need to do is include the --stats option. Does that not send Borg stats data to Healthchecks for you?

I believe that all you should need to do is include the `--stats` option. Does that not send Borg stats data to Healthchecks for you?
Author

I'm afraid that doesn't help

I'm afraid that doesn't help
Owner

Can get a look-see at the borgmatic command-line invocation you're using? Are you using the --monitoring-verbosity flag?

Here's a command-line invocation that works for me:

borgmatic create -c test.yaml --monitoring-verbosity 1 --stats

By "works", I mean that the borgmatic stats are showing up in Healthcheck's UI when I run that.

Can get a look-see at the borgmatic command-line invocation you're using? Are you using the `--monitoring-verbosity` flag? Here's a command-line invocation that works for me: ```bash borgmatic create -c test.yaml --monitoring-verbosity 1 --stats ``` By "works", I mean that the borgmatic stats are showing up in Healthcheck's UI when I run that.
Author

Forgot the --monitoring-verbosity 1 part. Now it works thanks :D

Forgot the `--monitoring-verbosity 1` part. Now it works thanks :D
Owner

I'm glad that did it!

I'm glad that did it!
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#277
No description provided.