Healthchecks integration for independent operations #366

Open
opened 2020-11-06 10:18:32 +00:00 by lgp171188 · 4 comments

What I'm trying to do and why

(This is related to witten/borgmatic#255)

I have 2 cron jobs, one that runs the prune and create operations multiple times a day and a check job that runs once a week at a given time.

I'd like to monitor both using the healthchecks integration.

Steps to reproduce (if a bug)

The healthchecks integration pings the healthchecks URL for both these cron jobs but it is impossible to set a schedule on a single healthchecks monitor that covers the expected starting times of both these cron jobs.

If I set the schedule of the monitor to match the multiple-times-a-day cron job, pings from the check job will be extraneous and will not trigger alerts when missed and vice-versa if I use the schedule of the 2nd cron job.

While I could stop using the integration and use manual curl commands to make it work, I then lose the benefits of the integration which provides more than just pinging URLs.

Actual behavior (if a bug)

See the previous section

Expected behavior (if a bug)

TBD as this is a feature request but it will be helpful to have a way to override/specify the healthchecks URL for specific actions.

Other notes / implementation ideas

Environment

borgmatic version: 1.5.10

borgmatic installation method: Installed in a virtualenv environment from PyPI.

Borg version: 1.1.14

Python version: 3.7.3

operating system and version: Debian 10 Buster

#### What I'm trying to do and why (This is related to https://projects.torsion.org/witten/borgmatic/issues/255) I have 2 cron jobs, one that runs the `prune` and `create` operations multiple times a day and a `check` job that runs once a week at a given time. I'd like to monitor both using the healthchecks integration. #### Steps to reproduce (if a bug) The healthchecks integration pings the healthchecks URL for both these cron jobs but it is impossible to set a schedule on a single healthchecks monitor that covers the expected starting times of both these cron jobs. If I set the schedule of the monitor to match the multiple-times-a-day cron job, pings from the `check` job will be extraneous and will not trigger alerts when missed and vice-versa if I use the schedule of the 2nd cron job. While I could stop using the integration and use manual `curl` commands to make it work, I then lose the benefits of the integration which provides more than just pinging URLs. #### Actual behavior (if a bug) See the previous section #### Expected behavior (if a bug) TBD as this is a feature request but it will be helpful to have a way to override/specify the healthchecks URL for specific actions. #### Other notes / implementation ideas #### Environment **borgmatic version:** 1.5.10 **borgmatic installation method:** Installed in a `virtualenv` environment from PyPI. **Borg version:** 1.1.14 **Python version:** 3.7.3 **operating system and version:** Debian 10 Buster
Owner

Thank you for taking the time to file this! A couple of thoughts here..

One way I could see to satisfy this need would be to expand the existing Healthchecks configuration schema to encompass different ping URLs per action. Made-up example:

hooks:
    healthchecks:
        create: https://hc-ping.com/uuid-here
        check: https://hc-ping.com/other-uuid-here
        prune: https://hc-ping.com/yet-another-uuid-here

Another idea is to leave the existing configuration schema as-is, but then override the ping URL on the command-line when running each action in a cron job. And use a separate ping URL / Healthchecks monitor for each cron job. Example:

borgmatic check --override hooks.healthchecks=https://hc-ping.com/your-uuid-here

This approach has the benefit of working today without any changes to borgmatic. The downside of course is you need to put the ping URL in your cron job.

Let me know your thoughts!

Thank you for taking the time to file this! A couple of thoughts here.. One way I could see to satisfy this need would be to expand the existing Healthchecks configuration schema to encompass different ping URLs per action. Made-up example: ```yaml hooks: healthchecks: create: https://hc-ping.com/uuid-here check: https://hc-ping.com/other-uuid-here prune: https://hc-ping.com/yet-another-uuid-here ``` Another idea is to leave the existing configuration schema as-is, but then override the ping URL on the command-line when running each action in a cron job. And use a separate ping URL / Healthchecks monitor for each cron job. Example: ```bash borgmatic check --override hooks.healthchecks=https://hc-ping.com/your-uuid-here ``` This approach has the benefit of working today without any changes to borgmatic. The downside of course is you need to put the ping URL in your cron job. Let me know your thoughts!
Author

@witten, thanks for the very quick response 🙂

Another idea is to leave the existing configuration schema as-is, but then override the ping URL on the command-line when running each action in a cron job. And use a separate ping URL / Healthchecks monitor for each cron job. Example:

This approach has the benefit of working today without any changes to borgmatic. The downside of course is you need to put the ping URL in your cron job.

I'm perfectly fine with using this solution for my needs if a lot of effort and thought has to be put into implementing the former. It will be good to document this scenario and solution.

One way I could see to satisfy this need would be to expand the existing Healthchecks configuration schema to encompass different ping URLs per action. Made-up example:

It might be easier to declare a default URL and override only the ones that have to be different.

@witten, thanks for the very quick response 🙂 > Another idea is to leave the existing configuration schema as-is, but then override the ping URL on the command-line when running each action in a cron job. And use a separate ping URL / Healthchecks monitor for each cron job. Example: > This approach has the benefit of working today without any changes to borgmatic. The downside of course is you need to put the ping URL in your cron job. I'm perfectly fine with using this solution for my needs if a lot of effort and thought has to be put into implementing the former. It will be good to document this scenario and solution. > One way I could see to satisfy this need would be to expand the existing Healthchecks configuration schema to encompass different ping URLs per action. Made-up example: It might be easier to declare a default URL and override only the ones that have to be different.
Owner

#518 is a very similar change on the Cronitor monitoring hook. It's possible that work there could apply to Healthchecks as well.

#518 is a very similar change on the Cronitor monitoring hook. It's possible that work there could apply to Healthchecks as well.

Subscribing since I have the same issue!

Subscribing since I have the same issue!
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#366
No description provided.