borgmatic_notifications/README.md

13 KiB

Desktop notifications from borgmatic when it is run from systemd

How to set up desktop notifications to an arbitrary user, while borgmatic is automatically run from a systemd timer. That implies a Linux machine, of course.

This includes workarounds for current (borgmatic 1.5.13, borg 1.1.16) limitations of borg/borgmatic. They may or may not be necessary in the future. This HowTo was written on 2021-05-17, some downloaded files may have changed since then.

The following needs to be set up for the notifications:

In the systemd timer

The template from the borgmatic site (theborgmatic.timer) is fine, insert a string for Description=…, and set the [Timer] section if not already done. No special changes here.

In the systemd service

Again, the template from the borgmatic site (the borgmatic.service) is good, but needs an essential change:

The line CapabilityBoundingSet=… must give the additional capabilities AP_SETUID and CAP_SETGID. This will allow borgmatic (and whatever is called from it!!) to impersonate a different user (other than root).

This means a softening of security settings. Make sure all permissions on borgmatic and scripts are set correctly!

In the borgmatic config

Notifications directly from borgmatic

A notification sent by borgmatic itself is set in its config.yaml for each hook, impersonating (sudo -u) the target user with their user name (NAME) and user id (UID). (This is what the additional capabilities in the timer were needed for.) NAME and UID can be looked up with userdbctl.

(Note: If the display is not ":0", the web knows a way to find the right value. This is not covered here.)

The notify-send command sets the urgency of the notifications, and sends a headline and a body text. The latter may include (very rudimentary) HTML formatting (to varying degrees, depending on the desktop). In the config.yaml it looks like this (replace NAME and UID):

hooks:
    before_backup:
            - sudo -u NAME DISPLAY=:0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/UID/bus notify-send --urgency=normal 'Headline' 'Body text goes <i>here</i>.'
            ```
            (Note: The config file is in YAML, you cannot use the shell line continuation (" \\"). And use spaces, not tabs.)

            -

            #### Notifications from a script
            Borgmatic calls an executable script that can do more magic and send the notifications in the same way as explained above.

            ```
            hooks:
                on_error:
                        - /etc/borgmatic/notify-error.sh "{configuration_filename}" "{repository}" "{error}" "{output}"
                        ```
                        (Note: The placeholders (`{configuration_filename}`, `{repository}`, `{error}`, and `{output}`) are not all supported under all hooks.)

                        (Note: Some scripts from the web can send notifications to *all* users. Also not covered here.)
                         
                         The notification command in the script:

                         ```
                         #!/usr/bin/bash

                         sudo -u NAME DISPLAY=:0 \
                             DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/UID/bus \
                                 notify-send --urgency=normal 'Headline' 'Body text goes here.'`
                                 ```
                                 (Note: Line continuation and the use of variables make complexer notifications substantially easier to set up than in `config.yaml`.)

                                 ---

                                 ### Example for Overdue Backups Alert
                                 #### In the borgmatic config
                                 To know when the last complete backup was made, even if there is no connection to the repository, the date and time needs to be stored locally (here in a `last-successful-backup` file), after every successful backup (hook `after_backup`).

                                 This example uses date and time of the last *complete* backup. Borgmatic does not supply this in a placeholder, so it is identified with `borgmatic list --successful --last 1`, returning only date and time (`--format {time}`) and without control characters (`--no-color `), then the header line is skipped (`sed -n 2p`), and the timezone (that the borgmatic return lacks) is appended (`date +'%:z'`). 

                                 It's a good idea to store this value together with the other files for that repository, so `/root/{repository}` would be nice. Unfortunately, `{repository}` is not resolved within borgmatic; the path must be manually copied from the top of the config file. 

                                 If an error occurs during backup, a script (here, `notify-error.sh`) will read that date and time and do the subsequent processing.

                                 Example for a remote repository:

                                 ```
                                 location:
                                     repositories:
                                             - BackupUser@BackupServer:/path/to/repository
                                             ...
                                             hooks:
                                                 after_backup:
                                                         - echo "$(borgmatic list --successful --last 1 --format {time} --no-color | sed -n 2p) $(date +'%:z')" \
                                                                     > "/root/BackupUser@BackupServer:/path/to/repository/last-successful-backup"
                                                                     ...
                                                                         on_error:
                                                                                 - /etc/borgmatic/notify-error.sh "{configuration_filename}" "{repository}" "{error}" "{output}"
                                                                                 ```
                                                                                 Note: Inside the quotes of `echo …` we can use line continuation (" \") for better readability.

                                                                                 #### The notification script
                                                                                 For easy date and time calculations, this script makes use of `dateutils`. It will send slightly different notifications, depending on the age of the last successful backup:

                                                                                 ```
                                                                                 #!/usr/bin/bash

                                                                                 # Notifies user of overdue borgmatic backups.

                                                                                 # Is called by borgmatic on errors during a prune, create, or check action as
                                                                                 # /etc/borgmatic/notify.sh "{configuration_filename}" "{repository}" "{error}" "{output}"

                                                                                 # Requires: dateutils


                                                                                 # set user to be notified (find name and id with userdbctl)
                                                                                 NOTIFYUSER=XXXXX
                                                                                 NOTIFYUSERID=NNNN

                                                                                 # read date of last successful backup, get current datetime
                                                                                 LASTBACKUP=$(<"/root/BackupUser@BackupServer:/path/to/repository/last-successful-backup")
                                                                                 NOW=$(date +'%F %H:%M %Z')

                                                                                 # time calculations: backup age...
                                                                                 # ...in full hours for branching by age
                                                                                 BACKUPAGEHOURS=$(datediff -i "%a, %F %T %Z" -f "%rH" \
                                                                                     "$LASTBACKUP" now)
                                                                                     # ...as string for notifications, with removal of zero values
                                                                                     BACKUPAGESTRING=$(datediff -i "%a, %F %T %Z" -f "%rY years %rm months %rw weeks %rd days %rH hours %rM minutes" \
                                                                                         "$LASTBACKUP" now | sed -E 's/(0 years |0 months |0 weeks |0 days |0 hours)//g')

                                                                                         # set message text
                                                                                         NOTIFYTEXT="Backup attempted $NOW.
                                                                                         <i>Last full backup: <u>$BACKUPAGESTRING</u> ago.</i>
                                                                                         Error details (more info in systemd journal):
                                                                                         ⚫ Configuration file
                                                                                         $1
                                                                                         ⚫ Repository
                                                                                         $2
                                                                                         ⚫ Command output
                                                                                         $4
                                                                                         ⚫ Error Message
                                                                                         $3"


                                                                                         # different actions depending on backup age
                                                                                         if   [ "$BACKUPAGEHOURS" -gt 72 ] # backup is older than 72 hours
                                                                                             then
                                                                                                     sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
                                                                                                             notify-send --urgency=critical 'Borgmatic Backup SERIOUSLY OVERDUE!' "$NOTIFYTEXT"
                                                                                                             elif [ "$BACKUPAGEHOURS" -gt 24 ] # backup is older than 24 hours
                                                                                                                 then
                                                                                                                         sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
                                                                                                                                 notify-send --urgency=critical 'Borgmatic Backup Overdue' "$NOTIFYTEXT"
                                                                                                                                 else                              # backup age is 24 hours or less
                                                                                                                                     sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
                                                                                                                                         notify-send --urgency=critical "Borgmatic Backup Failed" "$NOTIFYTEXT"
                                                                                                                                         fi
                                                                                                                                         ```