borgmatic_notifications/Readme.md

# Desktop notifications from borgmatic when it is run from systemd

This HowTo shows the way to set up *notifications from `borgmatic` to an arbitrary user*, when borgmatic runs as `root` because it was automatically started from a `system` timer. That implies a Linux machine, of course.

It includes workarounds for current (borgmatic 1.5.13, borg 1.1.16) limitations of `borgmatic` and/or `borg`. Also some downloaded files may have changed since the HowTo was written (2021-05-17).

The following needs to be set up for the notifications:

---

### systemd timer
The template from the borgmatic site (the`borgmatic.timer`) is fine, insert a string for `Description=…`, and set the `[Timer]` section if not already done. No special changes here.

---

### systemd service
Again, the template from the borgmatic site (the `borgmatic.service`) is good, but needs an essential change: 

The line `CapabilityBoundingSet=…` must grant the additional capabilities `AP_SETUID `and `CAP_SETGID`. This will allow borgmatic (and whatever is called from it!!) to act as a different user (other than root).

*__This weakens security settings.__ Make sure all permissions on borgmatic and scripts are set correctly!*

---

### borgmatic config

#### Notifications directly from borgmatic
A notification sent by borgmatic itself is set in its `config.yaml` for each hook, impersonating (`sudo -u`) the target user with their user name (`NAME`) and user id (`UID`). (This is what the additional capabilities in the `timer` were needed for.) `NAME` and `UID` can be looked up with `userdbctl`.

(Note: If the display is not `:0` the web knows a way to find the right value. This is not covered here.)

The `notify-send` command sets the urgency of the notifications, and sends a headline and a body text. The latter may include (very rudimentary) HTML formatting (rendered to varying degrees in the various desktops). In the `config.yaml` it looks like this (replace `NAME` and `UID`):

```
hooks:
    before_backup:
        - sudo -u NAME DISPLAY=:0 DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/UID/bus notify-send --urgency=normal 'Headline' 'Body text goes <i>here</i>.'
```
(Note: All after `sudo …` is one line. The config file is YAML, so there's no shell line continuation (` \`). And spaces, not tabs.)

---

#### Notifications from a script
Borgmatic calls an executable script that can do more magic and send the notifications in the same way as explained above.

```
hooks:
    on_error:
        - /etc/borgmatic/notify-error.sh "{configuration_filename}" "{repository}" "{error}" "{output}"
```

(Note: The placeholders (`{configuration_filename}`, `{repository}`, `{error}`, and `{output}`) are not all supported under all hooks.)

(Note: Some scripts from the web can send notifications to *all* users. Also not covered here.)
 
The notification command in the script:

```
#!/usr/bin/bash

sudo -u NAME DISPLAY=:0 \
    DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/UID/bus \
    notify-send --urgency=normal 'Headline' 'Body text goes <i>here</i>.'
```
(Note: Line continuation and the use of variables make more complex notifications substantially easier to set up than in `config.yaml`.)

---
---

### Example for Overdue Backups Alerts
#### borgmatic config
To know when the last complete backup was made, even if there is no connection to the repository, the date and time needs to be stored locally (here in a `last-successful-backup` file), after every successful backup (hook `after_backup:`).

This example uses date and time of the last *complete* backup. Borgmatic does not supply this in a placeholder, so it is identified with `borgmatic list --successful --last 1`, returning only date and time (`--format {time}`) and without control characters (`--no-color `), then the header line is skipped (`sed -n 2p`), and the timezone (that the borgmatic return lacks) is appended (`date +'%:z'`). 

It's a good idea to store this value together with the other files for that repository, so `/root/{repository}` would be nice. Unfortunately, `{repository}` is not resolved within borgmatic; the path must be manually copied from the top of the config file and append to `/root/`, *but __without the `:`__ after the URL!*

If an error occurs during backup, a script (here, `notify-error.sh`) will read that date and time and do the subsequent processing.

Example for a remote repository:

```
location:
    repositories:
        - BackupUser@BackupServer:/path/to/repository
...
hooks:
    after_backup:
        - echo "$(borgmatic list --successful --last 1 --format {time} --no-color \
            | sed -n 2p) $(date +'%:z')" \
            > "/root/BackupUser@BackupServer/path/to/repository/last-successful-backup"
...
    on_error:
        - /etc/borgmatic/notify-error.sh "{configuration_filename}" "{repository}" "{error}" "{output}"
```
(Note: Line continuation (` \`) can be used inside the quotes of `echo …` to improve readability.)

---

#### The notification script
For easy date and time calculations, this script makes use of `dateutils`. It will send slightly different notifications, depending on the age of the last successful backup:

```
#!/usr/bin/bash

# Notifies user of overdue borgmatic backups.

# Is called by borgmatic on errors during a prune, create, or check action as
# /etc/borgmatic/notify.sh "{configuration_filename}" "{repository}" "{error}" "{output}"

# Requires: dateutils


# set user to be notified (find NAME and UID with userdbctl)
NOTIFYUSER=NAME
NOTIFYUSERID=UID

# read date of last successful backup
LASTBACKUP=$(<"/root/BackupUser@BackupServer:/path/to/repository/last-successful-backup")
# get current datetime in a format for calculations and notification string
# (set LANG for compatibility with datediff's time format)
NOW="$(LANG=us_US date +'%a, %F %T %:z')"

# time calculations: backup age...
# ...in full hours for branching by age
BACKUPAGEHOURS=$(datediff -i "%a, %F %T %Z" \
    -f "%rH" \
    "$LASTBACKUP" now)
# ...as string for notifications, zero values removed
BACKUPAGESTRING=$(datediff -i "%a, %F %T %Z" \
    -f "%rY years %rm months %rw weeks %rd days %rH hours %rM minutes" \
    "$LASTBACKUP" now \
    | sed -E 's/(0 years |0 months |0 weeks |0 days |0 hours)//g')

# set message text
NOTIFYTEXT="Backup attempted $NOW.
<i>Last full backup: <u>$BACKUPAGESTRING</u> ago.</i>
Error details (more info in systemd journal):
⚫ Configuration file
$1
⚫ Repository
$2
⚫ Command output
$4
⚫ Error Message
$3"

# different actions depending on backup age
if   [ "$BACKUPAGEHOURS" -gt 72 ] # backup is older than 72 hours
    then
        sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
        notify-send --urgency=critical 'Borgmatic Backup SERIOUSLY OVERDUE!' "$NOTIFYTEXT"
elif [ "$BACKUPAGEHOURS" -gt 24 ] # backup is older than 24 hours
    then
        sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
        notify-send --urgency=critical 'Borgmatic Backup Overdue' "$NOTIFYTEXT"
else                              # backup age is 24 hours or less
    sudo -u "$NOTIFYUSER" DISPLAY=:0 "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$NOTIFYUSERID/bus" \
    notify-send --urgency=critical "Borgmatic Backup Failed" "$NOTIFYTEXT"
fi
```
![picture](https://projects.torsion.org/lasimik/borgmatic_notifications/raw/branch/master/borgmatic-notification1.png)
![picture](https://projects.torsion.org/lasimik/borgmatic_notifications/raw/branch/master/borgmatic-notification2.png)

      Don't let it come to this.