Unable to automate with either crontabs or systemd timers #482

What I'm trying to do and why

Attempting to automate my backups through the use of either crontabs or systemdtimers. I'm not a sysadmin so bear with me as I've probably made some obvious mistakes along the way. Hopefully this experience will be of value to others who likewise don't come from that sort of technical background. Here goes.

Steps to reproduce (if a bug)

Tested a manual backup using borgmatic create, no issues there. Proceeded to create a crontab with the following line

*/5 * * * * exampleuser PATH=$PATH:/usr/bin:/usr/local/bin /home/exampleuser/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1

I used the above path as thats what the "whereis borgmatic" told me was where the borgmatic executables are located

whereis borgmatic
borgmatic: /usr/bin/borgmatic /etc/borgmatic /home/exampleuser/.local/bin/borgmatic

crontab -l shows the tab as active
*/5 * * * * exampleuser PATH=$PATH:/usr/bin:/usr/local/bin /home/exampleuser/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1

However no new backups are created as shown when I check for new backups

The borgmatic list command lists only my initial manual backup (borgmatic create)

I'm logged in as a user with sudo privileges. Using an preinitialized borgbase repository.

No success on the crontab front, I then proceeded to have a go at the systemd timers route.

systemd timer service file config pasted below

#!/bin/bash

[Unit]
Description=borgmatic backup
Wants=network-online.target
After=network-online.target
ConditionACPower=false

[Service]
Type=oneshot

Lower CPU and I/O priority.

Nice=19
CPUSchedulingPolicy=batch
IOSchedulingClass=best-effort
IOSchedulingPriority=7
IOWeight=100

Restart=no

LogRateLimitIntervalSec=0

ExecStart=systemd-inhibit --who="borgmatic" --why="Prevents interrupting scheduled backup" /usr/bin/borgmatic --syslog-verbosity 1

Suspect I dont need to paste the contents of the timer file since it does trigger the service as scheduled (albeit with critical errors)

Logs yield these the following results

sudo systemctl status borgauto.service
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL @.repo.borgbase.com:repo: Error running actions for repository
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Remote: Host key verification failed.
Connection closed by remote host. Is borg working on the server?
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- --info @.repo.borgbase.com:repo' returned non-zero exit status 2.
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL
Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues
Dec 17 11:38:04 fedora systemd-inhibit[66035]: /usr/bin/borgmatic failed with exit status 1.
Dec 17 11:38:04 fedora systemd[1]: borgauto.service: Main process exited, code=exited, status=1/FAILURE
Dec 17 11:38:04 fedora systemd[1]: borgauto.service: Failed with result 'exit-code'.
Dec 17 11:38:04 fedora systemd[1]: Failed to start borgmatic backup.

Other notes / implementation ideas

I do have my ssh keypair locally encrypted with a password,Although it only asks for the password that encrypts the password the first time I connected to the repo through the terminal.

Not sure if there is anything else that might provide further clues towards a diagnosis of the problem. Do I need to pass any environmental variables or the ssh key pair to root?

Kind Regards

Environment

borgmatic version: 1.5.21

borgmatic installation method:

Installed Borgmatic through Fedora's package manager. sudo dnf install borgmatic.

Borg version: borg 1.1.17 .

Installed through Fedora's package manager sudo dnf install borgbackup

operating system and version:

Fedora 35

#### What I'm trying to do and why Attempting to automate my backups through the use of either crontabs or systemdtimers. I'm not a sysadmin so bear with me as I've probably made some obvious mistakes along the way. Hopefully this experience will be of value to others who likewise don't come from that sort of technical background. Here goes. #### Steps to reproduce (if a bug) Tested a manual backup using borgmatic create, no issues there. Proceeded to create a crontab with the following line */5 * * * * exampleuser PATH=$PATH:/usr/bin:/usr/local/bin /home/exampleuser/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1 I used the above path as thats what the "whereis borgmatic" told me was where the borgmatic executables are located whereis borgmatic borgmatic: /usr/bin/borgmatic /etc/borgmatic /home/exampleuser/.local/bin/borgmatic crontab -l shows the tab as active */5 * * * * exampleuser PATH=$PATH:/usr/bin:/usr/local/bin /home/exampleuser/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1 However no new backups are created as shown when I check for new backups The borgmatic list command lists only my initial manual backup (borgmatic create) I'm logged in as a user with sudo privileges. Using an preinitialized borgbase repository. No success on the crontab front, I then proceeded to have a go at the systemd timers route. systemd timer service file config pasted below #!/bin/bash [Unit] Description=borgmatic backup Wants=network-online.target After=network-online.target ConditionACPower=false [Service] Type=oneshot ## Lower CPU and I/O priority. Nice=19 CPUSchedulingPolicy=batch IOSchedulingClass=best-effort IOSchedulingPriority=7 IOWeight=100 Restart=no LogRateLimitIntervalSec=0 ExecStart=systemd-inhibit --who="borgmatic" --why="Prevents interrupting scheduled backup" /usr/bin/borgmatic --syslog-verbosity 1 Suspect I dont need to paste the contents of the timer file since it does trigger the service as scheduled (albeit with critical errors) Logs yield these the following results sudo systemctl status borgauto.service Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL *****@*****.repo.borgbase.com:repo: Error running actions for repository Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Remote: Host key verification failed. Connection closed by remote host. Is borg working on the server? Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- --info *****@*****.repo.borgbase.com:repo' returned non-zero exit status 2. Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Dec 17 11:38:04 fedora borgmatic[66036]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues Dec 17 11:38:04 fedora systemd-inhibit[66035]: /usr/bin/borgmatic failed with exit status 1. Dec 17 11:38:04 fedora systemd[1]: borgauto.service: Main process exited, code=exited, status=1/FAILURE Dec 17 11:38:04 fedora systemd[1]: borgauto.service: Failed with result 'exit-code'. Dec 17 11:38:04 fedora systemd[1]: Failed to start borgmatic backup. #### Other notes / implementation ideas I do have my ssh keypair locally encrypted with a password,Although it only asks for the password that encrypts the password the first time I connected to the repo through the terminal. Not sure if there is anything else that might provide further clues towards a diagnosis of the problem. Do I need to pass any environmental variables or the ssh key pair to root? Kind Regards #### Environment borgmatic version: 1.5.21 borgmatic installation method: Installed Borgmatic through Fedora's package manager. sudo dnf install borgmatic. Borg version: borg 1.1.17 . Installed through Fedora's package manager sudo dnf install borgbackup operating system and version: Fedora 35

CRITICAL Remote: Host key verification failed.

The user exampleuser (per your config example above) doesn't trust the remote host's SSH key. You can use ssh-keyscan to fetch the key and add it to the known_hosts file. Alternatively, if you connect using SSH as this user once you'll be prompted to add the key with something like this:

The authenticity of host 'localhost (::1)' can't be established.
ED25519 key fingerprint is SHA256:ABACADFAFAFAbXMoc.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Thanks @s1shed! And @borgwarrior, let us know if this solution doesn't work for you.

Sincerest apologies for the late reply, I've since reinstalled all packages but I'm running into an entirely different problem time time around in that when I try to enable the borgmatic timer using sudo systemctl enable --now borgmatic.timer I get the response

Failed to enable unit: Unit file borgmatic.timer does not exist.

Both the service and the timer file are clearly in /etc/systemd/system.

Failed to enable unit: Unit file borgmatic.timer does not exist.

Both the service and the timer file are clearly in /etc/systemd/system.

You probably didn't run systemctl daemon-reload after copying these files in place.

I ran sudo systemctl daemon-reload and then proceeded with "
sudo systemctl enable --now borgmatic.timer but I'm getting the same Failed to start borgmatic.timer: Unit borgmatic.timer not found." error message.

Most of the preinstalled timers/services are symlinked to other directories like /usr/lib/systemd/system but my timer+service files are not.

Thank you for your patience.

Screenshot from 2021-12-29 00-11-23.png

56 KiB

I ran sudo systemctl daemon-reload and then proceeded with "
sudo systemctl enable --now borgmatic.timer but I'm getting the same Failed to start borgmatic.timer: Unit borgmatic.timer not found." error message.

Most of the preinstalled timers/services are symlinked to other directories like /usr/lib/systemd/system but my timer+service files are not.

Thank you for your patience.

Please can you provide the output of ls -lZ /etc/systemd/system to see permissions.

Did you copied the service & timer files from /usr/lib/systemd manually?
You wrote you installed borgmatic via package manager. This will normally install both files under /usr/lib/systemd...
Do you have the same issue 'unit file ... does not exist' after reboot?

I made an error the first time by copying the files manually even though borgmatic installed through package manager already installs them in their desired locations. Attempting to correct my error, I removed the service & timer files I had in both /etc/systemd/system and /usr/lib/systemd/system. I then removed and reinstalled borgmatic through the package manager(dnf).

I then navigated to the /usr/lib directory and a file listing showed both files being installed there automatically as a result of the reinstallation of borgmatic (via dnf)

I then proceeded to run sudo systemctl enable --now borgmatic.timer which resulted in the following terminal output

*Created symlink /etc/systemd/system/timers.target.wants/borgmatic.timer → /usr/lib/systemd/system/borgmatic.timer.
*

The ls -lZ /etc/systemd/system permissions check yielded the following output

*drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root unconfinedu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 * 94 Oct 26 01:41 basic.target.wants
34 Oct 26 01:41 bluetooth.target.wants
37 Oct 26 01:41 ctrl-alt-del.target -> /usr/lib/systemd/system/reboot.target
41 Oct 26 01:41 dbus-org.bluez.service -> /usr/lib/systemd/system/bluetooth.service
41 Oct 26 01:42 dbus-org.fedoraproject.FirewallD1.service -> /usr/lib/systemd/system/firewalld.service
44 Oct 26 01:41 dbus-org.freedesktop.Avahi.service -> /usr/lib/systemd/system/avahi-daemon.service
45 Oct 26 01:41 dbus-org.freedesktop.home1.service -> /usr/lib/systemd/system/systemd-homed.service
44 Oct 26 01:41 dbus-org.freedesktop.ModemManager1.service -> /usr/lib/systemd/system/ModemManager.service
57 Oct 26 01:41 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service
44 Oct 26 01:41 dbus-org.freedesktop.oom1.service -> /usr/lib/systemd/system/systemd-oomd.service
48 Oct 26 01:41 dbus-org.freedesktop.resolve1.service -> /usr/lib/systemd/system/systemd-resolved.service
40 Oct 26 01:42 dbus-org.freedesktop.thermald.service -> /usr/lib/systemd/system/thermald.service
43 Oct 26 01:41 dbus.service -> /usr/lib/systemd/system/dbus-broker.service
40 Dec 18 13:07 default.target -> /usr/lib/systemd/system/graphical.target
48 Oct 26 01:42 'dev-virtio\x2dports-org.qemu.guestagent.0.device.wants'
35 Oct 26 01:42 display-manager.service -> /usr/lib/systemd/system/gdm.service
36 Oct 26 01:41 getty.target.wants
254 Oct 26 01:41 graphical.target.wants
44 Oct 26 01:41 local-fs.target.wants
924 Dec 19 07:24 multi-user.target.wants
68 Oct 26 01:41 network-online.target.wants
38 Oct 26 01:41 nfs-blkmap.service.requires
38 Oct 26 01:41 nfs-idmapd.service.requires
38 Oct 26 01:41 nfs-mountd.service.requires
38 Oct 26 01:41 nfs-server.service.requires
60 Oct 26 01:41 remote-fs.target.wants
38 Oct 26 01:41 rpc-gssd.service.requires
38 Oct 26 01:41 rpc-statd-notify.service.requires
38 Oct 26 01:41 rpc-statd.service.requires
708 Oct 26 01:42 sockets.target.wants
224 Dec 18 13:09 sysinit.target.wants
60 Oct 26 01:41 systemd-homed.service.wants
238 Jan 7 09:02 timers.target.wants
36 Oct 26 01:42 user@.service.wants
30 Oct 26 01:41 vmtoolsd.service.requires

Possibly promising, a query of the enabled timers this time shows the borgmatic timer within its listings

sudo systemctl list-timers

*Fri 2022-01-07 09:51:10 EST 41min left Fri 2022-01-07 03:22:31 EST 5h 47min ago dnf-makecache.timer dnf-makecache.service
Fri 2022-01-07 21:27:43 EST 12h left Thu 2022-01-06 16:54:31 EST 16h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
Sat 2022-01-08 00:00:00 EST 14h left n/a n/a borgmatic.timer borgmatic.service
Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago logrotate.timer logrotate.service
Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago mlocate-updatedb.timer mlocate-updatedb.service
Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago unbound-anchor.timer unbound-anchor.service
Sun 2022-01-09 01:00:00 EST 1 day 15h left Sun 2022-01-02 01:01:01 EST 5 days ago raid-check.timer raid-check.service
Mon 2022-01-10 01:09:18 EST 2 days left Tue 2022-01-04 09:16:02 EST 2 days ago fstrim.timer fstrim.service
*

Will try automating again under the fresh installation and report back. Once again, thank you for your patience.

I made an error the first time by copying the files manually even though borgmatic installed through package manager already installs them in their desired locations. Attempting to correct my error, I removed the service & timer files I had in both /etc/systemd/system and /usr/lib/systemd/system. I then removed and reinstalled borgmatic through the package manager(dnf). I then navigated to the /usr/lib directory and a file listing showed both files being installed there automatically as a result of the reinstallation of borgmatic (via dnf) I then proceeded to run sudo systemctl enable --now borgmatic.timer which resulted in the following terminal output *Created symlink /etc/systemd/system/timers.target.wants/borgmatic.timer → /usr/lib/systemd/system/borgmatic.timer. * The ls -lZ /etc/systemd/system permissions check yielded the following output *drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 94 Oct 26 01:41 basic.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 34 Oct 26 01:41 bluetooth.target.wants lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 37 Oct 26 01:41 ctrl-alt-del.target -> /usr/lib/systemd/system/reboot.target lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 41 Oct 26 01:41 dbus-org.bluez.service -> /usr/lib/systemd/system/bluetooth.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 41 Oct 26 01:42 dbus-org.fedoraproject.FirewallD1.service -> /usr/lib/systemd/system/firewalld.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 44 Oct 26 01:41 dbus-org.freedesktop.Avahi.service -> /usr/lib/systemd/system/avahi-daemon.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 45 Oct 26 01:41 dbus-org.freedesktop.home1.service -> /usr/lib/systemd/system/systemd-homed.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 44 Oct 26 01:41 dbus-org.freedesktop.ModemManager1.service -> /usr/lib/systemd/system/ModemManager.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 57 Oct 26 01:41 dbus-org.freedesktop.nm-dispatcher.service -> /usr/lib/systemd/system/NetworkManager-dispatcher.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 44 Oct 26 01:41 dbus-org.freedesktop.oom1.service -> /usr/lib/systemd/system/systemd-oomd.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 48 Oct 26 01:41 dbus-org.freedesktop.resolve1.service -> /usr/lib/systemd/system/systemd-resolved.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 40 Oct 26 01:42 dbus-org.freedesktop.thermald.service -> /usr/lib/systemd/system/thermald.service lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 43 Oct 26 01:41 dbus.service -> /usr/lib/systemd/system/dbus-broker.service lrwxrwxrwx. 1 root root unconfinedu:objectr:systemdunitfilet:s0 40 Dec 18 13:07 default.target -> /usr/lib/systemd/system/graphical.target drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 48 Oct 26 01:42 'dev-virtio\x2dports-org.qemu.guestagent.0.device.wants' lrwxrwxrwx. 1 root root systemu:objectr:systemdunitfilet:s0 35 Oct 26 01:42 display-manager.service -> /usr/lib/systemd/system/gdm.service drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 36 Oct 26 01:41 getty.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 254 Oct 26 01:41 graphical.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 44 Oct 26 01:41 local-fs.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 924 Dec 19 07:24 multi-user.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 68 Oct 26 01:41 network-online.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 nfs-blkmap.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 nfs-idmapd.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 nfs-mountd.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 nfs-server.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 60 Oct 26 01:41 remote-fs.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 rpc-gssd.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 rpc-statd-notify.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 38 Oct 26 01:41 rpc-statd.service.requires drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 708 Oct 26 01:42 sockets.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 224 Dec 18 13:09 sysinit.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 60 Oct 26 01:41 systemd-homed.service.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 238 Jan 7 09:02 timers.target.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 36 Oct 26 01:42 user@.service.wants drwxr-xr-x. 1 root root systemu:objectr:systemdunitfilet:s0 30 Oct 26 01:41 vmtoolsd.service.requires * Possibly promising, a query of the enabled timers this time shows the borgmatic timer within its listings sudo systemctl list-timers *Fri 2022-01-07 09:51:10 EST 41min left Fri 2022-01-07 03:22:31 EST 5h 47min ago dnf-makecache.timer dnf-makecache.service Fri 2022-01-07 21:27:43 EST 12h left Thu 2022-01-06 16:54:31 EST 16h ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service Sat 2022-01-08 00:00:00 EST 14h left n/a n/a borgmatic.timer borgmatic.service Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago logrotate.timer logrotate.service Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago mlocate-updatedb.timer mlocate-updatedb.service Sat 2022-01-08 00:00:00 EST 14h left Fri 2022-01-07 00:01:01 EST 9h ago unbound-anchor.timer unbound-anchor.service Sun 2022-01-09 01:00:00 EST 1 day 15h left Sun 2022-01-02 01:01:01 EST 5 days ago raid-check.timer raid-check.service Mon 2022-01-10 01:09:18 EST 2 days left Tue 2022-01-04 09:16:02 EST 2 days ago fstrim.timer fstrim.service * Will try automating again under the fresh installation and report back. Once again, thank you for your patience.

Another update, with my last install, a status check on the service/timer revealed that the timer failed due to the fact that no valid borgmatic configuration files could be found. I performed another fresh install, and probably due to tunnel vision I am not exactly sure what I did differently but this time the timer failed for a different reason (specified below).

Side note: Probably not distro specific but can anyone get borgmatic automation running on fedora (via a borgbase repo)?

sudo journalctl -u borgmatic --since="yesterday" yielded

Jan 10 03:00:02 fedora systemd[1]: Starting borgmatic backup...
Jan 10 03:01:02 fedora borgmatic[4343]: Mon Jan 10 03:01:02 AM EST 2022 - Starting backup
Jan 10 03:01:02 fedora borgmatic[4343]: WARNING Mon Jan 10 03:01:02 AM EST 2022 - Starting backup
Jan 10 03:01:05 fedora borgmatic[4343]: @.repo.borgbase.com:repo: Error running actions for repository
Jan 10 03:01:05 fedora borgmatic[4343]: Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- @.repo.borgbase.com:repo' returned non-zero exit status 2.
Jan 10 03:01:05 fedora borgmatic[4343]: /etc/borgmatic/config.yaml: Error running configuration file
Jan 10 03:01:05 fedora borgmatic[4343]: summary:
Jan 10 03:01:05 fedora borgmatic[4343]: /etc/borgmatic/config.yaml: Error running configuration file
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL @.repo.borgbase.com:repo: Error running actions for repository
Jan 10 03:01:05 fedora borgmatic[4343]: @.repo.borgbase.com:repo: Error running actions for repository
Jan 10 03:01:05 fedora borgmatic[4343]: Remote: Permission denied, please try again.
Jan 10 03:01:05 fedora borgmatic[4343]: Remote: Permission denied, please try again.
Jan 10 03:01:05 fedora borgmatic[4343]: Remote: @.repo.borgbase.com: Permission denied (publickey,password).
Jan 10 03:01:05 fedora borgmatic[4343]: Connection closed by remote host. Is borg working on the server?
Jan 10 03:01:05 fedora borgmatic[4343]: Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- @.repo.borgbase.com:repo' returned non-zero exit status 2.
Jan 10 03:01:05 fedora borgmatic[4343]: Need some help? https://torsion.org/borgmatic/#issues
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- @.repo.borgbase.com:repo' returned non-zero exit status 2.
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL summary:
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL @.repo.borgbase.com:repo: Error running actions for repository
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Remote: Permission denied, please try again.
Remote: Permission denied, please try again.
Remote: @.repo.borgbase.com: Permission denied (publickey,password).
Connection closed by remote host. Is borg working on the server?
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- @.repo.borgbase.com:repo' returned non-zero exit status 2.
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL
Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues
Jan 10 03:01:05 fedora systemd[1]: borgmatic.service: Main process exited, code=exited, status=1/FAILURE
Jan 10 03:01:05 fedora systemd[1]: borgmatic.service: Failed with result 'exit-code'.
Jan 10 03:01:05 fedora systemd[1]: Failed to start borgmatic backup.

Another update, with my last install, a status check on the service/timer revealed that the timer failed due to the fact that no valid borgmatic configuration files could be found. I performed another fresh install, and probably due to tunnel vision I am not exactly sure what I did differently but this time the timer failed for a different reason (specified below). Side note: Probably not distro specific but can anyone get borgmatic automation running on fedora (via a borgbase repo)? sudo journalctl -u borgmatic --since="yesterday" yielded Jan 10 03:00:02 fedora systemd[1]: Starting borgmatic backup... Jan 10 03:01:02 fedora borgmatic[4343]: Mon Jan 10 03:01:02 AM EST 2022 - Starting backup Jan 10 03:01:02 fedora borgmatic[4343]: WARNING Mon Jan 10 03:01:02 AM EST 2022 - Starting backup Jan 10 03:01:05 fedora borgmatic[4343]: ********@********.repo.borgbase.com:repo: Error running actions for repository Jan 10 03:01:05 fedora borgmatic[4343]: Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- ********@********.repo.borgbase.com:repo' returned non-zero exit status 2. Jan 10 03:01:05 fedora borgmatic[4343]: /etc/borgmatic/config.yaml: Error running configuration file Jan 10 03:01:05 fedora borgmatic[4343]: summary: Jan 10 03:01:05 fedora borgmatic[4343]: /etc/borgmatic/config.yaml: Error running configuration file Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL ********@********.repo.borgbase.com:repo: Error running actions for repository Jan 10 03:01:05 fedora borgmatic[4343]: ********@********.repo.borgbase.com:repo: Error running actions for repository Jan 10 03:01:05 fedora borgmatic[4343]: Remote: Permission denied, please try again. Jan 10 03:01:05 fedora borgmatic[4343]: Remote: Permission denied, please try again. Jan 10 03:01:05 fedora borgmatic[4343]: Remote: ********@********.repo.borgbase.com: Permission denied (publickey,password). Jan 10 03:01:05 fedora borgmatic[4343]: Connection closed by remote host. Is borg working on the server? Jan 10 03:01:05 fedora borgmatic[4343]: Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- ********@********.repo.borgbase.com:repo' returned non-zero exit status 2. Jan 10 03:01:05 fedora borgmatic[4343]: Need some help? https://torsion.org/borgmatic/#issues Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- ********@********.repo.borgbase.com:repo' returned non-zero exit status 2. Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL summary: Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL /etc/borgmatic/config.yaml: Error running configuration file Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL ********@********.repo.borgbase.com:repo: Error running actions for repository Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Remote: Permission denied, please try again. Remote: Permission denied, please try again. Remote: ********@********.repo.borgbase.com: Permission denied (publickey,password). Connection closed by remote host. Is borg working on the server? Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Command 'borg prune --keep-daily 3 --keep-weekly 4 --keep-monthly 12 --keep-yearly 2 --prefix {hostname}- ********@********.repo.borgbase.com:repo' returned non-zero exit status 2. Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues Jan 10 03:01:05 fedora systemd[1]: borgmatic.service: Main process exited, code=exited, status=1/FAILURE Jan 10 03:01:05 fedora systemd[1]: borgmatic.service: Failed with result 'exit-code'. Jan 10 03:01:05 fedora systemd[1]: Failed to start borgmatic backup.

The user who's running the backup needs to have an ssh key that matches the repository. This failure is from SSH.

Jan 10 03:01:05 fedora borgmatic[4343]: CRITICAL Remote: Permission denied, please try again.
Remote: Permission denied, please try again.
Remote: @.repo.borgbase.com: Permission denied (publickey,password).

By default there's a commented out stanza for ssh_command in the borgmatic config. You can specify the path to the key here:

    # Command to use instead of "ssh". This can be used to specify
    # ssh options.  Defaults to not set.
    # ssh_command: ssh -i /path/to/private/key

Overlooked that as I used the borgmatic configuration file automatically generated by the borgbase setup wizard which did not include that stanza. Used the borgmatic config provided by borgmatic and placed the path to the private ssh key but upon trying to validate the config file I am now getting the "did not find expected key" error message. I've checked and the key is definitely in the correct location.

Details below.

--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 105, in parse_configuration
config = load.load_configuration(config_filename)
File "/usr/lib/python3.10/site-packages/borgmatic/config/load.py", line 20, in load_configuration
return yaml.load(open(filename))
File "/usr/lib/python3.10/site-packages/ruamel/yaml/main.py", line 341, in load
return constructor.get_single_data()
File "/usr/lib/python3.10/site-packages/ruamel/yaml/constructor.py", line 111, in get_single_data
node = self.composer.get_single_node()
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 706, in _ruamel_yaml.CParser.get_single_node
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 724, in _ruamel_yaml.CParser._compose_document
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 891, in _ruamel_yaml.CParser._compose_mapping_node
File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event
ruamel.yaml.parser.ParserError: while parsing a block mapping
in "/etc/borgmatic/config.yaml", line 8, column 5
did not find expected key
in "/etc/borgmatic/config.yaml", line 187, column 6

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/borgmatic/commands/validate_config.py", line 45, in main
validate.parse_configuration(config_filename, validate.schema_filename())
File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 108, in parse_configuration
raise Validation_error(config_filename, (str(error),))
borgmatic.config.validate.Validation_error:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/python3.10/logging/init.py", line 1098, in emit
msg = self.format(record)
File "/usr/lib64/python3.10/logging/init.py", line 942, in format
return fmt.format(record)
File "/usr/lib64/python3.10/logging/init.py", line 678, in format
record.message = record.getMessage()
File "/usr/lib64/python3.10/logging/init.py", line 366, in getMessage
msg = str(self.msg)
File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 59, in str
) + '\n'.join(format_error(error) for error in self.errors)
File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 59, in
) + '\n'.join(format_error(error) for error in self.errors)
File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 32, in format_error
if not error.path:
AttributeError: 'str' object has no attribute 'path'
Call stack:
File "/usr/bin/validate-borgmatic-config", line 33, in
sys.exit(load_entry_point('borgmatic==1.5.21', 'console_scripts', 'validate-borgmatic-config')())
File "/usr/lib/python3.10/site-packages/borgmatic/commands/validate_config.py", line 48, in main
logging.critical(error)
Message: Validation_error('/etc/borgmatic/config.yaml', ('while parsing a block mapping\n in "/etc/borgmatic/config.yaml", line 8, column 5\ndid not find expected key\n in "/etc/borgmatic/config.yaml", line 187, column 6',))
Arguments: ()

Overlooked that as I used the borgmatic configuration file automatically generated by the borgbase setup wizard which did not include that stanza. Used the borgmatic config provided by borgmatic and placed the path to the private ssh key but upon trying to validate the config file I am now getting the "did not find expected key" error message. I've checked and the key is definitely in the correct location. Details below. --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 105, in parse_configuration config = load.load_configuration(config_filename) File "/usr/lib/python3.10/site-packages/borgmatic/config/load.py", line 20, in load_configuration return yaml.load(open(filename)) File "/usr/lib/python3.10/site-packages/ruamel/yaml/main.py", line 341, in load return constructor.get_single_data() File "/usr/lib/python3.10/site-packages/ruamel/yaml/constructor.py", line 111, in get_single_data node = self.composer.get_single_node() File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 706, in _ruamel_yaml.CParser.get_single_node File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 724, in _ruamel_yaml.CParser._compose_document File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 889, in _ruamel_yaml.CParser._compose_mapping_node File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 775, in _ruamel_yaml.CParser._compose_node File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 891, in _ruamel_yaml.CParser._compose_mapping_node File "ruamel.yaml.clib/_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event ruamel.yaml.parser.ParserError: while parsing a block mapping in "/etc/borgmatic/config.yaml", line 8, column 5 did not find expected key in "/etc/borgmatic/config.yaml", line 187, column 6 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/borgmatic/commands/validate_config.py", line 45, in main validate.parse_configuration(config_filename, validate.schema_filename()) File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 108, in parse_configuration raise Validation_error(config_filename, (str(error),)) borgmatic.config.validate.Validation_error: <unprintable Validation_error object> During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib64/python3.10/logging/__init__.py", line 1098, in emit msg = self.format(record) File "/usr/lib64/python3.10/logging/__init__.py", line 942, in format return fmt.format(record) File "/usr/lib64/python3.10/logging/__init__.py", line 678, in format record.message = record.getMessage() File "/usr/lib64/python3.10/logging/__init__.py", line 366, in getMessage msg = str(self.msg) File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 59, in __str__ ) + '\n'.join(format_error(error) for error in self.errors) File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 59, in <genexpr> ) + '\n'.join(format_error(error) for error in self.errors) File "/usr/lib/python3.10/site-packages/borgmatic/config/validate.py", line 32, in format_error if not error.path: AttributeError: 'str' object has no attribute 'path' Call stack: File "/usr/bin/validate-borgmatic-config", line 33, in <module> sys.exit(load_entry_point('borgmatic==1.5.21', 'console_scripts', 'validate-borgmatic-config')()) File "/usr/lib/python3.10/site-packages/borgmatic/commands/validate_config.py", line 48, in main logging.critical(error) Message: Validation_error('/etc/borgmatic/config.yaml', ('while parsing a block mapping\n in "/etc/borgmatic/config.yaml", line 8, column 5\ndid not find expected key\n in "/etc/borgmatic/config.yaml", line 187, column 6',)) Arguments: ()

Since it's failing to validate the YAML, If you uncommented ssh_command, did you remember to uncomment # storage: as well?

Apologies for the late response, I think part of the problem is that I am following 2-3 different guides. The guides that borgbackup provides, the guides that the cloud repo provider (BorgBase) provides and lastly the guides available in the borgmatic documentation. Below is how I edited the storage stanza in the .yaml file and got the error message ("ndid not find expected key") posted earlier. Encryption passphrase (redacted) is the password I chose when I initialized the repository using the command borg init -e repokey-blake2 @.repo.borgbase.com:repo

storage:
# The standard output of this command is used to unlock the
# encryption key. Only use on repositories that were
# initialized with passcommand/repokey/keyfile encryption.
# Note that if both encryption_passcommand and
# encryption_passphrase are set, then encryption_passphrase
# takes precedence. Defaults to not set.
# encryption_passcommand: secret-tool lookup borg-repository repo-name

# Passphrase to unlock the encryption key with. Only use on
# repositories that were initialized with
# passphrase/repokey/keyfile encryption. Quote the value if it
# contains punctuation, so it parses correctly. And backslash
# any quote or backslash literals as well. Defaults to not
# set.
 encryption_passphrase: *****************
 
 
 Below is the ssh_command stanza as well.
 
 # Command to use instead of "ssh". This can be used to specify
 # ssh options.  Defaults to not set.
  ssh_command: ssh -i ~/.ssh/privateborgkey

Would it be possible to see your entire configuration file (redacted)? That should allow us to reproduce the error and diagnose it. Thank you for your patience!

Aha! Just based on the snippet you posted, it looks like you've got an extra space before the ssh_command option that's confusing the parser. If you remove that space and make the option line up with encryption_passhprase above, that error should go away. (Although it should ideally be fixed so as not to produce a traceback.)

# Where to look for files to backup, and where to store those backups.
# See https://borgbackup.readthedocs.io/en/stable/quickstart.html and
# https://borgbackup.readthedocs.io/en/stable/usage/create.html
# for details.
location:
    # List of source directories to backup (required). Globs and
    # tildes are expanded. Do not backslash spaces in path names.
    source_directories:
        - /home
#        - /etc
#        - /var/log/syslog*
#        - /home/user/path with spaces

    # Paths to local or remote repositories (required). Tildes are
    # expanded. Multiple repositories are backed up to in
    # sequence. Borg placeholders can be used. See the output of
    # "borg help placeholders" for details. See ssh_command for
    # SSH options like identity file or port. If systemd service
    # is used, then add local repository paths in the systemd
    # service file to the ReadWritePaths list.
    repositories:
        - ********@********.repo.borgbase.com:repo
 #       - user@backupserver:{fqdn}

    # Stay in same file system (do not cross mount points).
    # Defaults to false. But when a database hook is used, the
    # setting here is ignored and one_file_system is considered
    # true.
    # one_file_system: true

    # Only store/extract numeric user and group identifiers.
    # Defaults to false.
    # numeric_owner: true

    # Store atime into archive. Defaults to true.
    # atime: false

    # Store ctime into archive. Defaults to true.
    # ctime: false

    # Store birthtime (creation date) into archive. Defaults to
    # true.
    # birthtime: false

    # Use Borg's --read-special flag to allow backup of block and
    # other special devices. Use with caution, as it will lead to
    # problems if used when backing up special devices such as
    # /dev/zero. Defaults to false. But when a database hook is
    # used, the setting here is ignored and read_special is
    # considered true.
    # read_special: false

    # Record bsdflags (e.g. NODUMP, IMMUTABLE) in archive.
    # Defaults to true.
    # bsd_flags: true

    # Mode in which to operate the files cache. See
    # http://borgbackup.readthedocs.io/en/stable/usage/create.html
    # for details. Defaults to "ctime,size,inode".
    # files_cache: ctime,size,inode

    # Alternate Borg local executable. Defaults to "borg".
    # local_path: borg1

    # Alternate Borg remote executable. Defaults to "borg".
    # remote_path: borg1

    # Any paths matching these patterns are included/excluded from
    # backups. Globs are expanded. (Tildes are not.) Note that
    # Borg considers this option experimental. See the output of
    # "borg help patterns" for more details. Quote any value if it
    # contains leading punctuation, so it parses correctly.
    # patterns:
        # - R /
        # - '- /home/*/.cache'
        # - + /home/susan
        # - '- /home/*'

    # Read include/exclude patterns from one or more separate
    # named files, one pattern per line. Note that Borg considers
    # this option experimental. See the output of "borg help
    # patterns" for more details.
    # patterns_from:
        # - /etc/borgmatic/patterns

    # Any paths matching these patterns are excluded from backups.
    # Globs and tildes are expanded. (Note however that a glob
    # pattern must either start with a glob or be an absolute
    # path.) Do not backslash spaces in path names. See the output
    # of "borg help patterns" for more details.
    # exclude_patterns:
        # - '*.pyc'
        # - /home/*/.cache
        # - '*/.vim*.tmp'
        # - /etc/ssl
        # - /home/user/path with spaces

    # Read exclude patterns from one or more separate named files,
    # one pattern per line. See the output of "borg help patterns"
    # for more details.
    # exclude_from:
        # - /etc/borgmatic/excludes

    # Exclude directories that contain a CACHEDIR.TAG file. See
    # http://www.brynosaurus.com/cachedir/spec.html for details.
    # Defaults to false.
    # exclude_caches: true

    # Exclude directories that contain a file with the given
    # filenames. Defaults to not set.
    # exclude_if_present:
        # - .nobackup

    # If true, the exclude_if_present filename is included in
    # backups. Defaults to false, meaning that the
    # exclude_if_present filename is omitted from backups.
    # keep_exclude_tags: true

    # Exclude files with the NODUMP flag. Defaults to false.
    # exclude_nodump: true

    # Path for additional source files used for temporary internal
    # state like borgmatic database dumps. Note that changing this
    # path prevents "borgmatic restore" from finding any database
    # dumps created before the change. Defaults to ~/.borgmatic
    # borgmatic_source_directory: /tmp/borgmatic

# Repository storage options. See
# https://borgbackup.readthedocs.io/en/stable/usage/create.html and
# https://borgbackup.readthedocs.io/en/stable/usage/general.html for
# details.
 storage:
    # The standard output of this command is used to unlock the
    # encryption key. Only use on repositories that were
    # initialized with passcommand/repokey/keyfile encryption.
    # Note that if both encryption_passcommand and
    # encryption_passphrase are set, then encryption_passphrase
    # takes precedence. Defaults to not set.
    # encryption_passcommand: secret-tool lookup borg-repository repo-name

    # Passphrase to unlock the encryption key with. Only use on
    # repositories that were initialized with
    # passphrase/repokey/keyfile encryption. Quote the value if it
    # contains punctuation, so it parses correctly. And backslash
    # any quote or backslash literals as well. Defaults to not
    # set.
    encryption_passphrase: *************

    # Number of seconds between each checkpoint during a
    # long-running backup. See
    # https://borgbackup.readthedocs.io/en/stable/faq.html
    # for details. Defaults to checkpoints every 1800 seconds (30
    # minutes).
    # checkpoint_interval: 1800

    # Specify the parameters passed to then chunker
    # (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS,
    # HASH_WINDOW_SIZE). See
    # https://borgbackup.readthedocs.io/en/stable/internals.html
    # for details. Defaults to "19,23,21,4095".
    # chunker_params: 19,23,21,4095

    # Type of compression to use when creating archives. See
    # http://borgbackup.readthedocs.io/en/stable/usage/create.html
    # for details. Defaults to "lz4".
    # compression: lz4

    # Remote network upload rate limit in kiBytes/second. Defaults
    # to unlimited.
    # remote_rate_limit: 100

    # Number of times to retry a failing backup before giving up.
    # Defaults to 0 (i.e., does not attempt retry).
    # retries: 3

    # Wait time between retries (in seconds) to allow transient
    # issues to pass. Increases after each retry as a form of
    # backoff. Defaults to 0 (no wait).
    # retry_wait: 10

    # Directory where temporary files are stored. Defaults to
    # $TMPDIR
    # temporary_directory: /path/to/tmpdir

    # Command to use instead of "ssh". This can be used to specify
    # ssh options.  Defaults to not set.
    ssh_command: ssh -i ~/.ssh/borgbackup

    # Base path used for various Borg directories. Defaults to
    # $HOME, ~$USER, or ~.
    # borg_base_directory: /path/to/base

    # Path for Borg configuration files. Defaults to
    # $borg_base_directory/.config/borg
    # borg_config_directory: /path/to/base/config

    # Path for Borg cache files. Defaults to
    # $borg_base_directory/.cache/borg
    # borg_cache_directory: /path/to/base/cache

    # Path for Borg security and encryption nonce files. Defaults
    # to $borg_base_directory/.config/borg/security
    # borg_security_directory: /path/to/base/config/security

    # Path for Borg encryption key files. Defaults to
    # $borg_base_directory/.config/borg/keys
    # borg_keys_directory: /path/to/base/config/keys

    # Umask to be used for borg create. Defaults to 0077.
    # umask: 0077

    # Maximum seconds to wait for acquiring a repository/cache
    # lock. Defaults to 1.
    # lock_wait: 5

    # Name of the archive. Borg placeholders can be used. See the
    # output of "borg help placeholders" for details. Defaults to
    # "{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this
    # option, you must also specify a prefix in the retention
    # section to avoid accidental pruning of archives with a
    # different archive name format. And you should also specify a
    # prefix in the consistency section as well.
    # archive_name_format: '{hostname}-documents-{now}'

    # Bypass Borg error about a repository that has been moved.
    # Defaults to false.
    # relocated_repo_access_is_ok: true

    # Bypass Borg error about a previously unknown unencrypted
    # repository. Defaults to false.
    # unknown_unencrypted_repo_access_is_ok: true

    # Additional options to pass directly to particular Borg
    # commands, handy for Borg options that borgmatic does not yet
    # support natively. Note that borgmatic does not perform any
    # validation on these options. Running borgmatic with
    # "--verbosity 2" shows the exact Borg command-line
    # invocation.
    # extra_borg_options:
        # Extra command-line options to pass to "borg init".
        # init: --make-parent-dirs

        # Extra command-line options to pass to "borg prune".
        # prune: --save-space

        # Extra command-line options to pass to "borg create".
        # create: --no-files-cache

        # Extra command-line options to pass to "borg check".
        # check: --save-space

# Retention policy for how many backups to keep in each category. See
# https://borgbackup.readthedocs.io/en/stable/usage/prune.html for
# details. At least one of the "keep" options is required for pruning
# to work. To skip pruning entirely, run "borgmatic create" or "check"
# without the "prune" action. See borgmatic documentation for details.
retention:
    # Keep all archives within this time interval.
    # keep_within: 3H

    # Number of secondly archives to keep.
    # keep_secondly: 60

    # Number of minutely archives to keep.
    # keep_minutely: 60

    # Number of hourly archives to keep.
    # keep_hourly: 24

    # Number of daily archives to keep.
    keep_daily: 7

    # Number of weekly archives to keep.
    # keep_weekly: 4

    # Number of monthly archives to keep.
    # keep_monthly: 6

    # Number of yearly archives to keep.
    # keep_yearly: 1

    # When pruning, only consider archive names starting with this
    # prefix.  Borg placeholders can be used. See the output of
    # "borg help placeholders" for details. Defaults to
    # "{hostname}-". Use an empty value to disable the default.
    # prefix: sourcehostname

# Consistency checks to run after backups. See
# https://borgbackup.readthedocs.io/en/stable/usage/check.html and
# https://borgbackup.readthedocs.io/en/stable/usage/extract.html for
# details.
# consistency:
    # List of one or more consistency checks to run: "repository",
    # "archives", "data", and/or "extract". Defaults to
    # "repository" and "archives". Set to "disabled" to disable
    # all consistency checks. "repository" checks the consistency
    # of the repository, "archives" checks all of the archives,
    # "data" verifies the integrity of the data within the
    # archives, and "extract" does an extraction dry-run of the
    # most recent archive. Note that "data" implies "archives".
    # checks:
        # - repository
        # - archives

    # Paths to a subset of the repositories in the location
    # section on which to run consistency checks. Handy in case
    # some of your repositories are very large, and so running
    # consistency checks on them would take too long. Defaults to
    # running consistency checks on all repositories configured in
    # the location section.
    # check_repositories:
        # - user@backupserver:sourcehostname.borg

    # Restrict the number of checked archives to the last n.
    # Applies only to the "archives" check. Defaults to checking
    # all archives.
    # check_last: 3

    # When performing the "archives" check, only consider archive
    # names starting with this prefix. Borg placeholders can be
    # used. See the output of "borg help placeholders" for
    # details. Defaults to "{hostname}-". Use an empty value to
    # disable the default.
    # prefix: sourcehostname

# Options for customizing borgmatic's own output and logging.
# output:
    # Apply color to console output. Can be overridden with
    # --no-color command-line flag. Defaults to true.
    # color: false

# Shell commands, scripts, or integrations to execute at various
# points during a borgmatic run. IMPORTANT: All provided commands and
# scripts are executed with user permissions of borgmatic. Do not
# forget to set secure permissions on this configuration file (chmod
# 0600) as well as on any script called from a hook (chmod 0700) to
# prevent potential shell injection or privilege escalation.
# hooks:
    # List of one or more shell commands or scripts to execute
    # before creating a backup, run once per configuration file.
    # before_backup:
        # - echo "Starting a backup."

    # List of one or more shell commands or scripts to execute
    # before pruning, run once per configuration file.
    # before_prune:
        # - echo "Starting pruning."

    # List of one or more shell commands or scripts to execute
    # before consistency checks, run once per configuration file.
    # before_check:
        # - echo "Starting checks."

    # List of one or more shell commands or scripts to execute
    # before extracting a backup, run once per configuration file.
    # before_extract:
        # - echo "Starting extracting."

    # List of one or more shell commands or scripts to execute
    # after creating a backup, run once per configuration file.
    # after_backup:
        # - echo "Finished a backup."

    # List of one or more shell commands or scripts to execute
    # after pruning, run once per configuration file.
    # after_prune:
        # - echo "Finished pruning."

    # List of one or more shell commands or scripts to execute
    # after consistency checks, run once per configuration file.
    # after_check:
        # - echo "Finished checks."

    # List of one or more shell commands or scripts to execute
    # after extracting a backup, run once per configuration file.
    # after_extract:
        # - echo "Finished extracting."

    # List of one or more shell commands or scripts to execute
    # when an exception occurs during a "prune", "create", or
    # "check" action or an associated before/after hook.
    # on_error:
        # - echo "Error during prune/create/check."

    # List of one or more shell commands or scripts to execute
    # before running all actions (if one of them is "create").
    # These are collected from all configuration files and then
    # run once before all of them (prior to all actions).
    # before_everything:
        # - echo "Starting actions."

    # List of one or more shell commands or scripts to execute
    # after running all actions (if one of them is "create").
    # These are collected from all configuration files and then
    # run once after all of them (after any action).
    # after_everything:
        # - echo "Completed actions."

    # List of one or more PostgreSQL databases to dump before
    # creating a backup, run once per configuration file. The
    # database dumps are added to your source directories at
    # runtime, backed up, and removed afterwards. Requires
    # pg_dump/pg_dumpall/pg_restore commands. See
    # https://www.postgresql.org/docs/current/app-pgdump.html and
    # https://www.postgresql.org/docs/current/libpq-ssl.html for
    # details.
    # postgresql_databases:
        # Database name (required if using this hook). Or
        # "all" to dump all databases on the host. Note
        # that using this database hook implicitly enables
        # both read_special and one_file_system (see
        # above) to support dump and restore streaming.
        # - name: users

          # Database hostname to connect to. Defaults to
          # connecting via local Unix socket.
          # hostname: database.example.org

          # Port to connect to. Defaults to 5432.
          # port: 5433

          # Username with which to connect to the database.
          # Defaults to the username of the current user.
          # You probably want to specify the "postgres"
          # superuser here when the database name is "all".
          # username: dbuser

          # Password with which to connect to the database.
          # Omitting a password will only work if PostgreSQL
          # is configured to trust the configured username
          # without a password or you create a ~/.pgpass
          # file.
          # password: trustsome1

          # Database dump output format. One of "plain",
          # "custom", "directory", or "tar". Defaults to
          # "custom" (unlike raw pg_dump). See pg_dump
          # documentation for details. Note that format is
          # ignored when the database name is "all".
          # format: directory

          # SSL mode to use to connect to the database
          # server. One of "disable", "allow", "prefer",
          # "require", "verify-ca" or "verify-full".
          # Defaults to "disable".
          # ssl_mode: require

          # Path to a client certificate.
          # ssl_cert: /root/.postgresql/postgresql.crt

          # Path to a private client key.
          # ssl_key: /root/.postgresql/postgresql.key

          # Path to a root certificate containing a list of
          # trusted certificate authorities.
          # ssl_root_cert: /root/.postgresql/root.crt

          # Path to a certificate revocation list.
          # ssl_crl: /root/.postgresql/root.crl

          # Additional pg_dump/pg_dumpall options to pass
          # directly to the dump command, without performing
          # any validation on them. See pg_dump
          # documentation for details.
          # options: --role=someone

    # List of one or more MySQL/MariaDB databases to dump before
    # creating a backup, run once per configuration file. The
    # database dumps are added to your source directories at
    # runtime, backed up, and removed afterwards. Requires
    # mysqldump/mysql commands (from either MySQL or MariaDB). See
    # https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html or
    # https://mariadb.com/kb/en/library/mysqldump/ for details.
    # mysql_databases:
        # Database name (required if using this hook). Or
        # "all" to dump all databases on the host. Note
        # that using this database hook implicitly enables
        # both read_special and one_file_system (see
        # above) to support dump and restore streaming.
        # - name: users

          # Database hostname to connect to. Defaults to
          # connecting via local Unix socket.
          # hostname: database.example.org

          # Port to connect to. Defaults to 3306.
          # port: 3307

          # Username with which to connect to the database.
          # Defaults to the username of the current user.
          # username: dbuser

          # Password with which to connect to the database.
          # Omitting a password will only work if MySQL is
          # configured to trust the configured username
          # without a password.
          # password: trustsome1

          # Additional mysql options to pass directly to
          # the mysql command that lists available
          # databases, without performing any validation on
          # them. See mysql documentation for details.
          # list_options: --defaults-extra-file=my.cnf

          # Additional mysqldump options to pass directly to
          # the dump command, without performing any
          # validation on them. See mysqldump documentation
          # for details.
          # options: --skip-comments

    # Healthchecks ping URL or UUID to notify when a backup
    # begins, ends, or errors. Create an account at
    # https://healthchecks.io if you'd like to use this service.
    # See borgmatic monitoring documentation for details.
    # healthchecks: https://hc-ping.com/your-uuid-here

    # Cronitor ping URL to notify when a backup begins, ends, or
    # errors. Create an account at https://cronitor.io if you'd
    # like to use this service. See borgmatic monitoring
    # documentation for details.
    # cronitor: https://cronitor.link/d3x0c1

    # PagerDuty integration key used to notify PagerDuty when a
    # backup errors. Create an account at
    # https://www.pagerduty.com/ if you'd like to use this
    # service. See borgmatic monitoring documentation for details.
    # pagerduty: a177cad45bd374409f78906a810a3074

    # Cronhub ping URL to notify when a backup begins, ends, or
    # errors. Create an account at https://cronhub.io if you'd
    # like to use this service. See borgmatic monitoring
    # documentation for details.
    # cronhub: https://cronhub.io/start/1f5e3410-254c-11e8-b61d-55875966d01

    # Umask used when executing hooks. Defaults to the umask that
    # borgmatic is run with.
    # umask: 0077

``` # Where to look for files to backup, and where to store those backups. # See https://borgbackup.readthedocs.io/en/stable/quickstart.html and # https://borgbackup.readthedocs.io/en/stable/usage/create.html # for details. location: # List of source directories to backup (required). Globs and # tildes are expanded. Do not backslash spaces in path names. source_directories: - /home # - /etc # - /var/log/syslog* # - /home/user/path with spaces # Paths to local or remote repositories (required). Tildes are # expanded. Multiple repositories are backed up to in # sequence. Borg placeholders can be used. See the output of # "borg help placeholders" for details. See ssh_command for # SSH options like identity file or port. If systemd service # is used, then add local repository paths in the systemd # service file to the ReadWritePaths list. repositories: - ********@********.repo.borgbase.com:repo # - user@backupserver:{fqdn} # Stay in same file system (do not cross mount points). # Defaults to false. But when a database hook is used, the # setting here is ignored and one_file_system is considered # true. # one_file_system: true # Only store/extract numeric user and group identifiers. # Defaults to false. # numeric_owner: true # Store atime into archive. Defaults to true. # atime: false # Store ctime into archive. Defaults to true. # ctime: false # Store birthtime (creation date) into archive. Defaults to # true. # birthtime: false # Use Borg's --read-special flag to allow backup of block and # other special devices. Use with caution, as it will lead to # problems if used when backing up special devices such as # /dev/zero. Defaults to false. But when a database hook is # used, the setting here is ignored and read_special is # considered true. # read_special: false # Record bsdflags (e.g. NODUMP, IMMUTABLE) in archive. # Defaults to true. # bsd_flags: true # Mode in which to operate the files cache. See # http://borgbackup.readthedocs.io/en/stable/usage/create.html # for details. Defaults to "ctime,size,inode". # files_cache: ctime,size,inode # Alternate Borg local executable. Defaults to "borg". # local_path: borg1 # Alternate Borg remote executable. Defaults to "borg". # remote_path: borg1 # Any paths matching these patterns are included/excluded from # backups. Globs are expanded. (Tildes are not.) Note that # Borg considers this option experimental. See the output of # "borg help patterns" for more details. Quote any value if it # contains leading punctuation, so it parses correctly. # patterns: # - R / # - '- /home/*/.cache' # - + /home/susan # - '- /home/*' # Read include/exclude patterns from one or more separate # named files, one pattern per line. Note that Borg considers # this option experimental. See the output of "borg help # patterns" for more details. # patterns_from: # - /etc/borgmatic/patterns # Any paths matching these patterns are excluded from backups. # Globs and tildes are expanded. (Note however that a glob # pattern must either start with a glob or be an absolute # path.) Do not backslash spaces in path names. See the output # of "borg help patterns" for more details. # exclude_patterns: # - '*.pyc' # - /home/*/.cache # - '*/.vim*.tmp' # - /etc/ssl # - /home/user/path with spaces # Read exclude patterns from one or more separate named files, # one pattern per line. See the output of "borg help patterns" # for more details. # exclude_from: # - /etc/borgmatic/excludes # Exclude directories that contain a CACHEDIR.TAG file. See # http://www.brynosaurus.com/cachedir/spec.html for details. # Defaults to false. # exclude_caches: true # Exclude directories that contain a file with the given # filenames. Defaults to not set. # exclude_if_present: # - .nobackup # If true, the exclude_if_present filename is included in # backups. Defaults to false, meaning that the # exclude_if_present filename is omitted from backups. # keep_exclude_tags: true # Exclude files with the NODUMP flag. Defaults to false. # exclude_nodump: true # Path for additional source files used for temporary internal # state like borgmatic database dumps. Note that changing this # path prevents "borgmatic restore" from finding any database # dumps created before the change. Defaults to ~/.borgmatic # borgmatic_source_directory: /tmp/borgmatic # Repository storage options. See # https://borgbackup.readthedocs.io/en/stable/usage/create.html and # https://borgbackup.readthedocs.io/en/stable/usage/general.html for # details. storage: # The standard output of this command is used to unlock the # encryption key. Only use on repositories that were # initialized with passcommand/repokey/keyfile encryption. # Note that if both encryption_passcommand and # encryption_passphrase are set, then encryption_passphrase # takes precedence. Defaults to not set. # encryption_passcommand: secret-tool lookup borg-repository repo-name # Passphrase to unlock the encryption key with. Only use on # repositories that were initialized with # passphrase/repokey/keyfile encryption. Quote the value if it # contains punctuation, so it parses correctly. And backslash # any quote or backslash literals as well. Defaults to not # set. encryption_passphrase: ************* # Number of seconds between each checkpoint during a # long-running backup. See # https://borgbackup.readthedocs.io/en/stable/faq.html # for details. Defaults to checkpoints every 1800 seconds (30 # minutes). # checkpoint_interval: 1800 # Specify the parameters passed to then chunker # (CHUNK_MIN_EXP, CHUNK_MAX_EXP, HASH_MASK_BITS, # HASH_WINDOW_SIZE). See # https://borgbackup.readthedocs.io/en/stable/internals.html # for details. Defaults to "19,23,21,4095". # chunker_params: 19,23,21,4095 # Type of compression to use when creating archives. See # http://borgbackup.readthedocs.io/en/stable/usage/create.html # for details. Defaults to "lz4". # compression: lz4 # Remote network upload rate limit in kiBytes/second. Defaults # to unlimited. # remote_rate_limit: 100 # Number of times to retry a failing backup before giving up. # Defaults to 0 (i.e., does not attempt retry). # retries: 3 # Wait time between retries (in seconds) to allow transient # issues to pass. Increases after each retry as a form of # backoff. Defaults to 0 (no wait). # retry_wait: 10 # Directory where temporary files are stored. Defaults to # $TMPDIR # temporary_directory: /path/to/tmpdir # Command to use instead of "ssh". This can be used to specify # ssh options. Defaults to not set. ssh_command: ssh -i ~/.ssh/borgbackup # Base path used for various Borg directories. Defaults to # $HOME, ~$USER, or ~. # borg_base_directory: /path/to/base # Path for Borg configuration files. Defaults to # $borg_base_directory/.config/borg # borg_config_directory: /path/to/base/config # Path for Borg cache files. Defaults to # $borg_base_directory/.cache/borg # borg_cache_directory: /path/to/base/cache # Path for Borg security and encryption nonce files. Defaults # to $borg_base_directory/.config/borg/security # borg_security_directory: /path/to/base/config/security # Path for Borg encryption key files. Defaults to # $borg_base_directory/.config/borg/keys # borg_keys_directory: /path/to/base/config/keys # Umask to be used for borg create. Defaults to 0077. # umask: 0077 # Maximum seconds to wait for acquiring a repository/cache # lock. Defaults to 1. # lock_wait: 5 # Name of the archive. Borg placeholders can be used. See the # output of "borg help placeholders" for details. Defaults to # "{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}". If you specify this # option, you must also specify a prefix in the retention # section to avoid accidental pruning of archives with a # different archive name format. And you should also specify a # prefix in the consistency section as well. # archive_name_format: '{hostname}-documents-{now}' # Bypass Borg error about a repository that has been moved. # Defaults to false. # relocated_repo_access_is_ok: true # Bypass Borg error about a previously unknown unencrypted # repository. Defaults to false. # unknown_unencrypted_repo_access_is_ok: true # Additional options to pass directly to particular Borg # commands, handy for Borg options that borgmatic does not yet # support natively. Note that borgmatic does not perform any # validation on these options. Running borgmatic with # "--verbosity 2" shows the exact Borg command-line # invocation. # extra_borg_options: # Extra command-line options to pass to "borg init". # init: --make-parent-dirs # Extra command-line options to pass to "borg prune". # prune: --save-space # Extra command-line options to pass to "borg create". # create: --no-files-cache # Extra command-line options to pass to "borg check". # check: --save-space # Retention policy for how many backups to keep in each category. See # https://borgbackup.readthedocs.io/en/stable/usage/prune.html for # details. At least one of the "keep" options is required for pruning # to work. To skip pruning entirely, run "borgmatic create" or "check" # without the "prune" action. See borgmatic documentation for details. retention: # Keep all archives within this time interval. # keep_within: 3H # Number of secondly archives to keep. # keep_secondly: 60 # Number of minutely archives to keep. # keep_minutely: 60 # Number of hourly archives to keep. # keep_hourly: 24 # Number of daily archives to keep. keep_daily: 7 # Number of weekly archives to keep. # keep_weekly: 4 # Number of monthly archives to keep. # keep_monthly: 6 # Number of yearly archives to keep. # keep_yearly: 1 # When pruning, only consider archive names starting with this # prefix. Borg placeholders can be used. See the output of # "borg help placeholders" for details. Defaults to # "{hostname}-". Use an empty value to disable the default. # prefix: sourcehostname # Consistency checks to run after backups. See # https://borgbackup.readthedocs.io/en/stable/usage/check.html and # https://borgbackup.readthedocs.io/en/stable/usage/extract.html for # details. # consistency: # List of one or more consistency checks to run: "repository", # "archives", "data", and/or "extract". Defaults to # "repository" and "archives". Set to "disabled" to disable # all consistency checks. "repository" checks the consistency # of the repository, "archives" checks all of the archives, # "data" verifies the integrity of the data within the # archives, and "extract" does an extraction dry-run of the # most recent archive. Note that "data" implies "archives". # checks: # - repository # - archives # Paths to a subset of the repositories in the location # section on which to run consistency checks. Handy in case # some of your repositories are very large, and so running # consistency checks on them would take too long. Defaults to # running consistency checks on all repositories configured in # the location section. # check_repositories: # - user@backupserver:sourcehostname.borg # Restrict the number of checked archives to the last n. # Applies only to the "archives" check. Defaults to checking # all archives. # check_last: 3 # When performing the "archives" check, only consider archive # names starting with this prefix. Borg placeholders can be # used. See the output of "borg help placeholders" for # details. Defaults to "{hostname}-". Use an empty value to # disable the default. # prefix: sourcehostname # Options for customizing borgmatic's own output and logging. # output: # Apply color to console output. Can be overridden with # --no-color command-line flag. Defaults to true. # color: false # Shell commands, scripts, or integrations to execute at various # points during a borgmatic run. IMPORTANT: All provided commands and # scripts are executed with user permissions of borgmatic. Do not # forget to set secure permissions on this configuration file (chmod # 0600) as well as on any script called from a hook (chmod 0700) to # prevent potential shell injection or privilege escalation. # hooks: # List of one or more shell commands or scripts to execute # before creating a backup, run once per configuration file. # before_backup: # - echo "Starting a backup." # List of one or more shell commands or scripts to execute # before pruning, run once per configuration file. # before_prune: # - echo "Starting pruning." # List of one or more shell commands or scripts to execute # before consistency checks, run once per configuration file. # before_check: # - echo "Starting checks." # List of one or more shell commands or scripts to execute # before extracting a backup, run once per configuration file. # before_extract: # - echo "Starting extracting." # List of one or more shell commands or scripts to execute # after creating a backup, run once per configuration file. # after_backup: # - echo "Finished a backup." # List of one or more shell commands or scripts to execute # after pruning, run once per configuration file. # after_prune: # - echo "Finished pruning." # List of one or more shell commands or scripts to execute # after consistency checks, run once per configuration file. # after_check: # - echo "Finished checks." # List of one or more shell commands or scripts to execute # after extracting a backup, run once per configuration file. # after_extract: # - echo "Finished extracting." # List of one or more shell commands or scripts to execute # when an exception occurs during a "prune", "create", or # "check" action or an associated before/after hook. # on_error: # - echo "Error during prune/create/check." # List of one or more shell commands or scripts to execute # before running all actions (if one of them is "create"). # These are collected from all configuration files and then # run once before all of them (prior to all actions). # before_everything: # - echo "Starting actions." # List of one or more shell commands or scripts to execute # after running all actions (if one of them is "create"). # These are collected from all configuration files and then # run once after all of them (after any action). # after_everything: # - echo "Completed actions." # List of one or more PostgreSQL databases to dump before # creating a backup, run once per configuration file. The # database dumps are added to your source directories at # runtime, backed up, and removed afterwards. Requires # pg_dump/pg_dumpall/pg_restore commands. See # https://www.postgresql.org/docs/current/app-pgdump.html and # https://www.postgresql.org/docs/current/libpq-ssl.html for # details. # postgresql_databases: # Database name (required if using this hook). Or # "all" to dump all databases on the host. Note # that using this database hook implicitly enables # both read_special and one_file_system (see # above) to support dump and restore streaming. # - name: users # Database hostname to connect to. Defaults to # connecting via local Unix socket. # hostname: database.example.org # Port to connect to. Defaults to 5432. # port: 5433 # Username with which to connect to the database. # Defaults to the username of the current user. # You probably want to specify the "postgres" # superuser here when the database name is "all". # username: dbuser # Password with which to connect to the database. # Omitting a password will only work if PostgreSQL # is configured to trust the configured username # without a password or you create a ~/.pgpass # file. # password: trustsome1 # Database dump output format. One of "plain", # "custom", "directory", or "tar". Defaults to # "custom" (unlike raw pg_dump). See pg_dump # documentation for details. Note that format is # ignored when the database name is "all". # format: directory # SSL mode to use to connect to the database # server. One of "disable", "allow", "prefer", # "require", "verify-ca" or "verify-full". # Defaults to "disable". # ssl_mode: require # Path to a client certificate. # ssl_cert: /root/.postgresql/postgresql.crt # Path to a private client key. # ssl_key: /root/.postgresql/postgresql.key # Path to a root certificate containing a list of # trusted certificate authorities. # ssl_root_cert: /root/.postgresql/root.crt # Path to a certificate revocation list. # ssl_crl: /root/.postgresql/root.crl # Additional pg_dump/pg_dumpall options to pass # directly to the dump command, without performing # any validation on them. See pg_dump # documentation for details. # options: --role=someone # List of one or more MySQL/MariaDB databases to dump before # creating a backup, run once per configuration file. The # database dumps are added to your source directories at # runtime, backed up, and removed afterwards. Requires # mysqldump/mysql commands (from either MySQL or MariaDB). See # https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html or # https://mariadb.com/kb/en/library/mysqldump/ for details. # mysql_databases: # Database name (required if using this hook). Or # "all" to dump all databases on the host. Note # that using this database hook implicitly enables # both read_special and one_file_system (see # above) to support dump and restore streaming. # - name: users # Database hostname to connect to. Defaults to # connecting via local Unix socket. # hostname: database.example.org # Port to connect to. Defaults to 3306. # port: 3307 # Username with which to connect to the database. # Defaults to the username of the current user. # username: dbuser # Password with which to connect to the database. # Omitting a password will only work if MySQL is # configured to trust the configured username # without a password. # password: trustsome1 # Additional mysql options to pass directly to # the mysql command that lists available # databases, without performing any validation on # them. See mysql documentation for details. # list_options: --defaults-extra-file=my.cnf # Additional mysqldump options to pass directly to # the dump command, without performing any # validation on them. See mysqldump documentation # for details. # options: --skip-comments # Healthchecks ping URL or UUID to notify when a backup # begins, ends, or errors. Create an account at # https://healthchecks.io if you'd like to use this service. # See borgmatic monitoring documentation for details. # healthchecks: https://hc-ping.com/your-uuid-here # Cronitor ping URL to notify when a backup begins, ends, or # errors. Create an account at https://cronitor.io if you'd # like to use this service. See borgmatic monitoring # documentation for details. # cronitor: https://cronitor.link/d3x0c1 # PagerDuty integration key used to notify PagerDuty when a # backup errors. Create an account at # https://www.pagerduty.com/ if you'd like to use this # service. See borgmatic monitoring documentation for details. # pagerduty: a177cad45bd374409f78906a810a3074 # Cronhub ping URL to notify when a backup begins, ends, or # errors. Create an account at https://cronhub.io if you'd # like to use this service. See borgmatic monitoring # documentation for details. # cronhub: https://cronhub.io/start/1f5e3410-254c-11e8-b61d-55875966d01 # Umask used when executing hooks. Defaults to the umask that # borgmatic is run with. # umask: 0077 ```

As shown above I've lined both (encryption_passphrase and ssh_command_ up so that they're both on column 5 of their respective lines but still getting the same error. There is a space between the password string itself and the colon preceding it but I presume the formatting is correct there.

Looks like the problem is the space before the storage: key!

Thank you so much!, removing that space seemed to have done the trick, the config file now passes the validation check, however, once I try creating a backup, I get the following error.

$ sudo validate-borgmatic-config
All given configuration files are valid: /etc/borgmatic/config.yaml
$ borgmatic create
/etc/borgmatic/config.yaml /etc/borgmatic.d /home/user/.config/borgmatic/config.yaml /home/user/.config/borgmatic.d: No valid configuration files found

summary:
/etc/borgmatic/config.yaml /etc/borgmatic.d /home/user/.config/borgmatic/config.yaml /home/user/.config/borgmatic.d: No valid configuration files found

Need some help? https://torsion.org/borgmatic/#issues

Super weird! I assume the configuration file is indeed at /etc/borgmatic/config.yaml? Any clues if you run borgmatic create with --verbosity 2?

$ borgmatic create --verbosity 2
Ensuring legacy configuration is upgraded
/etc/borgmatic/config.yaml /etc/borgmatic.d /home/user/.config/borgmatic/config.yaml /home/user/.config/borgmatic.d: No valid configuration files found

summary:
/etc/borgmatic/config.yaml /etc/borgmatic.d /home/user/.config/borgmatic/config.yaml /home/user/.config/borgmatic.d: No valid configuration files found

Need some help? https://torsion.org/borgmatic/#issues

Installed borgmatic through Fedora (dnf).

Is it possible you don't have permissions to the configuration file? Have you tried running borgmatic create as root?

Yes, I've tried that, not sure which password to use there (I only have the encryption password I set when I initialized the repository and that doesnt work, do not have the ssh password for the borgbase server.

$ sudo borgmatic create
[sudo] password for user:
The authenticity of host '.repo.borgbase.com (...**)' can't be established.
ED25519 key fingerprint is SHA256:*******************************************.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:1: ********.repo.borgbase.com
~/.ssh/known_hosts:2: ********.repo.borgbase.com
~/.ssh/known_hosts:3: ********.repo.borgbase.com
~/.ssh/known_hosts:4: ********.repo.borgbase.com
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
@.repo.borgbase.com's password:
@.repo.borgbase.com's password:

Looks like it's working! Except you'll need to set up SSH keys so that it can connect to BorgBase. I believe these are the steps for that process: https://docs.borgbase.com/setup/cli/#step-3-create-and-assign-ssh-key-for-authentication

Thanks again for your patience and for dealing with any unintended headaches, this is what I had done (through the borgbase website gui) but no luck there. Do I need to run ssh-keygen -o -a 100 -t ed25519 with sudo privileges? because I usually run it as documented on borgbase. I'll try again and report back.

Thanks for your patience as well! Yes, if you're connecting to BorgBase via borgmatic as root, then your SSH key will need to be generated as root. And then that's the key you'll need to give to BorgBase in their web UI.

Feel like I'm getting closer. I generated an ssh key as root and replaced the old key with the new one (through the borgbase gui). Didnt create a new repository just replaced the keys. I then edited the config file so that ssh_command now points to the new private keyfile as shown below

ssh_command: ssh -i /root/.ssh/id_ed25519

I then proceeded to run sudo borgmatic create and got the following error message

@.repo.borgbase.com:repo: Error running actions for repository
Command 'borg create @.repo.borgbase.com:repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /home' returned non-zero exit status 2.
/etc/borgmatic/config.yaml: Error running configuration file

summary:
/etc/borgmatic/config.yaml: Error running configuration file
@.repo.borgbase.com:repo: Error running actions for repository
Failed to create/acquire the lock /srv/repos/********/repo/lock.exclusive (timeout).
Command 'borg create @.repo.borgbase.com:repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /home' returned non-zero exit status 2.

Need some help? https://torsion.org/borgmatic/#issues

Sounds like there might be a stale lock on BorgBase's side? You could try borg break-lock username@hostname.repo.borgbase.com with your actual repository name.

ran sudo borg break-lock @.repo.borgbase.com:repo

then ran sudo borgmatic create but its been running for a couple of minutes without any logging or results.

Sounds promising! Next time, you can include the --files flag if you want to see file-by-file output.

Things are looking really good! backup was completed successfully, really appreciate the help. Silly mistake on my part. I'll try to automate with a systemd timer and see how it goes from there.

Awesome, so glad to hear it!

Apologies for the late update! Automated backups using systemd timers are working flawlessly /etc/systemd/system/now that I've solved the permissions issues on the ssh keypair generation side of things that I had earlier overlooked.

One thing to bear in mind for those of you who've installed borgmatic using package managers like I did (dnf on fedora)is that your preinstalled .service and .timer files may be found in the /usr/lib/systemd/system/ (rather than /etc/systemd/system/) directory.