borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
9
74
Fork 55
Code Issues 52 Pull requests 4 Releases 159 Packages 1 Activity Actions 1

Credentials via systemd don't work #1271

New issue
Closed
opened 2026-02-22 10:36:59 +00:00 by muemo · 5 comments
muemo commented 2026-02-22 10:36:59 +00:00
Copy link

What I'm trying to do and why

I have been trying to follow the documentation in setting up encrypted credentials via systemd.

Steps to reproduce

I used systemd-ask-password -n | systemd-creds encrypt - /etc/credstore.encrypted/borgmatic.pw to first encrypt the secret, added encryption_passphrase: "{credential systemd borgmatic.pw}" to my borgmatic config file, and debian set LoadCredentialEncrypted=borgmatic.pw in the borgmatic service file by default.
Then, running sudo borgmatic or systemctl start borgmatic.service results in the outputs below in Actual behavior.

Actual behavior

No commands to run for pre-everything hook
/etc/borgmatic/config.yaml: Skipping prune/compact actions due to configured skip_actions
/etc/borgmatic/config.yaml: Calling systemd hook function load_credential
/etc/borgmatic/config.yaml: /etc/borgmatic/config.yaml: Error getting local Borg version
/etc/borgmatic/config.yaml: Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set
/etc/borgmatic/config.yaml: An error occurred
No commands to run for post-everything hook

summary:
/etc/borgmatic/config.yaml: Loading configuration file
An error occurred
/etc/borgmatic/config.yaml: Error getting local Borg version
Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set

Need some help? https://torsion.org/borgmatic/#issues

Feb 22 11:29:03 redacted systemd[1]: Starting borgmatic.service - borgmatic backup...
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: /etc/borgmatic/config.yaml: Error getting local Borg version
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: Cannot load invalid credential name "borgmatic.pw"
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: An error occurred
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL summary:
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL An error occurred
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: Error getting local Borg version
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Cannot load invalid credential name "borgmatic.pw"
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL
Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues
Feb 22 11:30:05 redacted systemd-inhibit[3613]: /usr/bin/borgmatic failed with exit status 1.
Feb 22 11:30:05 redacted systemd[1]: borgmatic.service: Main process exited, code=exited, status=1/FAILURE
Feb 22 11:30:05 redacted systemd[1]: borgmatic.service: Failed with result 'exit-code'.
Feb 22 11:30:05 redacted systemd[1]: Failed to start borgmatic.service - borgmatic backup.

Expected behavior

Borgmatic decrypts the credentials and uses them.

Other notes / implementation ideas

I have tried running

systemd-run --pipe --wait --property=LoadCredentialEncrypted=borgmatic.pw systemd-creds cat borgmatic.pw

and it correctly shows the decrypted secret.
Also, replacing the encryption_passphrase with just the passphrase in plain text makes everything work.

borgmatic version

1.9.14

borgmatic installation method

Debian package

Borg version

borg 1.4.0

Python version

Python 3.13.5

Database version (if applicable)

No response

Operating system and version

PRETTY_NAME="Debian GNU/Linux 13 (trixie)" NAME="Debian GNU/Linux" VERSION_ID="13" VERSION="13 (trixie)" VERSION_CODENAME=trixie DEBIAN_VERSION_FULL=13.3 ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

### What I'm trying to do and why I have been trying to follow [the documentation](https://torsion.org/borgmatic/reference/configuration/credentials/systemd/) in setting up encrypted credentials via systemd. ### Steps to reproduce I used `systemd-ask-password -n | systemd-creds encrypt - /etc/credstore.encrypted/borgmatic.pw` to first encrypt the secret, added `encryption_passphrase: "{credential systemd borgmatic.pw}"` to my borgmatic config file, and debian set `LoadCredentialEncrypted=borgmatic.pw` in the borgmatic service file by default. Then, running `sudo borgmatic` or `systemctl start borgmatic.service` results in the outputs below in Actual behavior. ### Actual behavior ``` No commands to run for pre-everything hook /etc/borgmatic/config.yaml: Skipping prune/compact actions due to configured skip_actions /etc/borgmatic/config.yaml: Calling systemd hook function load_credential /etc/borgmatic/config.yaml: /etc/borgmatic/config.yaml: Error getting local Borg version /etc/borgmatic/config.yaml: Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set /etc/borgmatic/config.yaml: An error occurred No commands to run for post-everything hook summary: /etc/borgmatic/config.yaml: Loading configuration file An error occurred /etc/borgmatic/config.yaml: Error getting local Borg version Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set Need some help? https://torsion.org/borgmatic/#issues ``` --- ``` Feb 22 11:29:03 redacted systemd[1]: Starting borgmatic.service - borgmatic backup... Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: /etc/borgmatic/config.yaml: Error getting local Borg version Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: Cannot load invalid credential name "borgmatic.pw" Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: An error occurred Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL summary: Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL An error occurred Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL /etc/borgmatic/config.yaml: Error getting local Borg version Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Cannot load invalid credential name "borgmatic.pw" Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Feb 22 11:30:05 redacted borgmatic[3617]: CRITICAL Need some help? https://torsion.org/borgmatic/#issues Feb 22 11:30:05 redacted systemd-inhibit[3613]: /usr/bin/borgmatic failed with exit status 1. Feb 22 11:30:05 redacted systemd[1]: borgmatic.service: Main process exited, code=exited, status=1/FAILURE Feb 22 11:30:05 redacted systemd[1]: borgmatic.service: Failed with result 'exit-code'. Feb 22 11:30:05 redacted systemd[1]: Failed to start borgmatic.service - borgmatic backup. ``` ### Expected behavior Borgmatic decrypts the credentials and uses them. ### Other notes / implementation ideas I have tried running ``` systemd-run --pipe --wait --property=LoadCredentialEncrypted=borgmatic.pw systemd-creds cat borgmatic.pw ``` and it correctly shows the decrypted secret. Also, replacing the `encryption_passphrase` with just the passphrase in plain text makes everything work. ### borgmatic version 1.9.14 ### borgmatic installation method Debian package ### Borg version borg 1.4.0 ### Python version Python 3.13.5 ### Database version (if applicable) _No response_ ### Operating system and version PRETTY_NAME="Debian GNU/Linux 13 (trixie)" NAME="Debian GNU/Linux" VERSION_ID="13" VERSION="13 (trixie)" VERSION_CODENAME=trixie DEBIAN_VERSION_FULL=13.3 ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
witten commented 2026-02-22 18:31:21 +00:00
Owner
Copy link

A few thoughts here:

  • Error getting local Borg version: This suggest that borgmatic could not find Borg. Is borg's parent directory in the system PATH, both at the command-line and when running under systemd? If not, you can add it to the PATH or just set a local_path option in borgmatic.
  • Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set: Can I get a look at your borgmatic systemd service? Are you getting this error even when running borgmatic from systemd, or only when run manually from the command-line? If you want to run borgmatic manually from the command-line even when systemd credentials are configured, you'll need to upgrade to a newer version of borgmatic. (See the docs about this. You'll need 2.0.9+.)
A few thoughts here: * `Error getting local Borg version`: This suggest that borgmatic could not find Borg. Is `borg`'s parent directory in the system `PATH`, both at the command-line and when running under systemd? If not, you can add it to the `PATH` or just set a `local_path` option in borgmatic. * `Cannot load credential "borgmatic.pw" because the systemd CREDENTIALS_DIRECTORY environment variable is not set`: Can I get a look at your borgmatic systemd service? Are you getting this error even when running borgmatic from `systemd`, or only when run manually from the command-line? If you want to run borgmatic manually from the command-line even when systemd credentials are configured, you'll need to upgrade to a newer version of borgmatic. (See [the docs](https://torsion.org/borgmatic/reference/configuration/credentials/systemd/) about this. You'll need 2.0.9+.)
muemo commented 2026-02-22 21:29:11 +00:00
Author
Copy link

Thanks for your response!

  • Considering it works both via systemd and on the command line with the plain text passphrase as encryption_passphrase, I am convinced, that borg is in fact in PATH. In fact, I even get the Error getting local Borg version problem, when the local_path is specified to be /usr/bin/borg.
  • My systemd service is the default debian one, listed below. I get the error both on command line as well as via systemd, see above for the two different logs. I would prefer to stay on the version in debian, and only need it to work in the systemd service.
borgmatic.service 
[Unit]
Description=borgmatic backup
Wants=network-online.target
After=network-online.target
# Prevent borgmatic from running unless the machine is plugged into power. Remove this line if you
# want to allow borgmatic to run anytime.
ConditionACPower=true
Documentation=https://torsion.org/borgmatic/
ConditionFileNotEmpty=|/etc/borgmatic/config.yaml
ConditionPathExistsGlob=|/etc/borgmatic.d/*.yaml

[Service]
Type=oneshot
RuntimeDirectory=borgmatic
StateDirectory=borgmatic

# Load single encrypted credential.
LoadCredentialEncrypted=borgmatic.pw

# Load multiple encrypted credentials.
# LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/

# Security settings for systemd running as root, optional but recommended to improve security. You
# can disable individual settings if they cause problems for your use case. For more details, see
# the systemd manual: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
LockPersonality=true
# Certain borgmatic features like Healthchecks integration need MemoryDenyWriteExecute to be off.
# But you can try setting it to "yes" for improved security if you don't use those features.
MemoryDenyWriteExecute=no
NoNewPrivileges=yes
PrivateDevices=yes
PrivateTmp=yes
ProtectClock=yes
ProtectControlGroups=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM
# To restrict write access further, change "ProtectSystem" to "strict" and
# uncomment "ReadWritePaths", "TemporaryFileSystem", "BindPaths" and
# "BindReadOnlyPaths". Then add any local repository paths to the list of
# "ReadWritePaths". This leaves most of the filesystem read-only to borgmatic.
ProtectSystem=full
# ReadWritePaths=-/mnt/my_backup_drive
# This will mount a tmpfs on top of /root and pass through needed paths
# TemporaryFileSystem=/root:ro
# BindPaths=-/root/.cache/borg -/root/.config/borg -/root/.borgmatic
# BindReadOnlyPaths=-/root/.ssh

# May interfere with running external programs within borgmatic hooks.
CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW

# Lower CPU and I/O priority.
Nice=19
CPUSchedulingPolicy=batch
IOSchedulingClass=best-effort
IOSchedulingPriority=7
IOWeight=100

Restart=no
# Prevent rate limiting of borgmatic log events. If you are using an older version of systemd that
# doesn't support this (pre-240 or so), you may have to remove this option.
LogRateLimitIntervalSec=0

# Delay start to prevent backups running during boot. Note that systemd-inhibit requires dbus and
# dbus-user-session to be installed.
ExecStartPre=sleep 1m
ExecStart=systemd-inhibit --who="borgmatic" --what="sleep:shutdown" --why="Prevent interrupting scheduled backup" /usr/bin/borgmatic --verbosity -2 --syslog-verbosity 1
Thanks for your response! * Considering it works both via systemd and on the command line with the plain text passphrase as `encryption_passphrase`, I am convinced, that `borg` _is_ in fact in `PATH`. In fact, I even get the `Error getting local Borg version` problem, when the `local_path` is specified to be `/usr/bin/borg`. * My systemd service is the default debian one, listed below. I get the error both on command line as well as via systemd, see above for the two different logs. I would prefer to stay on the version in debian, and only need it to work in the systemd service. <details> <summary>borgmatic.service</summary> ``` [Unit] Description=borgmatic backup Wants=network-online.target After=network-online.target # Prevent borgmatic from running unless the machine is plugged into power. Remove this line if you # want to allow borgmatic to run anytime. ConditionACPower=true Documentation=https://torsion.org/borgmatic/ ConditionFileNotEmpty=|/etc/borgmatic/config.yaml ConditionPathExistsGlob=|/etc/borgmatic.d/*.yaml [Service] Type=oneshot RuntimeDirectory=borgmatic StateDirectory=borgmatic # Load single encrypted credential. LoadCredentialEncrypted=borgmatic.pw # Load multiple encrypted credentials. # LoadCredentialEncrypted=borgmatic:/etc/credstore.encrypted/borgmatic/ # Security settings for systemd running as root, optional but recommended to improve security. You # can disable individual settings if they cause problems for your use case. For more details, see # the systemd manual: https://www.freedesktop.org/software/systemd/man/systemd.exec.html LockPersonality=true # Certain borgmatic features like Healthchecks integration need MemoryDenyWriteExecute to be off. # But you can try setting it to "yes" for improved security if you don't use those features. MemoryDenyWriteExecute=no NoNewPrivileges=yes PrivateDevices=yes PrivateTmp=yes ProtectClock=yes ProtectControlGroups=yes ProtectHostname=yes ProtectKernelLogs=yes ProtectKernelModules=yes ProtectKernelTunables=yes RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 AF_NETLINK RestrictNamespaces=yes RestrictRealtime=yes RestrictSUIDSGID=yes SystemCallArchitectures=native SystemCallFilter=@system-service SystemCallErrorNumber=EPERM # To restrict write access further, change "ProtectSystem" to "strict" and # uncomment "ReadWritePaths", "TemporaryFileSystem", "BindPaths" and # "BindReadOnlyPaths". Then add any local repository paths to the list of # "ReadWritePaths". This leaves most of the filesystem read-only to borgmatic. ProtectSystem=full # ReadWritePaths=-/mnt/my_backup_drive # This will mount a tmpfs on top of /root and pass through needed paths # TemporaryFileSystem=/root:ro # BindPaths=-/root/.cache/borg -/root/.config/borg -/root/.borgmatic # BindReadOnlyPaths=-/root/.ssh # May interfere with running external programs within borgmatic hooks. CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW # Lower CPU and I/O priority. Nice=19 CPUSchedulingPolicy=batch IOSchedulingClass=best-effort IOSchedulingPriority=7 IOWeight=100 Restart=no # Prevent rate limiting of borgmatic log events. If you are using an older version of systemd that # doesn't support this (pre-240 or so), you may have to remove this option. LogRateLimitIntervalSec=0 # Delay start to prevent backups running during boot. Note that systemd-inhibit requires dbus and # dbus-user-session to be installed. ExecStartPre=sleep 1m ExecStart=systemd-inhibit --who="borgmatic" --what="sleep:shutdown" --why="Prevent interrupting scheduled backup" /usr/bin/borgmatic --verbosity -2 --syslog-verbosity 1 ``` </details>
witten commented 2026-02-22 21:39:12 +00:00
Owner
Copy link

Thanks for providing those details. It looks like the version of borgmatic included in Debian has a bug preventing use of credential names that contain the "." character (#1044). If you don't want to upgrade to get the fix, then you can rename the credential (e.g. to borgmaticpw) in both the systemd service and borgmatic's configuration. Make sure to reload the service as well.

Let me know how that works out for you!

Thanks for providing those details. It looks like the version of borgmatic included in Debian has a bug preventing use of credential names that contain the "." character (#1044). If you don't want to upgrade to get the fix, then you can rename the credential (e.g. to `borgmaticpw`) in both the systemd service and borgmatic's configuration. Make sure to reload the service as well. Let me know how that works out for you!
muemo commented 2026-02-22 22:10:25 +00:00
Author
Copy link

Thank you a lot, this does indeed work!

Thank you a lot, this does indeed work!
muemo closed this issue 2026-02-22 22:10:25 +00:00
witten commented 2026-02-22 22:17:56 +00:00
Owner
Copy link

Great, glad to hear it!

Great, glad to hear it!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
command-hook-steps
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
borgmatic-collective/borgmatic#1271
No description provided.