borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 57
Code Issues 71 Pull requests 3 Releases 162 Packages 1 Activity Actions 1

Cannot get credentials from systemd store directory when run from command line #1180

New issue
Open
opened 2025-11-11 08:34:40 +00:00 by thecodehen · 8 comments
thecodehen commented 2025-11-11 08:34:40 +00:00
Copy link

What I'm trying to do and why

I'm trying to run borgamtic list in the command line with borgmatic configs that have systemd credentials in it. The credentials are stored in a custom directory /etc/credstore.encrypted/borgmatic but borgmatic list returns "Failed to read encrypted credential data: No such file or directory" errors.

I've followed the instructions here for creating multiple credentials under a systemd credentials directory.

Steps to reproduce

The relevant config should be:

encryption_passphrase: '{credential systemd borgmatic_pw1}'

zfs:

I am using the zfs hook. The file list in the systemd credstore:

root@ubuntu:/etc/credstore.encrypted/borgmatic# tree
.
├── pw1
└── pw2

1 directory, 2 files

Actual behavior

Running the config through systemd is fine and exists successfully. However, running borgmatic list fails.

root@ubuntu:~# borgmatic list
Failed to read encrypted credential data: No such file or directory
/etc/borgmatic.d/config.yaml: /etc/borgmatic.d/config.yaml: Error getting local Borg version
/etc/borgmatic.d/config.yaml: Command '('systemd-creds', 'decrypt', '/etc/credstore.encrypted/borgmatic_pw1')' returned non-zero exit status 1.
/etc/borgmatic.d/config.yaml: Error running configuration
/etc/borgmatic.d/config.yaml: An error occurred

summary:
An error occurred
/etc/borgmatic.d/config.yaml: Error getting local Borg version
Command '('systemd-creds', 'decrypt', '/etc/credstore.encrypted/borgmatic_pw1')' returned non-zero exit status 1.
Error running configuration

Expected behavior

borgmatic list succeeds.

Other notes / implementation ideas

Seems like there are some edge cases in interpreting the credstore name in the config not considered in the code.

borgmatic version

2.0.11

borgmatic installation method

pipx

Borg version

1.4.1

Python version

No response

Database version (if applicable)

No response

Operating system and version

Ubuntu 24.04.3 LTS

### What I'm trying to do and why I'm trying to run `borgamtic list` in the command line with borgmatic configs that have systemd credentials in it. The credentials are stored in a custom directory `/etc/credstore.encrypted/borgmatic` but `borgmatic list` returns "Failed to read encrypted credential data: No such file or directory" errors. I've followed the instructions [here](https://torsion.org/borgmatic/reference/configuration/credentials/systemd/) for creating multiple credentials under a systemd credentials directory. ### Steps to reproduce The relevant config should be: ``` encryption_passphrase: '{credential systemd borgmatic_pw1}' zfs: ``` I am using the zfs hook. The file list in the systemd credstore: ``` root@ubuntu:/etc/credstore.encrypted/borgmatic# tree . ├── pw1 └── pw2 1 directory, 2 files ``` ### Actual behavior Running the config through systemd is fine and exists successfully. However, running `borgmatic list` fails. ``` root@ubuntu:~# borgmatic list Failed to read encrypted credential data: No such file or directory /etc/borgmatic.d/config.yaml: /etc/borgmatic.d/config.yaml: Error getting local Borg version /etc/borgmatic.d/config.yaml: Command '('systemd-creds', 'decrypt', '/etc/credstore.encrypted/borgmatic_pw1')' returned non-zero exit status 1. /etc/borgmatic.d/config.yaml: Error running configuration /etc/borgmatic.d/config.yaml: An error occurred summary: An error occurred /etc/borgmatic.d/config.yaml: Error getting local Borg version Command '('systemd-creds', 'decrypt', '/etc/credstore.encrypted/borgmatic_pw1')' returned non-zero exit status 1. Error running configuration ``` ### Expected behavior `borgmatic list` succeeds. ### Other notes / implementation ideas Seems like there are some edge cases in interpreting the credstore name in the config not considered in the code. ### borgmatic version 2.0.11 ### borgmatic installation method pipx ### Borg version 1.4.1 ### Python version _No response_ ### Database version (if applicable) _No response_ ### Operating system and version Ubuntu 24.04.3 LTS
witten commented 2025-11-12 01:01:55 +00:00
Owner
Copy link

Thanks for filing this! So there's a feature that falls back to using systemd-creds to load a credential when running borgmatic outside of systemd, but it appears that the feature was never tested along with the instructions in the documentation for creating multiple credentials! And therefore the path loaded by borgmatic's systemd-creds invocation is incorrect.

One solution I can think of is to change the documentation to recommend storing borgmatic credentials directly in /etc/credstore.encrypted/ without the borgmatic/ subdirectory ... and with the borgmatic_... prefix. That way, the credential name used in borgmatic's config file would match up with its filesystem path.

A different solution would be for borgmatic to take a credential name like borgmatic_pw1 and replace the _ with a / to make the credential path. That would allow you to leave everything in the borgmatic/ subdirectory, but it's potentially a little less intuitive and a little more brittle.

Thoughts?

Thanks for filing this! So there's a feature that falls back to using `systemd-creds` to load a credential when running borgmatic outside of systemd, but it appears that the feature was never tested along with the instructions in the documentation for creating multiple credentials! And therefore the path loaded by borgmatic's `systemd-creds` invocation is incorrect. One solution I can think of is to change the documentation to recommend storing borgmatic credentials directly in `/etc/credstore.encrypted/` without the `borgmatic/` subdirectory ... and *with* the `borgmatic_...` prefix. That way, the credential name used in borgmatic's config file would match up with its filesystem path. A different solution would be for borgmatic to take a credential name like `borgmatic_pw1` and replace the `_` with a `/` to make the credential path. That would allow you to leave everything in the `borgmatic/` subdirectory, but it's potentially a little less intuitive and a little more brittle. Thoughts?
thecodehen commented 2025-11-12 08:36:40 +00:00
Author
Copy link

I think whatever the case, there should only be only one place that stores credentials. For the documentation solution, it seems like there would be a problem if someone stores credentials in both /etc/credstore.encrypted/borgmatic/pw1 and /etc/credstore.encrypted/borgmatic_pw1 where the systemd instance would read the credential from the former file and the cli instance would read the credential from the latter, causing problems and confusion in the future.

I would prefer the second solution, but also we need to add more documentation on how the name is parsed, such as: there cannot be any underscores in the directory name. There may be a better way to store the directory info and parse it as well. I’m not sure how the folder name is being parsed now though, is it being done by borgmatic or systemd? Can you have sub folders in the credential name, e.g., borgmatic/host1/pw1?

If you point me to some relevant parts of the codebase, maybe I can take a stab at this bug as well.

I think whatever the case, there should only be only one place that stores credentials. For the documentation solution, it seems like there would be a problem if someone stores credentials in both `/etc/credstore.encrypted/borgmatic/pw1` and `/etc/credstore.encrypted/borgmatic_pw1` where the `systemd` instance would read the credential from the former file and the cli instance would read the credential from the latter, causing problems and confusion in the future. I would prefer the second solution, but also we need to add more documentation on how the name is parsed, such as: there cannot be any underscores in the directory name. There may be a better way to store the directory info and parse it as well. I’m not sure how the folder name is being parsed now though, is it being done by borgmatic or systemd? Can you have sub folders in the credential name, e.g., borgmatic/host1/pw1? If you point me to some relevant parts of the codebase, maybe I can take a stab at this bug as well.
witten commented 2025-11-12 18:28:26 +00:00
Owner
Copy link

I think whatever the case, there should only be only one place that stores credentials. For the documentation solution, it seems like there would be a problem if someone stores credentials in both /etc/credstore.encrypted/borgmatic/pw1 and /etc/credstore.encrypted/borgmatic_pw1 where the systemd instance would read the credential from the former file and the cli instance would read the credential from the latter, causing problems and confusion in the future.

Yeah, that seems problematic.

I would prefer the second solution, but also we need to add more documentation on how the name is parsed, such as: there cannot be any underscores in the directory name. There may be a better way to store the directory info and parse it as well.

There's already a encrypted_credentials_directory option under borgmatic's systemd: option, so in theory the user can already override that path. It does occur to me that adding the following to your configuration would almost work:

systemd:
    encrypted_credentials_directory: /etc/credstore.encrypted/borgmatic

Except then the credential names don't quite match up with their filenames (e.g. pw1 vs borgmatic_pw1). But that could be addressed by renaming the credentials (and then updating the documentation accordingly).

I’m not sure how the folder name is being parsed now though, is it being done by borgmatic or systemd? Can you have sub folders in the credential name, e.g., borgmatic/host1/pw1?

If you point me to some relevant parts of the codebase, maybe I can take a stab at this bug as well.

Here's the relevant code: borgmatic/hooks/credential/systemd.py ... When you borgmatic list outside of systemd, that path is constructed entirely by borgmatic. And sub-folders won't work currently, because there's a single encrypted_credentials_directory used when you run borgmatic outside of systemd (and in fact when you run borgmatic inside a systemd service as well).

Let me know what you decide to do!

> I think whatever the case, there should only be only one place that stores credentials. For the documentation solution, it seems like there would be a problem if someone stores credentials in both `/etc/credstore.encrypted/borgmatic/pw1` and `/etc/credstore.encrypted/borgmatic_pw1` where the `systemd` instance would read the credential from the former file and the cli instance would read the credential from the latter, causing problems and confusion in the future. Yeah, that seems problematic. > I would prefer the second solution, but also we need to add more documentation on how the name is parsed, such as: there cannot be any underscores in the directory name. There may be a better way to store the directory info and parse it as well. There's already a `encrypted_credentials_directory` option under borgmatic's `systemd:` option, so in theory the user can already override that path. It does occur to me that adding the following to your configuration would *almost* work: ```yaml systemd: encrypted_credentials_directory: /etc/credstore.encrypted/borgmatic ``` Except then the credential names don't quite match up with their filenames (e.g. `pw1` vs `borgmatic_pw1`). But that could be addressed by renaming the credentials (and then updating the documentation accordingly). > I’m not sure how the folder name is being parsed now though, is it being done by borgmatic or systemd? Can you have sub folders in the credential name, e.g., borgmatic/host1/pw1? > > If you point me to some relevant parts of the codebase, maybe I can take a stab at this bug as well. Here's the relevant code: [borgmatic/hooks/credential/systemd.py](https://projects.torsion.org/borgmatic-collective/borgmatic/src/branch/main/borgmatic/hooks/credential/systemd.py#L43) ... When you `borgmatic list` outside of systemd, that path is constructed entirely by borgmatic. And sub-folders won't work currently, because there's a single `encrypted_credentials_directory` used when you run borgmatic outside of systemd (and in fact when you run borgmatic inside a systemd service as well). Let me know what you decide to do!
Afetzner commented 2025-11-24 09:01:41 +00:00
Copy link

Check out issue #1123
"Use systemd credential only when running as a systemd unit, defer to manual entry otherwise"

Check out issue #1123 "Use systemd credential only when running as a systemd unit, defer to manual entry otherwise"
👍 1
witten commented 2025-11-26 22:06:57 +00:00
Owner
Copy link

Just checking in... @thecodehen, did you plan to submit a PR for this? Either way, please let me know. Thanks!

Just checking in... @thecodehen, did you plan to submit a PR for this? Either way, please let me know. Thanks!
thecodehen commented 2025-11-27 00:43:31 +00:00
Author
Copy link

Yep, I think I have a fix by checking if the name has '_' and adding that to the directory path. Is there any branch naming conventions?

Yep, I think I have a fix by checking if the name has '_' and adding that to the directory path. Is there any branch naming conventions?
thecodehen commented 2025-11-27 00:57:33 +00:00
Author
Copy link

Ok this is my first time submitting a PR here--let me know any feedback!

Ok this is my first time submitting a PR here--let me know any feedback!
witten commented 2025-12-02 18:31:50 +00:00
Owner
Copy link

Yep, I think I have a fix by checking if the name has '_' and adding that to the directory path. Is there any branch naming conventions?

No real branch naming conventions. I generally try to go for descriptive names, but that's not always practical depending on the complexity of the issue.

> Yep, I think I have a fix by checking if the name has '_' and adding that to the directory path. Is there any branch naming conventions? No real branch naming conventions. I generally try to go for descriptive names, but that's not always practical depending on the complexity of the issue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
parallel-borgmatic
command-hook-steps
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#1180
No description provided.