Run borgmatic as non-root user & read password from a file (Questions) #68

New Issue

borgos · 2018-06-12T20:36:36Z

borgos commented

2018-06-12 20:36:36 +00:00

How can I start borgmatic as non-root user?
I can set a password in /etc/borgmatic/config.yaml and it works but I would like borgmatic to read the password from a file. I tried with this:
storage:
export BORG_PASSCOMMAND="cat /home/myuser/.borg-passphrase"
but it didn't work. How can it be done?

1. How can I start borgmatic as non-root user? 2. I can set a password in /etc/borgmatic/config.yaml and it works but I would like borgmatic to read the password from a file. I tried with this: storage: export BORG_PASSCOMMAND="cat /home/myuser/.borg-passphrase" but it didn't work. How can it be done?

witten commented

2018-06-13 03:55:38 +00:00

Just do it! Run borgmatic manually as a non-root user. Or, if you like, use the user's own crontab. Since a non-root user won't typically have access to /etc/borgmatic, you'll probably want to specify an alternate config file path with the -c/--config option. Let me know if you have any problems with non-root usage.
There's an encryption_passcommand: option in the storage section of borgmatic's configuration file. I would expect this to do what you're looking for:

storage:
   encryption_passcommand: cat /home/myuser/.borg-passphrase

However, note that under the hood, this just constructs the BORG_PASSCOMMAND environment variable as you've already tried. So if that's not working, perhaps this is an issue with Borg? Have you tried increasing the verbosity level to see if Borg is picking up that environment variable and consuming it?

1. Just do it! Run borgmatic manually as a non-root user. Or, if you like, use the user's own crontab. Since a non-root user won't typically have access to `/etc/borgmatic`, you'll probably want to specify an alternate config file path with the `-c`/`--config` option. Let me know if you have any problems with non-root usage. 2. There's an `encryption_passcommand:` option in the `storage` section of borgmatic's configuration file. I would expect this to do what you're looking for: ``` storage: encryption_passcommand: cat /home/myuser/.borg-passphrase ``` However, note that under the hood, this just constructs the `BORG_PASSCOMMAND` environment variable as you've already tried. So if that's not working, perhaps this is an issue with Borg? Have you tried increasing the verbosity level to see if Borg is picking up that environment variable and consuming it?

witten added the

question / support

label 2018-06-13 03:55:44 +00:00

borgos commented

2018-06-13 10:59:02 +00:00

"Just do it! Run borgmatic manually as a non-root user. Or, if you like, use the user’s own crontab. Since a non-root user won’t typically have access to /etc/borgmatic, you’ll probably want to specify an alternate config file path with the -c/--config option. Let me know if you have any problems with non-root usage."

I did run it as a non-root user but it seems that borgmatic still has created the archives as root since I didn't have access as normal user to the repo. I would like to use systemd with timer.
I have created a new config file with:
generate-borgmatic-config --destination /home/jose/.borgmatic/home.yaml
renamed the first config file to /etc/borgmatic/config.yaml.bak and tried to run borgmatic again. Now I'm getting this error:
$ borgmatic -v 2
Ensuring legacy configuration is upgraded
Error: No configuration files found in: /etc/borgmatic/config.yaml /etc/borgmatic.d

"Just do it! Run borgmatic manually as a non-root user. Or, if you like, use the user’s own crontab. Since a non-root user won’t typically have access to /etc/borgmatic, you’ll probably want to specify an alternate config file path with the -c/--config option. Let me know if you have any problems with non-root usage." I did run it as a non-root user but it seems that borgmatic still has created the archives as root since I didn't have access as normal user to the repo. I would like to use systemd with timer. I have created a new config file with: generate-borgmatic-config --destination /home/jose/.borgmatic/home.yaml renamed the first config file to /etc/borgmatic/config.yaml.bak and tried to run borgmatic again. Now I'm getting this error: $ borgmatic -v 2 Ensuring legacy configuration is upgraded Error: No configuration files found in: /etc/borgmatic/config.yaml /etc/borgmatic.d

borgos closed this issue

2018-06-13 11:00:42 +00:00

borgos reopened this issue

2018-06-13 11:02:50 +00:00

witten commented

2018-06-14 04:29:37 +00:00

If borgmatic, run as a non-root user, doesn't have access to a repo because it's owned by root, that probably means the repo was created by the root user. If there's nothing in it, I'd recommend deleting the repo and recreating it as a non-root user.

As for most recent error, I think you need to provide borgmatic with the --config option plus your config filename, so that borgmatic knows where to find your non-root configuration file. If you don't provide that, borgmatic looks in /etc/borgmatic/ by default.

If borgmatic, run as a non-root user, doesn't have access to a repo because it's owned by root, that probably means the repo was created by the root user. If there's nothing in it, I'd recommend deleting the repo and recreating it as a non-root user. As for most recent error, I think you need to provide borgmatic with the `--config` option plus your config filename, so that borgmatic knows where to find your non-root configuration file. If you don't provide that, borgmatic looks in `/etc/borgmatic/` by default.

borgos commented

2018-06-23 10:09:47 +00:00

"If borgmatic, run as a non-root user, doesn’t have access to a repo because it’s owned by root, that probably means the repo was created by the root user. If there’s nothing in it, I’d recommend deleting the repo and recreating it as a non-root user."

The repo was not created by the root user but something went wrong, I don't know what, and the permissions of some borg files changed from normal user to root user.

borgmatic/systemd seems to be creating backups normally (I can list them) but when I run:
~ borgmatic --verbosity 2 --config /home/jose/.borgmatic/home_jose.yaml
I'm getting an error:
/home/jose/.borgmatic/home_jose.yaml: Running command for on-error hook
/home/jose/.borgmatic/home_jose.yaml: Hook command: echo "Error while creating a backup."
Error while creating a backup.
Command '('borg', 'create', '/mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S}', '/home/jose', '--exclude-from', '/mnt/BACKUPS/borg_exclude.txt', '--exclude-caches', '--exclude-if-present', '.nobackup', '--compression', 'lz4', '--one-file-system', '--lock-wait', '5', '--debug', '--list', '--stats')' returned non-zero exit status 1.

"If borgmatic, run as a non-root user, doesn’t have access to a repo because it’s owned by root, that probably means the repo was created by the root user. If there’s nothing in it, I’d recommend deleting the repo and recreating it as a non-root user." The repo was not created by the root user but something went wrong, I don't know what, and the permissions of some borg files changed from normal user to root user. borgmatic/systemd seems to be creating backups normally (I can list them) but when I run: ~ borgmatic --verbosity 2 --config /home/jose/.borgmatic/home_jose.yaml I'm getting an error: /home/jose/.borgmatic/home_jose.yaml: Running command for on-error hook /home/jose/.borgmatic/home_jose.yaml: Hook command: echo "Error while creating a backup." Error while creating a backup. Command '('borg', 'create', '/mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S}', '/home/jose', '--exclude-from', '/mnt/BACKUPS/borg_exclude.txt', '--exclude-caches', '--exclude-if-present', '.nobackup', '--compression', 'lz4', '--one-file-system', '--lock-wait', '5', '--debug', '--list', '--stats')' returned non-zero exit status 1.

witten commented

2018-06-23 20:42:39 +00:00

A few different ideas to help with debugging:

You could try running the Borg command directly by itself to see what precise Borg error is occurring. Example: borg create /mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S} /home/jose --exclude-from /mnt/BACKUPS/borg_exclude.txt --exclude-caches --exclude-if-present .nobackup --compression lz4 --one-file-system --lock-wait 5 --debug --list --stats
You can try commenting out the on-error hook from borgmatic's config file, and then try running borgmatic again. The hook may be obscuring the actual error.
You can post your borgmatic config file here (with anything sensitive redacted). That may help me understand what's going on. It may also be helpful to post the entire borgmatic runtime spew instead of just a portion of it.

A few different ideas to help with debugging: 1. You could try running the Borg command directly by itself to see what precise Borg error is occurring. Example: `borg create /mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S} /home/jose --exclude-from /mnt/BACKUPS/borg_exclude.txt --exclude-caches --exclude-if-present .nobackup --compression lz4 --one-file-system --lock-wait 5 --debug --list --stats` 2. You can try commenting out the on-error hook from borgmatic's config file, and then try running borgmatic again. The hook may be obscuring the actual error. 3. You can post your borgmatic config file here (with anything sensitive redacted). That may help me understand what's going on. It may also be helpful to post the entire borgmatic runtime spew instead of just a portion of it.

borgos commented

2018-06-24 10:59:43 +00:00

1.I run the following without getting any errors:
borg create -spv --compression lz4 --list --exclude-caches --exclude-if-present .nobackup --exclude-from ‘/mnt/BACKUPS/borg_exclude.txt’ --one-file-system --lock-wait 5 --debug --list --stats /mnt/BACKUPS/Borg/home/::mybackup-{now:%Y%m%d-%H%M%S} /home/jose/

2.Already tried that. This is what I get:
check_free_space: required bytes 232765978, free bytes 1238147207 security: saving state for … to /home/jose/.config/borg/security/… security: current location /mnt/BACKUPS/Borg/home security: key type 3

security: manifest timestamp 2018-06-24T10:41:13.092337
Archive name: myhost-home-backup-20180624-123844 Archive fingerprint: … Time (start): Sun, 2018-06-24 12:38:45 Time (end): Sun, 2018-06-24 12:41:13 Duration: 2 minutes 27.98 seconds Number of files: 488136

Utilization of max. archive size: 0%
				Original size      Compressed size    Deduplicated size
This archive: 298.21 GB 251.63 GB 11.80 MB All archives: 5.32 TB 4.47 TB 325.29 GB

	Unique chunks         Total chunks
Chunk index: 570768 9916109
/home/jose/.borgmatic/home_jose.yaml: No commands to run for on-error hook Command ‘(‘borg’, ‘create’, ‘/mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S}’, ‘/home/jose’, ‘--exclude-from’, ‘/mnt/BACKUPS/borg_exclude.txt’, ‘--exclude-caches’, ‘--exclude-if-present’, ‘.nobackup’, ‘--compression’, ‘lz4’, ‘--one-file-system’, ‘--lock-wait’, ‘5’, ‘--debug’, ‘--list’, ‘--stats’)’ returned non-zero exit status 1.

3.My borgmatic config file:
location:

source_directories:
	- /home/jose

one_file_system: true

repositories:
	- /mnt/BACKUPS/Borg/home/

exclude_from:
	- /mnt/BACKUPS/borg_exclude.txt

exclude_caches: true

exclude_if_present: .nobackup
storage:

encryption_passcommand: cat /home/jose/.borg-passphrase

compression: lz4

lock_wait: 5

archive_name_format: '{hostname}-home-backup-{now:%Y%m%d-%H%M%S}'
retention:

keep_within: 3H

keep_minutely: 60

keep_hourly: 24

keep_daily: 7

keep_weekly: 4

keep_monthly: 6

keep_yearly: 1

prefix: sourcehostname
consistency:

checks:
	- repository
	- archives

check_last: 3
hooks:

on_error:
	- echo "Error while creating a backup."

1.I run the following without getting any errors: borg create -spv --compression lz4 --list --exclude-caches --exclude-if-present .nobackup --exclude-from ‘/mnt/BACKUPS/borg_exclude.txt’ --one-file-system --lock-wait 5 --debug --list --stats /mnt/BACKUPS/Borg/home/::mybackup-{now:%Y%m%d-%H%M%S} /home/jose/ 2.Already tried that. This is what I get: check_free_space: required bytes 232765978, free bytes 1238147207 security: saving state for … to /home/jose/.config/borg/security/… security: current location /mnt/BACKUPS/Borg/home security: key type 3 security: manifest timestamp 2018-06-24T10:41:13.092337 Archive name: myhost-home-backup-20180624-123844 Archive fingerprint: … Time (start): Sun, 2018-06-24 12:38:45 Time (end): Sun, 2018-06-24 12:41:13 Duration: 2 minutes 27.98 seconds Number of files: 488136 Utilization of max. archive size: 0% Original size Compressed size Deduplicated size This archive: 298.21 GB 251.63 GB 11.80 MB All archives: 5.32 TB 4.47 TB 325.29 GB Unique chunks Total chunks Chunk index: 570768 9916109 /home/jose/.borgmatic/home_jose.yaml: No commands to run for on-error hook Command ‘(‘borg’, ‘create’, ‘/mnt/BACKUPS/Borg/home/::{hostname}-home-backup-{now:%Y%m%d-%H%M%S}’, ‘/home/jose’, ‘--exclude-from’, ‘/mnt/BACKUPS/borg_exclude.txt’, ‘--exclude-caches’, ‘--exclude-if-present’, ‘.nobackup’, ‘--compression’, ‘lz4’, ‘--one-file-system’, ‘--lock-wait’, ‘5’, ‘--debug’, ‘--list’, ‘--stats’)’ returned non-zero exit status 1. 3.My borgmatic config file: location: source_directories: - /home/jose one_file_system: true repositories: - /mnt/BACKUPS/Borg/home/ exclude_from: - /mnt/BACKUPS/borg_exclude.txt exclude_caches: true exclude_if_present: .nobackup storage: encryption_passcommand: cat /home/jose/.borg-passphrase compression: lz4 lock_wait: 5 archive_name_format: '{hostname}-home-backup-{now:%Y%m%d-%H%M%S}' retention: keep_within: 3H keep_minutely: 60 keep_hourly: 24 keep_daily: 7 keep_weekly: 4 keep_monthly: 6 keep_yearly: 1 prefix: sourcehostname consistency: checks: - repository - archives check_last: 3 hooks: on_error: - echo "Error while creating a backup."

witten commented

2018-06-24 16:06:30 +00:00

Given that Borg appears to be running without errors, but borgmatic thinks it's erroring, my recommendation would be to run that bare Borg command by itself again, and then immediately after: echo $? to see its numeric exit code. If it's non-zero, then that would explain why borgmatic thinks it's erroring. And then we can look up that exit code in Borg documentation or source to see what the code is indicating.

If the exit code is just zero, then something really weird is going on.

Given that Borg *appears* to be running without errors, but borgmatic thinks it's erroring, my recommendation would be to run that bare Borg command by itself again, and then immediately after: `echo $?` to see its numeric exit code. If it's non-zero, then that would explain why borgmatic thinks it's erroring. And then we can look up that exit code in Borg documentation or source to see what the code is indicating. If the exit code is just zero, then something really weird is going on.

borgos commented

2018-06-24 16:45:49 +00:00

I ran borg again and after it has finished I did echo?. The exit code is 1.
So borgmatic isn't the cause of the error. Any ideas how can I check what is causing the error?

I ran borg again and after it has finished I did $ echo $?. The exit code is 1. So borgmatic isn't the cause of the error. Any ideas how can I check what is causing the error?

witten commented

2018-06-24 18:07:35 +00:00

I was hoping for a more exotic error code like 255 or something, so we could look that up. Given that you're just getting 1, I recommend doing one of two things:

File a Borg support ticket with your reproduction steps, plus the actual behavior (exit code 1) and the expected behavior (exit code 0).
See if there's anything you might be missing in the raw Borg output that indicates an error is occurring. Looks like you've already got --debug on, so that's good. You might try removing the -v flag in case that's overriding --debug.

I was hoping for a more exotic error code like 255 or something, so we could look that up. Given that you're just getting 1, I recommend doing one of two things: 1. File a Borg support ticket with your reproduction steps, plus the actual behavior (exit code 1) and the expected behavior (exit code 0). 2. See if there's anything you might be missing in the raw Borg output that indicates an error is occurring. Looks like you've already got `--debug` on, so that's good. You might try removing the `-v` flag in case that's overriding `--debug`.

borgos commented

2018-07-03 09:53:24 +00:00

A permission issue was causing exit code 1. It's OK now. Thanks!

borgos closed this issue

2018-07-03 09:53:24 +00:00

witten referenced this issue from a commit

2023-04-06 20:59:09 +00:00

feat: add logfile name to hook context for interpolation

everydayrain referenced this issue

2023-09-05 16:35:25 +00:00

Cron job config for virtual environments #752

Sign in to join this conversation.