Make options like "--umask" available for all Borg commands.

ams_tschoening commented

2021-08-16 14:47:10 +00:00

What I'm trying to do and why

I have created a special backup user to execute BorgMatic and maintain it's config files, those files related to Borg itself etc. That user is having some entries for CRON associated which then execute the actual backups using sudo, so things run as root in the end. The point is that some tasks don't need to be done as root, e.g. listing and deleting archives, because those are stored on an external device.

The problem with this setup was that Borg applied a pretty restrictive umask of 077 by default, which prevented my special backup user from accessing those files created, updated etc. when running as root. The good news is that configs of BorgMatic (somewhat) prepare for those cases already using extra_borg_options and using those I can change the umask to a more permissive value of e.g. 0007. That seems to work for init, prune, create and check.

It doesn't seem to work for all other commands of interest supported by BorgMatic, like list. It as well doesn't seem to work when the special command borg is used to forward arbitrary unsupported commands. Would be great if one would be able to set those options for any commands.

Actual behavior

The following is when adding list to extra_borg_options. Thought it might work:

ams_d_bak_borg@amsoft:~$ sudo borgmatic --config ~/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml --verbosity 2 list
Ensuring legacy configuration is upgraded
/home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: No valid configuration files found

summary:
/home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: Error parsing configuration file
An error occurred while parsing a configuration file at /home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml:
At 'storage.extra_borg_options': Additional properties are not allowed ('list' was unexpected)
/home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: No valid configuration files found

The following is without the former changes and shows that with list the default umask of Borg is used. This is the same for any other arbitrary Borg command.

ams_d_bak_borg@amsoft:~$ sudo borgmatic --config ~/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml --verbosity 2 list
Ensuring legacy configuration is upgraded
amsoft-sbox.bitstore.group:bak_borg/mysql/files/de.am-soft.noc: Listing archives
borg list --debug --show-rc amsoft-sbox.bitstore.group:bak_borg/mysql/files/de.am-soft.noc
using builtin fallback logging configuration
35 self tests completed in 0.04 seconds
SSH command line: ['ssh', '-F', '/home/ams_d_bak_borg/.ssh/config', 'amsoft-sbox.bitstore.group', 'borg', 'serve', '--umask=077', '--debug']
TAM-verified manifest
[...]

Expected behavior

Would be great to be able to forward options like umask really to all commands of Borg, because otherwise permission issues might happen. Remember that this option is not only applied for the repos, but for local cache, security etc. files maintained by Borg as well.

https://github.com/borgbackup/borg/issues/1859#issuecomment-261727374

--umask M set umask to M (local and remote, default: 0077)

Thanks for considering!

Environment

borgmatic version: 1.5.15
borgmatic installation method: PIP, system wide
Borg version: 1.1.16
Python version: 3.8.10
operating system and version: Ubuntu 20.04

#### What I'm trying to do and why I have created a special backup user to execute BorgMatic and maintain it's config files, those files related to Borg itself etc. That user is having some entries for CRON associated which then execute the actual backups using `sudo`, so things run as `root` in the end. The point is that some tasks don't need to be done as `root`, e.g. listing and deleting archives, because those are stored on an external device. The problem with this setup was that Borg applied a pretty restrictive `umask` of `077` by default, which prevented my special backup user from accessing those files created, updated etc. when running as `root`. The good news is that configs of BorgMatic (somewhat) prepare for those cases already using `extra_borg_options` and using those I can change the `umask` to a more permissive value of e.g. `0007`. That seems to work for `init`, `prune`, `create` and `check`. It doesn't seem to work for all other commands of interest supported by BorgMatic, like `list`. It as well doesn't seem to work when the special command `borg` is used to forward arbitrary unsupported commands. Would be great if one would be able to set those options for any commands. #### Actual behavior The following is when adding `list` to `extra_borg_options`. Thought it might work: ``` ams_d_bak_borg@amsoft:~$ sudo borgmatic --config ~/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml --verbosity 2 list Ensuring legacy configuration is upgraded /home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: No valid configuration files found summary: /home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: Error parsing configuration file An error occurred while parsing a configuration file at /home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: At 'storage.extra_borg_options': Additional properties are not allowed ('list' was unexpected) /home/ams_d_bak_borg/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml: No valid configuration files found ``` The following is without the former changes and shows that with `list` the default `umask` of Borg is used. This is the same for any other arbitrary Borg command. ``` ams_d_bak_borg@amsoft:~$ sudo borgmatic --config ~/.config/borgmatic.d/mysql/files/de.am-soft.noc.yaml --verbosity 2 list Ensuring legacy configuration is upgraded amsoft-sbox.bitstore.group:bak_borg/mysql/files/de.am-soft.noc: Listing archives borg list --debug --show-rc amsoft-sbox.bitstore.group:bak_borg/mysql/files/de.am-soft.noc using builtin fallback logging configuration 35 self tests completed in 0.04 seconds SSH command line: ['ssh', '-F', '/home/ams_d_bak_borg/.ssh/config', 'amsoft-sbox.bitstore.group', 'borg', 'serve', '--umask=077', '--debug'] TAM-verified manifest [...] ``` #### Expected behavior Would be great to be able to forward options like `umask` really to all commands of Borg, because otherwise permission issues might happen. Remember that this option is not only applied for the repos, but for local cache, security etc. files maintained by Borg as well. https://github.com/borgbackup/borg/issues/1859#issuecomment-261727374 > --umask M set umask to M (local and remote, default: 0077) **Thanks for considering!** #### Environment **borgmatic version:** 1.5.15 **borgmatic installation method:** PIP, system wide **Borg version:** 1.1.16 **Python version:** 3.8.10 **operating system and version:** Ubuntu 20.04

ams_tschoening commented

2021-08-16 21:40:52 +00:00

Author

I had a look at my YAML file again and seem to have missed the already available storage.umask with the following documentation:

    # Umask to be used for borg create. Defaults to 0077.
    # umask: 0077

That made me wonder if that setting really is only applied to create or more and after looking at the code, it's applied to other commands like prune, mount etc. as well. Though, it's still not applied to all commands, I couldn't find check, list, borg etc.

The problem in my opinion is that while umask seems to make sense for changing commands only, BORG applies that value to locally maintained files for caches, security files etc. as well and some of those seem to be rewritten even if read-only oprations are applied to a repo only. So even after those permissions in locally maintained data might simply be wrong.

BORG documents that setting for all commands as well:

All Borg commands share these options:
[...]
--umask M set umask to M (local and remote, default: 0077)

https://borgbackup.readthedocs.io/en/stable/usage/general.html#common-options

I had a look at my YAML file again and seem to have missed the already available `storage.umask` with the following documentation: ``` # Umask to be used for borg create. Defaults to 0077. # umask: 0077 ``` That made me wonder if that setting really is only applied to `create` or more and after looking at the code, it's applied to other commands like `prune`, `mount` etc. as well. Though, it's still not applied to all commands, I couldn't find `check`, `list`, `borg` etc. The problem in my opinion is that while `umask` seems to make sense for changing commands only, BORG applies that value to locally maintained files for caches, security files etc. as well and some of those seem to be rewritten even if read-only oprations are applied to a repo only. So even after those permissions in locally maintained data might simply be wrong. BORG documents that setting for all commands as well: > All Borg commands share these options: > [...] > --umask M set umask to M (local and remote, default: 0077) https://borgbackup.readthedocs.io/en/stable/usage/general.html#common-options