The sample systemd unit file prevents the ZFS integration from working #1114

New Issue

Closed

opened 2025-06-29 11:06:56 +00:00 by Cysioland · 2 comments

Cysioland commented 2025-06-29 11:06:56 +00:00

Copy Link

What I'm trying to do and why

When trying to use the ZFS integration to make snapshots of backed up directories, while running Borgmatic through the enclosed systemd unit file, one gets permission errors

Steps to reproduce

1. Install the ZFS Debian package (they use a systemd unit file consistent with what's provided under sample/systemd/borgmatic.service)
2. Setup ZFS integration in your config file

Actual behavior

systemd-inhibit[1613548]: /dev/zfs and /proc/self/mounts are required.
systemd-inhibit[1613548]: Try running 'udevadm trigger' and 'mount -t proc proc /proc' as root.
borgmatic[1613220]: CRITICAL nas: Error running actions for repository
CRITICAL nas: Command '('zfs', 'list', '-H', '-t', 'filesystem', '-o', 'name,mountpoint,canmount,org.torsion.borgmatic:backup')' returned non-zero exit status 1.
borgmatic[1613220]: CRITICAL /etc/borgmatic/config.yaml: An error occurred

CRITICAL cannot create snapshots : permission denied
CRITICAL Command 'zfs snapshot hdd@borgmatic-1670783' returned non-zero exit status 1.

CRITICAL filesystem 'hdd@borgmatic-1674374' can not be mounted: Operation not permitted
CRITICAL Command 'mount -t zfs -o ro hdd@borgmatic-1674374 /run/borgmatic/zfs_snapshots/eb72ccdc3f97b208385c/hdd' returned non-zero exit status 1.

Expected behavior

The snapshots should be created successfully

Other notes / implementation ideas

I tracked down the offending unit file settings.

The ZFS dataset listing fails due to the PrivateDevices=yes line (the borgmatic process loses access to /dev/zfs that way)
Snapshotting fails due to the CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW line. I'm not sure which capabilities the ZFS snapshot process needs, but clearing the setting allowed the snapshots to be made.
Mounting snapshots is prevented by the SystemCallFilter=@system-service line. Setting it to SystemCallFilter=@system-service @mount allowed the backup to proceed.

borgmatic version

1.9.14 (the offending settings are present in the current version's sample unit file too)

borgmatic installation method

Debian package

Borg version

borg 1.2.8

Python version

Python 3.12.3

Database version (if applicable)

No response

Operating system and version

PRETTY_NAME="Ubuntu 24.04.2 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.2 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo

### What I'm trying to do and why When trying to use the ZFS integration to make snapshots of backed up directories, while running Borgmatic through the enclosed systemd unit file, one gets permission errors ### Steps to reproduce 1. Install the ZFS Debian package (they use a systemd unit file consistent with what's provided under sample/systemd/borgmatic.service) 2. Setup ZFS integration in your config file ### Actual behavior ``` systemd-inhibit[1613548]: /dev/zfs and /proc/self/mounts are required. systemd-inhibit[1613548]: Try running 'udevadm trigger' and 'mount -t proc proc /proc' as root. borgmatic[1613220]: CRITICAL nas: Error running actions for repository CRITICAL nas: Command '('zfs', 'list', '-H', '-t', 'filesystem', '-o', 'name,mountpoint,canmount,org.torsion.borgmatic:backup')' returned non-zero exit status 1. borgmatic[1613220]: CRITICAL /etc/borgmatic/config.yaml: An error occurred ``` ``` CRITICAL cannot create snapshots : permission denied CRITICAL Command 'zfs snapshot hdd@borgmatic-1670783' returned non-zero exit status 1. ``` ``` CRITICAL filesystem 'hdd@borgmatic-1674374' can not be mounted: Operation not permitted CRITICAL Command 'mount -t zfs -o ro hdd@borgmatic-1674374 /run/borgmatic/zfs_snapshots/eb72ccdc3f97b208385c/hdd' returned non-zero exit status 1. ``` ### Expected behavior The snapshots should be created successfully ### Other notes / implementation ideas I tracked down the offending unit file settings. The ZFS dataset listing fails due to the `PrivateDevices=yes` line (the borgmatic process loses access to /dev/zfs that way) Snapshotting fails due to the `CapabilityBoundingSet=CAP_DAC_READ_SEARCH CAP_NET_RAW` line. I'm not sure which capabilities the ZFS snapshot process needs, but clearing the setting allowed the snapshots to be made. Mounting snapshots is prevented by the `SystemCallFilter=@system-service` line. Setting it to `SystemCallFilter=@system-service @mount` allowed the backup to proceed. ### borgmatic version 1.9.14 (the offending settings are present in the current version's sample unit file too) ### borgmatic installation method Debian package ### Borg version borg 1.2.8 ### Python version Python 3.12.3 ### Database version (if applicable) _No response_ ### Operating system and version PRETTY_NAME="Ubuntu 24.04.2 LTS" NAME="Ubuntu" VERSION_ID="24.04" VERSION="24.04.2 LTS (Noble Numbat)" VERSION_CODENAME=noble ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" UBUNTU_CODENAME=noble LOGO=ubuntu-logo

witten commented 2025-07-01 00:19:21 +00:00

Owner

Copy Link

Thanks for filing this and providing all the details about the offending system settings. Unfortunately, there's no one-size-fits-all systemd service given all the ways borgmatic can be used. So I think I'm going to treat this as a documentation issue (including potentially the comments in the sample systemd service).

Thanks for filing this and providing all the details about the offending system settings. Unfortunately, there's no one-size-fits-all systemd service given all the ways borgmatic can be used. So I think I'm going to treat this as a documentation issue (including potentially the comments in the sample systemd service).

witten referenced this issue from a commit 2025-07-19 04:33:08 +00:00

Document systemd configuration changes for the ZFS filesystem hook (#1114).

witten commented 2025-07-19 04:34:35 +00:00

Owner

Copy Link

I went ahead and documented this and also added a few relevant comments to the sample systemd service file. Additionally, I added @mount to the SystemCallFilter= setting as you suggested.

Thanks again!

I went ahead and documented this and also added a few relevant comments to the sample systemd service file. Additionally, I added `@mount` to the `SystemCallFilter=` setting as you suggested. Thanks again!

witten closed this issue 2025-07-19 04:34:35 +00:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

command-hook-steps

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

No Label

Milestone

No items

No Milestone

Assignees

Clear assignees

witten

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#1114