borgmatic-collective/borgmatic
11
70
Fork 40
Code Issues 50 Pull Requests 6 Actions 1 Packages Releases 152 Activity

Btrfs snapshots of raw subvolume fails #1043

New Issue
Closed
opened 2025-03-25 04:55:42 +00:00 by lugoues · 9 comments
lugoues commented 2025-03-25 04:55:42 +00:00
Copy Link

What I'm trying to do and why

I am attempting to back up /mnt/storage/@development which is a subvolume of /mnt/storage and is mounted to /home/pbrunner/dev. (commands below).

This is causing borgmatic to try to snapshot /mnt/storage with the command btrfs -v subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-28821/mnt/storage which fails silently with ERROR: Could not open: No such file or directory because /mnt/storage is not a subvolume.

If I change the borgmatic config to backup /home/pbrunner/dev it works as expected. It seems to be due to findmnt is not returning the raw subvolume but only /mnt/storage and /home/pbrunner/dev.

rsync.net: Snapshotting Btrfs subvolumes
rsync.net: /usr/bin/findmnt -t btrfs --json --list
rsync.net: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro
rsync.net: /usr/bin/btrfs property get -t subvol /mnt/storage ro
rsync.net: Creating Btrfs snapshot for /mnt/storage subvolume
rsync.net: /usr/bin/btrfs subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-28821/mnt/storage
Create readonly snapshot of '/mnt/storage' in '/mnt/storage/.borgmatic-snapshot-28821/mnt/storage'

Steps to reproduce

With the following config:

source_directories:
  - /mnt/storage/@development
#  - /home/pbrunner/dev

/mnt/storage is the root volume /mnt/storage/@development is a subvolume, and /home/me/dev is a mounted to the @development subvolume.

If run as is, it will not find any files but if I flip the comment (with no other changes to the config) and run it only on /home/me/dev it adds everything as expected

Actual behavior

λ findmnt -t btrfs --list --json
{
   "filesystems": [
      {
         "target": "/home/pbrunner/dev",
         "source": "/dev/sdb[/@development]",
         "fstype": "btrfs",
         "options": "rw,relatime,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/@development"
      },{
         "target": "/mnt/storage",
         "source": "/dev/sdb",
         "fstype": "btrfs",
         "options": "rw,relatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/"
      }
   ]
}

λ sudo btrfs subvolume list /mnt/storage
ID 256 gen 2249 top level 5 path @development
ID 258 gen 2249 top level 5 path @snapshots

λ clear; sudo ~/.asdf/installs/python/3.9.10/bin/borgmatic create --verbosity 2 --list
No commands to run for pre-everything hook
/etc/borgmatic/config.yaml: BORG_RSH=*** BORG_PASSPHRASE_FD=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** /usr/bin/borg --version --debug --show-rc
/etc/borgmatic/config.yaml: Borg 1.4.0
remote: Running actions for repository
remote: No commands to run for pre-actions hook
remote: No commands to run for pre-backup hook
remote: Creating archive
remote: Using runtime directory /tmp/borgmatic-vlgin5u0/borgmatic
remote: Calling bootstrap hook function remove_data_source_dumps
remote: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
remote: Calling btrfs hook function remove_data_source_dumps
remote: /usr/bin/findmnt -t btrfs --json --list
remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro
remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro
remote: Looking for snapshots to remove in /mnt/storage/.borgmatic-snapshot-*/mnt/storage
remote: Looking for snapshots to remove in /home/pbrunner/dev/.borgmatic-snapshot-*/home/pbrunner/dev
remote: Calling lvm hook function remove_data_source_dumps
remote: Calling mariadb hook function remove_data_source_dumps
remote: Removing MariaDB data source dumps
remote: Calling mongodb hook function remove_data_source_dumps
remote: Removing MongoDB data source dumps
remote: Calling mysql hook function remove_data_source_dumps
remote: Removing MySQL data source dumps
remote: Calling postgresql hook function remove_data_source_dumps
remote: Removing PostgreSQL data source dumps
remote: Calling sqlite hook function remove_data_source_dumps
remote: Removing SQLite data source dumps
remote: Calling zfs hook function remove_data_source_dumps
remote: Calling bootstrap hook function dump_data_sources
remote: Calling btrfs hook function dump_data_sources
remote: Snapshotting Btrfs subvolumes
remote: /usr/bin/findmnt -t btrfs --json --list
remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro
remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro
remote: Creating Btrfs snapshot for /mnt/storage subvolume
remote: /usr/bin/btrfs subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-43244/mnt/storage
Create readonly snapshot of '/mnt/storage' in '/mnt/storage/.borgmatic-snapshot-43244/mnt/storage'
remote: Writing patterns to /tmp/borgmatic-vlgin5u0/borgmatic/tmpclj1f1gn:
R /mnt/storage/.borgmatic-snapshot-43244/./mnt/storage/@development
! fm:*.pyc
! fm:*/node_modules
! fm:*/.terraform
! fm:*/.pants.d
! fm:*/.cache
! fm:*/*.pnpm-store
! fm:*/.n
R /etc/borgmatic/config.yaml
R /tmp/borgmatic-vlgin5u0/./borgmatic/bootstrap
! fm:/mnt/storage/.borgmatic-snapshot-43244/mnt/storage/.borgmatic-snapshot-43244
remote: BORG_RSH=*** BORG_PASSPHRASE_FD=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** /usr/bin/borg create --patterns-from /tmp/borgmatic-vlgin5u0/borgmatic/tmpclj1f1gn --remote-path borg14 --list --filter AMEx- --debug --show-rc ssh://xxxxxxxx/./thalia-peter-dev::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}   using builtin fallback logging configuration
33 self tests completed in 0.05 seconds
SSH command line: ['ssh', '-i', '/etc/borgmatic/id_ed25519', '-o', 'IdentitiesOnly=yes', 'xxxxxxxx', 'borg14', 'serve', '--debug']                                                                                                               Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./thalia-peter-dev'
Remote: Resolved repository path to '/data3/home/de4219/de4219s1/thalia-peter-dev'
Remote: Verified integrity of /data3/home/de4219/de4219s1/thalia-peter-dev/index.333
TAM-verified manifest
security: read previous location 'ssh://xxxxxxxx/./thalia-peter-dev'
security: read manifest timestamp '2025-03-25T04:28:32.344892'
security: determined newest manifest timestamp as 2025-03-25T04:28:32.344892
security: repository checks ok, allowing access
Creating archive at "ssh://xxxxxxxx/./thalia-peter-dev::thalia-peter-dev-2025-03-25T00:43:24.873703"
Verified integrity of /root/.cache/borg/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f/chunks
Reading files cache ...
Verified integrity of /root/.cache/borg/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f/files
security: read previous location 'ssh://xxxxxxxx/./thalia-peter-dev'
security: read manifest timestamp '2025-03-25T04:28:32.344892'
security: determined newest manifest timestamp as 2025-03-25T04:28:32.344892
security: repository checks ok, allowing access
Processing files ...
A /tmp/borgmatic-vlgin5u0/borgmatic/bootstrap/manifest.json
Remote: Cleaned up 0 uncommitted segment files (== everything after segment 333).                                                Remote: Verified integrity of /data3/home/de4219/de4219s1/thalia-peter-dev/hints.333
Remote: check_free_space: required bytes 587209418, free bytes 228156945920
security: saving state for 56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f to /root/.config/borg/security/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f
security: current location   ssh://xxxxxxxx/./thalia-peter-dev
security: key type           5
security: manifest timestamp 2025-03-25T04:43:28.199635
RemoteRepository: 4.20 kB bytes sent, 5.66 kB bytes received, 11 messages sent
terminating with success status, rc 0
remote: Calling bootstrap hook function remove_data_source_dumps
remote: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
remote: Removing bootstrap manifest at /tmp/borgmatic-vlgin5u0/borgmatic/bootstrap/manifest.json
remote: Calling btrfs hook function remove_data_source_dumps
remote: /usr/bin/findmnt -t btrfs --json --list
remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro
remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro
remote: Looking for snapshots to remove in /mnt/storage/.borgmatic-snapshot-*/mnt/storage
remote: Deleting Btrfs snapshot /mnt/storage/.borgmatic-snapshot-43244/mnt/storage
remote: /usr/bin/btrfs subvolume delete /mnt/storage/.borgmatic-snapshot-43244/mnt/storage
Delete subvolume 982 (no-commit): '/mnt/storage/.borgmatic-snapshot-43244/mnt/storage'
remote: Looking for snapshots to remove in /home/pbrunner/dev/.borgmatic-snapshot-*/home/pbrunner/dev
remote: Calling lvm hook function remove_data_source_dumps
remote: Calling mariadb hook function remove_data_source_dumps
remote: Removing MariaDB data source dumps
remote: Calling mongodb hook function remove_data_source_dumps
remote: Removing MongoDB data source dumps
remote: Calling mysql hook function remove_data_source_dumps
remote: Removing MySQL data source dumps
remote: Calling postgresql hook function remove_data_source_dumps
remote: Removing PostgreSQL data source dumps
remote: Calling sqlite hook function remove_data_source_dumps
remote: Removing SQLite data source dumps
remote: Calling zfs hook function remove_data_source_dumps
remote: No commands to run for post-backup hook
remote: No commands to run for post-actions hook
No commands to run for post-everything hook

summary:
/etc/borgmatic/config.yaml: Loading configuration file
Successfully ran configuration file

~                                                                                 5s ﴱ thalia pbrunner@thalia-peter-dev
λ

Expected behavior

Include all files found in /mnt/storage/@development

Other notes / implementation ideas

One potential solution would be to check if the source directory is itself a subvolume with btrfs subvolume show /mnt/storage/@development before searching up the tree to find a parent (sub)volume.

borgmatic version

1.9.14

borgmatic installation method

apt

Borg version

borg 1.4.0

Python version

λ python3 --version Python 3.9.10

Database version (if applicable)

No response

Operating system and version

Debian GNU/Linux trixie/sid

### What I'm trying to do and why I am attempting to back up `/mnt/storage/@development` which is a subvolume of `/mnt/storage` and is mounted to `/home/pbrunner/dev`. (commands below). This is causing borgmatic to try to snapshot `/mnt/storage` with the command `btrfs -v subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-28821/mnt/storage` which fails silently with `ERROR: Could not open: No such file or directory` because `/mnt/storage` is not a subvolume. If I change the borgmatic config to backup `/home/pbrunner/dev` it works as expected. It seems to be due to findmnt is not returning the raw subvolume but only `/mnt/storage` and `/home/pbrunner/dev`. ``` rsync.net: Snapshotting Btrfs subvolumes rsync.net: /usr/bin/findmnt -t btrfs --json --list rsync.net: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro rsync.net: /usr/bin/btrfs property get -t subvol /mnt/storage ro rsync.net: Creating Btrfs snapshot for /mnt/storage subvolume rsync.net: /usr/bin/btrfs subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-28821/mnt/storage Create readonly snapshot of '/mnt/storage' in '/mnt/storage/.borgmatic-snapshot-28821/mnt/storage' ``` ### Steps to reproduce With the following config: ``` source_directories: - /mnt/storage/@development # - /home/pbrunner/dev ``` `/mnt/storage` is the root volume `/mnt/storage/@development` is a subvolume, and `/home/me/dev` is a mounted to the `@development` subvolume. If run as is, it will not find any files but if I flip the comment (with no other changes to the config) and run it only on `/home/me/dev` it adds everything as expected ### Actual behavior ``` λ findmnt -t btrfs --list --json { "filesystems": [ { "target": "/home/pbrunner/dev", "source": "/dev/sdb[/@development]", "fstype": "btrfs", "options": "rw,relatime,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/@development" },{ "target": "/mnt/storage", "source": "/dev/sdb", "fstype": "btrfs", "options": "rw,relatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/" } ] } ``` ``` λ sudo btrfs subvolume list /mnt/storage ID 256 gen 2249 top level 5 path @development ID 258 gen 2249 top level 5 path @snapshots ``` ``` λ clear; sudo ~/.asdf/installs/python/3.9.10/bin/borgmatic create --verbosity 2 --list No commands to run for pre-everything hook /etc/borgmatic/config.yaml: BORG_RSH=*** BORG_PASSPHRASE_FD=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** /usr/bin/borg --version --debug --show-rc /etc/borgmatic/config.yaml: Borg 1.4.0 remote: Running actions for repository remote: No commands to run for pre-actions hook remote: No commands to run for pre-backup hook remote: Creating archive remote: Using runtime directory /tmp/borgmatic-vlgin5u0/borgmatic remote: Calling bootstrap hook function remove_data_source_dumps remote: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap remote: Calling btrfs hook function remove_data_source_dumps remote: /usr/bin/findmnt -t btrfs --json --list remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro remote: Looking for snapshots to remove in /mnt/storage/.borgmatic-snapshot-*/mnt/storage remote: Looking for snapshots to remove in /home/pbrunner/dev/.borgmatic-snapshot-*/home/pbrunner/dev remote: Calling lvm hook function remove_data_source_dumps remote: Calling mariadb hook function remove_data_source_dumps remote: Removing MariaDB data source dumps remote: Calling mongodb hook function remove_data_source_dumps remote: Removing MongoDB data source dumps remote: Calling mysql hook function remove_data_source_dumps remote: Removing MySQL data source dumps remote: Calling postgresql hook function remove_data_source_dumps remote: Removing PostgreSQL data source dumps remote: Calling sqlite hook function remove_data_source_dumps remote: Removing SQLite data source dumps remote: Calling zfs hook function remove_data_source_dumps remote: Calling bootstrap hook function dump_data_sources remote: Calling btrfs hook function dump_data_sources remote: Snapshotting Btrfs subvolumes remote: /usr/bin/findmnt -t btrfs --json --list remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro remote: Creating Btrfs snapshot for /mnt/storage subvolume remote: /usr/bin/btrfs subvolume snapshot -r /mnt/storage /mnt/storage/.borgmatic-snapshot-43244/mnt/storage Create readonly snapshot of '/mnt/storage' in '/mnt/storage/.borgmatic-snapshot-43244/mnt/storage' remote: Writing patterns to /tmp/borgmatic-vlgin5u0/borgmatic/tmpclj1f1gn: R /mnt/storage/.borgmatic-snapshot-43244/./mnt/storage/@development ! fm:*.pyc ! fm:*/node_modules ! fm:*/.terraform ! fm:*/.pants.d ! fm:*/.cache ! fm:*/*.pnpm-store ! fm:*/.n R /etc/borgmatic/config.yaml R /tmp/borgmatic-vlgin5u0/./borgmatic/bootstrap ! fm:/mnt/storage/.borgmatic-snapshot-43244/mnt/storage/.borgmatic-snapshot-43244 remote: BORG_RSH=*** BORG_PASSPHRASE_FD=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** /usr/bin/borg create --patterns-from /tmp/borgmatic-vlgin5u0/borgmatic/tmpclj1f1gn --remote-path borg14 --list --filter AMEx- --debug --show-rc ssh://xxxxxxxx/./thalia-peter-dev::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} using builtin fallback logging configuration 33 self tests completed in 0.05 seconds SSH command line: ['ssh', '-i', '/etc/borgmatic/id_ed25519', '-o', 'IdentitiesOnly=yes', 'xxxxxxxx', 'borg14', 'serve', '--debug'] Remote: using builtin fallback logging configuration Remote: borg selftest disabled via BORG_SELFTEST env variable Remote: using builtin fallback logging configuration Remote: Initialized logging system for JSON-based protocol Remote: Resolving repository path b'/./thalia-peter-dev' Remote: Resolved repository path to '/data3/home/de4219/de4219s1/thalia-peter-dev' Remote: Verified integrity of /data3/home/de4219/de4219s1/thalia-peter-dev/index.333 TAM-verified manifest security: read previous location 'ssh://xxxxxxxx/./thalia-peter-dev' security: read manifest timestamp '2025-03-25T04:28:32.344892' security: determined newest manifest timestamp as 2025-03-25T04:28:32.344892 security: repository checks ok, allowing access Creating archive at "ssh://xxxxxxxx/./thalia-peter-dev::thalia-peter-dev-2025-03-25T00:43:24.873703" Verified integrity of /root/.cache/borg/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f/chunks Reading files cache ... Verified integrity of /root/.cache/borg/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f/files security: read previous location 'ssh://xxxxxxxx/./thalia-peter-dev' security: read manifest timestamp '2025-03-25T04:28:32.344892' security: determined newest manifest timestamp as 2025-03-25T04:28:32.344892 security: repository checks ok, allowing access Processing files ... A /tmp/borgmatic-vlgin5u0/borgmatic/bootstrap/manifest.json Remote: Cleaned up 0 uncommitted segment files (== everything after segment 333). Remote: Verified integrity of /data3/home/de4219/de4219s1/thalia-peter-dev/hints.333 Remote: check_free_space: required bytes 587209418, free bytes 228156945920 security: saving state for 56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f to /root/.config/borg/security/56b91018dddca0885780afb53f634738a068b3a8395ba6c265805f41db224e4f security: current location ssh://xxxxxxxx/./thalia-peter-dev security: key type 5 security: manifest timestamp 2025-03-25T04:43:28.199635 RemoteRepository: 4.20 kB bytes sent, 5.66 kB bytes received, 11 messages sent terminating with success status, rc 0 remote: Calling bootstrap hook function remove_data_source_dumps remote: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap remote: Removing bootstrap manifest at /tmp/borgmatic-vlgin5u0/borgmatic/bootstrap/manifest.json remote: Calling btrfs hook function remove_data_source_dumps remote: /usr/bin/findmnt -t btrfs --json --list remote: /usr/bin/btrfs property get -t subvol /home/pbrunner/dev ro remote: /usr/bin/btrfs property get -t subvol /mnt/storage ro remote: Looking for snapshots to remove in /mnt/storage/.borgmatic-snapshot-*/mnt/storage remote: Deleting Btrfs snapshot /mnt/storage/.borgmatic-snapshot-43244/mnt/storage remote: /usr/bin/btrfs subvolume delete /mnt/storage/.borgmatic-snapshot-43244/mnt/storage Delete subvolume 982 (no-commit): '/mnt/storage/.borgmatic-snapshot-43244/mnt/storage' remote: Looking for snapshots to remove in /home/pbrunner/dev/.borgmatic-snapshot-*/home/pbrunner/dev remote: Calling lvm hook function remove_data_source_dumps remote: Calling mariadb hook function remove_data_source_dumps remote: Removing MariaDB data source dumps remote: Calling mongodb hook function remove_data_source_dumps remote: Removing MongoDB data source dumps remote: Calling mysql hook function remove_data_source_dumps remote: Removing MySQL data source dumps remote: Calling postgresql hook function remove_data_source_dumps remote: Removing PostgreSQL data source dumps remote: Calling sqlite hook function remove_data_source_dumps remote: Removing SQLite data source dumps remote: Calling zfs hook function remove_data_source_dumps remote: No commands to run for post-backup hook remote: No commands to run for post-actions hook No commands to run for post-everything hook summary: /etc/borgmatic/config.yaml: Loading configuration file Successfully ran configuration file ~ 5s ﴱ thalia pbrunner@thalia-peter-dev λ ``` ### Expected behavior Include all files found in /mnt/storage/@development ### Other notes / implementation ideas One potential solution would be to check if the source directory is itself a subvolume with ` btrfs subvolume show /mnt/storage/@development` before searching up the tree to find a parent (sub)volume. ### borgmatic version 1.9.14 ### borgmatic installation method apt ### Borg version borg 1.4.0 ### Python version λ python3 --version Python 3.9.10 ### Database version (if applicable) _No response_ ### Operating system and version Debian GNU/Linux trixie/sid
witten commented 2025-03-25 17:17:49 +00:00
Owner
Copy Link

This might just be a documentation issue. Or it might be me misunderstanding Btrfs. But the intent of the feature is that you list Btrfs subvolume mount points in source_directories. The idea is that source_directories is a list of actual paths to backup, and if one of them just happens to be mounted from a Btrfs subvolume, then borgmatic will transparently take advantage of that fact and make/use a snapshot instead of the mounted source files.

Anyway, let me know your thoughts on this. Was it just that you expected /mnt/storage/@development to work and were surprised when it didn't, or is there an argument for specifying the subvolume name rather than the mount point?

This might just be a documentation issue. Or it might be me misunderstanding Btrfs. But the intent of the feature is that you list Btrfs subvolume *mount points* in `source_directories`. The idea is that `source_directories` is a list of actual paths to backup, and if one of them just happens to be mounted from a Btrfs subvolume, then borgmatic will transparently take advantage of that fact and make/use a snapshot instead of the mounted source files. Anyway, let me know your thoughts on this. Was it just that you expected `/mnt/storage/@development` to work and were surprised when it didn't, or is there an argument for specifying the subvolume name rather than the mount point?
lugoues commented 2025-03-25 17:42:17 +00:00
Author
Copy Link

Ah that would be me misunderstanding the documentation then. I was indeed expexting /mnt/storage/@development to work.

There are two recommended/common patterns for subvolumes in btrfs, nested and flat (some mix them as well). You are describing the nested method where you mount your btrfs root to a point and create snapshots as needed in the tree. On the other hand I was describing the flat approach where you create subvolumes within the btrfs root and then mount those subvolumes to specific points.

The ability to specify the subvolume, instead of the mount, would support the flat layout and follow with that pattern. Not only do you not know all the places it is mounted without investigation, it would break the general idea of the subvolume at /mnt/storage/@development being the primary object.

ref: https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/SysadminGuide.html#Layout

Ah that would be me misunderstanding the documentation then. I was indeed expexting `/mnt/storage/@development` to work. There are two recommended/common patterns for subvolumes in btrfs, nested and flat (some mix them as well). You are describing the nested method where you mount your btrfs root to a point and create snapshots as needed in the tree. On the other hand I was describing the flat approach where you create subvolumes within the btrfs root and then mount those subvolumes to specific points. The ability to specify the subvolume, instead of the mount, would support the flat layout and follow with that pattern. Not only do you not know all the places it is mounted without investigation, it would break the general idea of the subvolume at `/mnt/storage/@development` being the primary object. ref: https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/SysadminGuide.html#Layout
witten commented 2025-03-26 19:57:44 +00:00
Owner
Copy Link

Ah that would be me misunderstanding the documentation then. I was indeed expexting /mnt/storage/@development to work.

I've updated the documentation to hopefully more accurately explain the current mount-point-oriented behavior.

The ability to specify the subvolume, instead of the mount, would support the flat layout and follow with that pattern. Not only do you not know all the places it is mounted without investigation, it would break the general idea of the subvolume at /mnt/storage/@development being the primary object.

Okay, so if I'm understanding what you're saying, it's possible to use the current borgmatic Btrfs hook with the flat layout, but it requires knowing/using a subvolume's mount point—which is not the primary object users think of when trying to snapshot their flat-layout subvolumes.

So let's say that the Btrfs hook adds support for specifying such unmounted subvolumes in source_directories. Then what's the extract story like? For instance, right now the idea is that you backup a Btrfs subvolume that's mounted at /home/pbrunner/dev, it gets stored in the Borg archive at /home/pbrunner/dev, and then you can easily find and extract it back to /home/pbrunner/dev later on. But what's the expectation for that user experience if borgmatic is backing up in terms of an unmounted subvolume path like /mnt/storage/@development and storing it as such in an archive? How would the user extract it?

I'm also not sure how I feel about allowing "non-real" file paths in source_directories. I can kind of pretend it's okay when implicitly snapshotting a "real" directory with actual files in it. Maybe a list of subvolumes to backup would make more sense as a new option under the btrfs: config?

A related thought: The existing borgmatic ZFS data source hook supports omitting ZFS datasets from source_directories but requesting that they be backed up by adding an application-specific borgmatic property to them, but I don't think btrfs property set supports application-specific keys. There might be a way to simulate that with setfattr though.

> Ah that would be me misunderstanding the documentation then. I was indeed expexting /mnt/storage/@development to work. I've updated the documentation to hopefully more accurately explain the current mount-point-oriented behavior. > The ability to specify the subvolume, instead of the mount, would support the flat layout and follow with that pattern. Not only do you not know all the places it is mounted without investigation, it would break the general idea of the subvolume at /mnt/storage/@development being the primary object. Okay, so if I'm understanding what you're saying, it's possible to use the current borgmatic Btrfs hook with the flat layout, but it requires knowing/using a subvolume's mount point—which is not the primary object users think of when trying to snapshot their flat-layout subvolumes. So let's say that the Btrfs hook adds support for specifying such unmounted subvolumes in `source_directories`. Then what's the extract story like? For instance, right now the idea is that you backup a Btrfs subvolume that's mounted at `/home/pbrunner/dev`, it gets stored in the Borg archive at `/home/pbrunner/dev`, and then you can easily find and extract it back to `/home/pbrunner/dev` later on. But what's the expectation for that user experience if borgmatic is backing up in terms of an unmounted subvolume path like `/mnt/storage/@development` and storing it as such in an archive? How would the user extract it? I'm also not sure how I feel about allowing "non-real" file paths in `source_directories`. I can kind of pretend it's okay when implicitly snapshotting a "real" directory with actual files in it. Maybe a list of subvolumes to backup would make more sense as a new option under the `btrfs:` config? A related thought: The existing borgmatic ZFS data source hook supports omitting ZFS datasets from `source_directories` but requesting that they be backed up by adding an application-specific borgmatic property to them, but I don't think `btrfs property set` supports application-specific keys. There might be a way to simulate that with `setfattr` though.
lugoues commented 2025-03-26 22:32:07 +00:00
Author
Copy Link

So let's say that the Btrfs hook adds support for specifying such unmounted subvolumes in source_directories. Then what's the extract story like? For instance, right now the idea is that you backup a Btrfs subvolume that's mounted at /home/pbrunner/dev, it gets stored in the Borg archive at /home/pbrunner/dev, and then you can easily find and extract it back to /home/pbrunner/dev later on. But what's the expectation for that user experience if borgmatic is backing up in terms of an unmounted subvolume path like /mnt/storage/@development and storing it as such in an archive? How would the user extract it?

I'm also not sure how I feel about allowing "non-real" file paths in source_directories. I can kind of pretend it's okay when implicitly snapshotting a "real" directory with actual files in it. Maybe a list of subvolumes to backup would make more sense as a new option under the btrfs: config?

I'm not 100% versed with ZFS so I'm not sure how much the following is the same.

In btrfs subvolumes are valid paths and actually contain real data (it's just a POSIX directory). I'd expect borgmatic it to store like it normally would a directory /mnt/storage/@development and to restore back to the same place. It's also 100% valid to create a subvolume and never mount it. Given that I'm not sure btrfs: needs any extra config.

IMO having the btrfs hook do a first pass of source_directories using btrfs subvolume show <path>(and checking for level 5 prior to running findmnt <path> would solve this problem and be essentially transparent to users. Point borgmatic to a directory and it will snapshot it if possible or find a parent to snapshot.

> So let's say that the Btrfs hook adds support for specifying such unmounted subvolumes in `source_directories`. Then what's the extract story like? For instance, right now the idea is that you backup a Btrfs subvolume that's mounted at `/home/pbrunner/dev`, it gets stored in the Borg archive at `/home/pbrunner/dev`, and then you can easily find and extract it back to `/home/pbrunner/dev` later on. But what's the expectation for that user experience if borgmatic is backing up in terms of an unmounted subvolume path like `/mnt/storage/@development` and storing it as such in an archive? How would the user extract it? > > I'm also not sure how I feel about allowing "non-real" file paths in `source_directories`. I can kind of pretend it's okay when implicitly snapshotting a "real" directory with actual files in it. Maybe a list of subvolumes to backup would make more sense as a new option under the `btrfs:` config? I'm not 100% versed with ZFS so I'm not sure how much the following is the same. In btrfs subvolumes *are* valid paths and actually contain real data (it's just a POSIX directory). I'd expect borgmatic it to store like it normally would a directory `/mnt/storage/@development` and to restore back to the same place. It's also 100% valid to create a subvolume and never mount it. Given that I'm not sure `btrfs:` needs any extra config. IMO having the btrfs hook do a first pass of source_directories using `btrfs subvolume show <path>`(and checking for `level 5` prior to running `findmnt <path>` would solve this problem and be essentially transparent to users. Point borgmatic to a directory and it will snapshot it if possible or find a parent to snapshot.
witten commented 2025-04-04 00:20:13 +00:00
Owner
Copy Link

Got it, thanks for the explanation. Given that the unmounted Btrfs subvolumes have visible files in them, your proposal makes a lot more sense to me now. When I get a chance, I'll have to look into how feasible it would be to implement this, especially given the current mount point reliance in the existing Btrfs hook.

Got it, thanks for the explanation. Given that the unmounted Btrfs subvolumes have visible files in them, your proposal makes a lot more sense to me now. When I get a chance, I'll have to look into how feasible it would be to implement this, especially given the current mount point reliance in the existing Btrfs hook.
lugoues commented 2025-05-23 04:52:12 +00:00
Author
Copy Link

Thanks!

For those who come after,

A short term hack I used was to use findmnt_command to override findmnt to a bash script that outputs the correct info and it works as expected.

#!/usr/bin/env bash
echo '
{
   "filesystems": [
      { "target": "/path/to/subvolume" }
    ]
}
'
Thanks! For those who come after, A short term hack I used was to use `findmnt_command` to override findmnt to a bash script that outputs the correct info and it works as expected. ``` #!/usr/bin/env bash echo ' { "filesystems": [ { "target": "/path/to/subvolume" } ] } ' ```
👍 1
witten commented 2025-06-25 22:50:03 +00:00
Owner
Copy Link

FYI I have a proposed WIP implementation of this here: #1113

FYI I have a proposed WIP implementation of this here: https://projects.torsion.org/borgmatic-collective/borgmatic/pulls/1113
witten referenced this issue from a commit 2025-06-28 02:21:28 +00:00
Clarify NEWS entry (#1043).
witten commented 2025-06-28 02:28:59 +00:00
Owner
Copy Link

This is implemented in main and will be part of the next release. The approach I used is to start with findmnt as a way to get a global list of Btrfs mount points (each of which is either a Btrfs subvolume path or a separate mount point), and then call btrfs subvolume list on each one to discover all the actual Btrfs subvolume paths.

As per your use case, that should allow you to include /mnt/storage/@development in source_directories such that it gets snapshotted and backed up, even if it's mounted elsewhere. And if you find out that that's not the case, please let me know.

This is implemented in main and will be part of the next release. The approach I used is to start with `findmnt` as a way to get a global list of Btrfs mount points (each of which is either a Btrfs subvolume path or a separate mount point), and then call `btrfs subvolume list` on each one to discover all the actual Btrfs subvolume paths. As per your use case, that should allow you to include `/mnt/storage/@development` in `source_directories` such that it gets snapshotted and backed up, even if it's mounted elsewhere. And if you find out that that's *not* the case, please let me know.
witten commented 2025-07-01 01:06:55 +00:00
Owner
Copy Link

Released in borgmatic 2.0.7! Feedback welcome.

Released in borgmatic 2.0.7! Feedback welcome.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
witten
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#1043
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?