borgmatic-collective/borgmatic

Watch

Fork 57

LVM snapshots feature fails to update patterns paths without a leading slash #1226

New issue

Open

opened 2026-01-06 13:22:10 +00:00 by akostadinov · 5 comments

akostadinov commented

2026-01-06 13:22:10 +00:00

What I'm trying to do and why

extracted from #962

When the pattern is specified according to Borg docs as an absolute path but without a leading /, then appending the snapshot path fails.

Example, if I have specified a pattern pf:home/myuser/.local/share/baloo, then I see in borgmatic verbose log that it ends up ! pf:home/myuser/.local/share/baloo.

Here a valid workaround is to specify the pattern as pf:/home/myuser/.local/share/baloo. According to man borg the patterns should not start with a leading / but that leading slash is removed automatically so the workaround is viable. Still it will be correct for borgmatic to handle patterns without a leading slash because they seem to be the canonical format.

So with pf:/home/myuser/.local/share/baloo, that is properly later translated to ! pf:/run/user/0/borgmatic/lvm_snapshots/f667480a34df1550a601/./home/myuser/.local/share/baloo.

Actually current approach may fail with some regular expressions, because for the re: pattern I don't see Borg removing leading slashes. I didn't test that though.

Discussion:

Ah, this is a tricky one. If you don't include a leading slash, borgmatic can't determine whether you're specifying a relative path or an absolute path, and so it can't reliably perform rewrites.

As far as I read and my testing goes, there are no relative paths in Borg. All paths are absolute and inmost pattern types, a leading / is removed. So what would a "relative" path mean in this context and do relative paths have a special borgmatic meaning?

Steps to reproduce

Specify excludes as absolute paths without a leading slash (according to man borg this is what I see as the canonical approach).
run borgmatic create

Actual behavior

...
/tmp/tmp.4CvGhzfZOu: Writing patterns to /run/user/1001/borgmatic/tmpmy9dnram:
R /tmp/tmp.WkwhRhMrLu
+ /tmp/tmp.WkwhRhMrLu
R /run/user/1001/./borgmatic/bootstrap
+ /run/user/1001/./borgmatic/bootstrap
R /tmp/tmp.C5h5f2f2j8
! fm:tmp/tmp.C5h5f2f2j8/home/user/testdir
...

Expected behavior

...
/tmp/tmp.4CvGhzfZOu: Writing patterns to /run/user/1001/borgmatic/tmpmy9dnram:
R /tmp/tmp.WkwhRhMrLu
+ /tmp/tmp.WkwhRhMrLu
R /run/user/1001/./borgmatic/bootstrap
+ /run/user/1001/./borgmatic/bootstrap
R /tmp/tmp.C5h5f2f2j8
! fm:PATH_TO_LVM_SNAPSHOT_MOUNT_POINT/./tmp/tmp.C5h5f2f2j8/home/user/testdir
...

Other notes / implementation ideas

Perhaps compare pattern prefixes with and without a leading slash (i.e. do not consider the leading slash as significant)

borgmatic version

2.0.13

borgmatic installation method

RPM

Borg version

borg 1.4.2

Python version

Python 3.13.9

Database version (if applicable)

No response

Operating system and version

Fedora Linux 42

### What I'm trying to do and why extracted from #962 When the pattern is specified **according to Borg docs** as an absolute path but **without** a leading `/`, then appending the snapshot path fails. Example, if I have specified a pattern `pf:home/myuser/.local/share/baloo`, then I see in borgmatic verbose log that it ends up `! pf:home/myuser/.local/share/baloo`. Here a valid workaround is to specify the pattern as `pf:/home/myuser/.local/share/baloo`. According to `man borg` the patterns should not start with a leading `/` but that leading slash is removed automatically so the workaround is viable. Still it will be correct for borgmatic to handle patterns without a leading slash because they seem to be the canonical format. So with `pf:/home/myuser/.local/share/baloo`, that is properly later translated to `! pf:/run/user/0/borgmatic/lvm_snapshots/f667480a34df1550a601/./home/myuser/.local/share/baloo`. Actually current approach may fail with some regular expressions, because for the `re:` pattern I don't see Borg removing leading slashes. I didn't test that though. Discussion: > Ah, this is a tricky one. If you don't include a leading slash, borgmatic can't determine whether you're specifying a relative path or an absolute path, and so it can't reliably perform rewrites. As far as I read and my testing goes, there are no relative paths in Borg. All paths are absolute and inmost pattern types, a leading `/` is removed. So what would a "relative" path mean in this context and do relative paths have a special borgmatic meaning? ### Steps to reproduce - Specify excludes as absolute paths without a leading slash (according to `man borg` this is what I see as the canonical approach). - run borgmatic create ### Actual behavior ``` ... /tmp/tmp.4CvGhzfZOu: Writing patterns to /run/user/1001/borgmatic/tmpmy9dnram: R /tmp/tmp.WkwhRhMrLu + /tmp/tmp.WkwhRhMrLu R /run/user/1001/./borgmatic/bootstrap + /run/user/1001/./borgmatic/bootstrap R /tmp/tmp.C5h5f2f2j8 ! fm:tmp/tmp.C5h5f2f2j8/home/user/testdir ... ``` ### Expected behavior ``` ... /tmp/tmp.4CvGhzfZOu: Writing patterns to /run/user/1001/borgmatic/tmpmy9dnram: R /tmp/tmp.WkwhRhMrLu + /tmp/tmp.WkwhRhMrLu R /run/user/1001/./borgmatic/bootstrap + /run/user/1001/./borgmatic/bootstrap R /tmp/tmp.C5h5f2f2j8 ! fm:PATH_TO_LVM_SNAPSHOT_MOUNT_POINT/./tmp/tmp.C5h5f2f2j8/home/user/testdir ... ``` ### Other notes / implementation ideas Perhaps compare pattern prefixes with and without a leading slash (i.e. do not consider the leading slash as significant) ### borgmatic version 2.0.13 ### borgmatic installation method RPM ### Borg version borg 1.4.2 ### Python version Python 3.13.9 ### Database version (if applicable) _No response_ ### Operating system and version Fedora Linux 42

akostadinov referenced this issue

2026-01-06 13:22:49 +00:00

LVM snapshots and exclude list #962

witten commented

2026-02-05 19:20:32 +00:00

Owner

As far as I read and my testing goes, there are no relative paths in Borg. All paths are absolute and inmost pattern types, a leading / is removed. So what would a "relative" path mean in this context and do relative paths have a special borgmatic meaning?

That's mostly correct. There are no relative paths within Borg archives, but there are in borgmatic. borgmatic has a working_directory option that allows you to use relative paths in your source_directories or patterns, which then get adjusted based on the working_directory value before getting passed to Borg.

Additionally, even if working_directory isn't set, you can still feed relative paths to Borg (relative to the current working directory where Borg was run), which get stored without any absolute path prefix. And then the archive itself doesn't distinguish between absolute and relative paths once stored. For example:

$ cd /home/witten/Downloads/tmp
$ borg create test.borg::archive test.yaml
$ borg list test.borg::archive
-rw------- witten witten    46495 Thu, 2026-02-05 10:37:55 test.yaml

The file isn't stored in the archive as /home/witten/Downloads/tmp/test.yaml; it's stored only as test.yaml.

So, getting to the problem you've identified, the challenge is that any particular pattern given to borgmatic might be a relative or absolute path.

Perhaps compare pattern prefixes with and without a leading slash (i.e. do not consider the leading slash as significant)

The edge case I can think of is a relative path pattern that's not referring to the same path as the snapshot. For instance, if you cd to /some/path and run borgmatic with and a relative pattern of home/myuser/.local/share/baloo, that would mean that the actual absolute path is /some/path/home/myuser/.local/share/baloo, which shouldn't necessarily match the snapshot path.

In fact, thinking about this a little more, I'm not sure how a path like pf:home/myuser/.local/share/baloo even works for you (putting aside snapshots altogether) unless you run borgmatic from /. Same thing with plain Borg—I'm not sure how a path like that would work unless you run Borg from the / directory. Example:

cd /home/witten/Downloads/tmp
$ borg create test.borg::archive2 etc
etc: stat: [Errno 2] No such file or directory: 'etc'

So what directory are you in when you're running borgmatic?

> As far as I read and my testing goes, there are no relative paths in Borg. All paths are absolute and inmost pattern types, a leading `/` is removed. So what would a "relative" path mean in this context and do relative paths have a special borgmatic meaning? That's mostly correct. There are no relative paths within Borg archives, but there are in borgmatic. borgmatic has a [`working_directory` option](https://torsion.org/borgmatic/reference/configuration/#working_directory-option) that allows you to use relative paths in your `source_directories` or `patterns`, which then get adjusted based on the `working_directory` value before getting passed to Borg. Additionally, even if `working_directory` isn't set, you can still feed relative paths to Borg (relative to the current working directory where Borg was run), which get stored without any absolute path prefix. And then the archive itself doesn't distinguish between absolute and relative paths once stored. For example: ```bash $ cd /home/witten/Downloads/tmp $ borg create test.borg::archive test.yaml $ borg list test.borg::archive -rw------- witten witten 46495 Thu, 2026-02-05 10:37:55 test.yaml ``` The file isn't stored in the archive as `/home/witten/Downloads/tmp/test.yaml`; it's stored only as `test.yaml`. So, getting to the problem you've identified, the challenge is that any particular pattern given to borgmatic might be a relative or absolute path. > Perhaps compare pattern prefixes with and without a leading slash (i.e. do not consider the leading slash as significant) The edge case I can think of is a relative path pattern that's *not* referring to the same path as the snapshot. For instance, if you cd to `/some/path` and run borgmatic with and a relative pattern of `home/myuser/.local/share/baloo`, that would mean that the actual absolute path is `/some/path/home/myuser/.local/share/baloo`, which shouldn't necessarily match the snapshot path. In fact, thinking about this a little more, I'm not sure how a path like `pf:home/myuser/.local/share/baloo` even works for you (putting aside snapshots altogether) unless you run borgmatic from `/`. Same thing with plain Borg—I'm not sure how a path like that would work unless you run Borg from the `/` directory. Example: ```bash cd /home/witten/Downloads/tmp $ borg create test.borg::archive2 etc etc: stat: [Errno 2] No such file or directory: 'etc' ``` So what directory are you in when you're running borgmatic?

akostadinov commented

2026-02-09 22:41:06 +00:00

Author

from man borg

   Fnmatch, selector fm:
          This is the default style for --exclude and --exclude-from.  These patterns use  a  variant  of
          shell pattern syntax, with '*' matching any number of characters, '?' matching any single char‐
          acter, '[...]' matching any single character specified, including ranges, and '[!...]' matching
          any  character  not specified. For the purpose of these patterns, the path separator (backslash
          for Windows and '/' on other systems) is not treated specially. Wrap meta-characters in  brack‐
          ets for a literal match (i.e. [?] to match the literal character ?). For a path to match a pat‐
          tern, the full path must match, or it must match from the start of the full path to just before
          a  path  separator.  Except  for the root path, paths will never end in the path separator when
          matching is attempted.  Thus, if a given pattern ends in a path separator, a  '*'  is  appended
          before matching is attempted. A leading path separator is always removed.

   Shell-style patterns, selector sh:
          This  is  the default style for --pattern and --patterns-from.  Like fnmatch patterns these are
          similar to shell patterns. The difference is that the pattern may include **/ for matching zero
          or more directory levels, * for matching zero or more arbitrary characters with  the  exception
          of any path separator. A leading path separator is always removed.

Please notice the last sentences - "A leading path separator is always removed.`

So while one can apparently (and surprising to me) specify relative paths, the exclude/include patterns should never start with a slash in pure borg.

It is confusing that borg accepts relative paths in places and not in other places. And it is even more complicated when borgmatic has its own logic. Maybe at least this deserves some more clarification in docs.

from `man borg` > Fnmatch, selector fm: > This is the default style for --exclude and --exclude-from. These patterns use a variant of > shell pattern syntax, with '*' matching any number of characters, '?' matching any single char‐ > acter, '[...]' matching any single character specified, including ranges, and '[!...]' matching > any character not specified. For the purpose of these patterns, the path separator (backslash > for Windows and '/' on other systems) is not treated specially. Wrap meta-characters in brack‐ > ets for a literal match (i.e. [?] to match the literal character ?). For a path to match a pat‐ > tern, the full path must match, or it must match from the start of the full path to just before > a path separator. Except for the root path, paths will never end in the path separator when > matching is attempted. Thus, if a given pattern ends in a path separator, a '*' is appended > before matching is attempted. A leading path separator is always removed. > > Shell-style patterns, selector sh: > This is the default style for --pattern and --patterns-from. Like fnmatch patterns these are > similar to shell patterns. The difference is that the pattern may include **/ for matching zero > or more directory levels, * for matching zero or more arbitrary characters with the exception > of any path separator. A leading path separator is always removed. > Please notice the last sentences - "A leading path separator is always removed.` So while one can apparently (and surprising to me) specify relative paths, the exclude/include patterns should never start with a slash in pure borg. It is confusing that borg accepts relative paths in places and not in other places. And it is even more complicated when borgmatic has its own logic. Maybe at least this deserves some more clarification in docs.

akostadinov commented

2026-02-09 22:49:15 +00:00

Author

I believe the exclude patterns match absolute paths, not the path within the archive. Because to work, my patterns need the full absolute path, not only the path after /./ element.

One option is to make borgmatic match borg behavior where it accepts relative paths or not.

Another option would be to handle relative paths everywhere but make sure that these are adjusted to correct absolute paths when passed down to borg.

In both cases a little more documentation would be an improvement.

I hope this makes sense, as it is quiet late here and not sure my brain works.

I believe the exclude patterns match absolute paths, not the path within the archive. Because to work, my patterns need the full absolute path, not only the path after `/./` element. One option is to make borgmatic match borg behavior where it accepts relative paths or not. Another option would be to handle relative paths everywhere but make sure that these are adjusted to correct absolute paths when passed down to borg. In both cases a little more documentation would be an improvement. I hope this makes sense, as it is quiet late here and not sure my brain works.

akostadinov commented

2026-02-10 22:26:01 +00:00

Author

So there are IMHO two valid options:

all patterns are considered absolute path whether they start with a slash (path separator) or not. This will be same as borg itself
relative paths are considered relative to the defined workdir (unlike borg), so that workdir is appended to them and in addition still the snapshot substitution is performed

Just passing unprocessed relative paths to borg does no good as that has no chance of working as far as I can tell. Or did I miss your point?

Sorry for the spam. But even if we agree that borgmatic wants to support relative paths in exludes and patterns, then it should append the workdir before passing to borg. Otherwise patterns will not be recognized. So there are IMHO two valid options: * all patterns are considered absolute path whether they start with a slash (path separator) or not. This will be same as borg itself * relative paths are considered relative to the defined workdir (unlike borg), so that workdir is appended to them and in addition still the snapshot substitution is performed Just passing unprocessed relative paths to borg does no good as that has no chance of working as far as I can tell. Or did I miss your point?

witten commented

2026-04-05 02:53:39 +00:00

Owner

Please notice the last sentences - "A leading path separator is always removed.`

So while one can apparently (and surprising to me) specify relative paths, the exclude/include patterns should never start with a slash in pure borg.

I don't think the second statement follows from the first. Even though Borg always removes a leading path separator upon storage in an archive, you still need to supply a leading path separator if you want to specify and resolve an absolute path instead of a relative one. And that's even when using Borg by itself without borgmatic.

It is confusing that borg accepts relative paths in places and not in other places. And it is even more complicated when borgmatic has its own logic. Maybe at least this deserves some more clarification in docs.

Yeah, clarification in the docs makes sense to me.

I believe the exclude patterns match absolute paths, not the path within the archive. Because to work, my patterns need the full absolute path, not only the path after /./ element.

The part after the /./ is the only part of the path stored in the archive. That's just how Borg (1.4+) works. But if that's not the case on your system, can you give more details (logs, etc.) about this happening?

One option is to make borgmatic match borg behavior where it accepts relative paths or not.

Could you say more about what that might look like?

Another option would be to handle relative paths everywhere but make sure that these are adjusted to correct absolute paths when passed down to borg.

Sorry for the spam. But even if we agree that borgmatic wants to support relative paths in exludes and patterns, then it should append the workdir before passing to borg. Otherwise patterns will not be recognized.

relative paths are considered relative to the defined workdir (unlike borg), so that workdir is appended to them and in addition still the snapshot substitution is performed

The difficulty is that some users actually want relative paths stored. For instance, many users who are using the working_directory option are using it because they want relative paths stored, not absolute.

all patterns are considered absolute path whether they start with a slash (path separator) or not. This will be same as borg itself

See the example from above:

cd /home/witten/Downloads/tmp
$ borg create test.borg::archive2 etc
etc: stat: [Errno 2] No such file or directory: 'etc'

In this example, Borg is being given a path without an initial path separator—and is not considering it an absolute path. But maybe what you're saying though is that once stored within a Borg archive, Borg doesn't distinguish between relative and absolute paths? If so, I totally agree.

Just passing unprocessed relative paths to borg does no good as that has no chance of working as far as I can tell. Or did I miss your point?

I think it works for many users, but their use cases are very likely different from yours.

So getting back to your initial problem, I see a couple of possible solutions:

If you want to specify absolute paths (e.g. in source_directories or exclude_patterns), then use leading path separators. In both Borg and borgmatic, paths without leading path separators are relatives paths (just for purposes of resolving files, before their paths are stored in an archive).
Or if you really want to use relative paths, use them in conjunction with the working_directory option (or alternatively a cd before running borgmatic, which is basically the same thing just less convenient). Note however to your point, these will be stored as relative paths in Borg as well, not absolute ones.

But if I'm missing something and there's really a use case for specifying relative paths but storing absolute ones, please let me know!

> Please notice the last sentences - "A leading path separator is always removed.` > > So while one can apparently (and surprising to me) specify relative paths, the exclude/include patterns should never start with a slash in pure borg. I don't think the second statement follows from the first. Even though Borg always removes a leading path separator upon storage in an archive, you still need to supply a leading path separator if you want to specify and resolve an absolute path instead of a relative one. And that's even when using Borg by itself without borgmatic. > It is confusing that borg accepts relative paths in places and not in other places. And it is even more complicated when borgmatic has its own logic. Maybe at least this deserves some more clarification in docs. Yeah, clarification in the docs makes sense to me. > I believe the exclude patterns match absolute paths, not the path within the archive. Because to work, my patterns need the full absolute path, not only the path after `/./` element. The part after the `/./` is the only part of the path stored in the archive. That's just how Borg (1.4+) works. But if that's not the case on your system, can you give more details (logs, etc.) about this happening? > One option is to make borgmatic match borg behavior where it accepts relative paths or not. Could you say more about what that might look like? > Another option would be to handle relative paths everywhere but make sure that these are adjusted to correct absolute paths when passed down to borg. > Sorry for the spam. But even if we agree that borgmatic wants to support relative paths in exludes and patterns, then it should append the workdir before passing to borg. Otherwise patterns will not be recognized. > relative paths are considered relative to the defined workdir (unlike borg), so that workdir is appended to them and in addition still the snapshot substitution is performed The difficulty is that some users actually want relative paths stored. For instance, many users who are using the `working_directory` option are using it because they want relative paths stored, not absolute. > all patterns are considered absolute path whether they start with a slash (path separator) or not. This will be same as borg itself See the example from above: ```bash cd /home/witten/Downloads/tmp $ borg create test.borg::archive2 etc etc: stat: [Errno 2] No such file or directory: 'etc' ``` In this example, Borg is being given a path without an initial path separator—and is not considering it an absolute path. But maybe what you're saying though is that *once stored within a Borg archive*, Borg doesn't distinguish between relative and absolute paths? If so, I totally agree. > Just passing unprocessed relative paths to borg does no good as that has no chance of working as far as I can tell. Or did I miss your point? I think it works for many users, but their use cases are very likely different from yours. So getting back to your initial problem, I see a couple of possible solutions: 1. If you want to specify absolute paths (e.g. in `source_directories` or `exclude_patterns`), then use leading path separators. In both Borg and borgmatic, paths without leading path separators are relatives paths (just for purposes of resolving files, before their paths are stored in an archive). 2. Or if you really want to use relative paths, use them in conjunction with the `working_directory` option (or alternatively a `cd` before running borgmatic, which is basically the same thing just less convenient). Note however to your point, these will be stored as relative paths in Borg as well, not absolute ones. But if I'm missing something and there's really a use case for specifying relative paths but storing absolute ones, please let me know!