findmnt output not being properly parsed #954
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: borgmatic-collective/borgmatic#954
Loading…
x
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What I'm trying to do and why
It looks like the output of
findmntfor the btrfs integration isn't being properly parsed. When I run any create job, I get the following in the output:(see below for full debug output)
Steps to reproduce
Here's a small reproducer, from scratch, with a simple config file.
Initialising:
Creating an archive (note the ERRORs):
Actual behavior
See reproducer.
Expected behavior
No
findmnterrors.Other notes / implementation ideas
findmnthas a--jsonoption which should avoid this and similar problems in the future.Current findmnt version is:
Sample output from findmnt:
borgmatic version
1.9.4
borgmatic installation method
Arch Linux package
Borg version
borg 1.4.0
Python version
Python 3.12.7
Database version (if applicable)
No response
Operating system and version
Arch Linux
According to the docs, the btrfs snapshot feature is only used when the
btrfs:option is added. But it looks like thefindmntruns anyway, breaking the archive creation. What's worse is I think this is causing my backups to fail without warning. The process succeeds but thecreatecommand is failing or creating empty archives.What's going on here is that the cleanup steps for all data source hooks (database + filesystem) are run unconditionally, with the idea that if a prior borgmatic run crashed or was interrupted, there may be leftover files from a hook previously running that need cleaning up—even if that hook isn't currently enabled in config. This is especially important with the database hooks, because a stale named pipe used for database streaming will cause Borg to hang.
In the case of snapshot hooks like the btrfs hook, there's perhaps less of a hang concern if there's a just stale snapshot left behind, so perhaps it's less important to run the cleanup steps of the hook unconditionally.
So that's one consideration here. The other is that regardless of when the cleanup steps run,
findmntshouldn't be blowing up. Thefindmntdocs say that--listmode (which is supposed to prevent the hierarchy from getting displayed e.g. with characters like├─) is implicitly enabled when when-tis used—which it is here. But can you try adding--listto thefindmntcommand to see if it "fixes" the output? E.g.:Switching to JSON output would probably be possible here though.
Thanks. Adding
--listfixes that issue, but now another one cropped up:With
I get:
Seems there's some interesting interactions happening between borgmatic and snapper snapshots.
Oh, I was just suggesting trying out that
findmntcommand with--listmanually on the command-line, not adding it to thefindmnt_commandoption.. It'll result in duplicate flags..-nand-t btrfswill get passed tofindmnttwice.findmntdoesn't seem to mind, but by specifying thebtrfs:option you are enabling borgmatic's Btrfs hook, which is causing it to try to snapshot any Btrfs subvolumes that are specified insource_directories.Anyway, about the error..
I'm honestly not sure how this ("file exists") is possible, since borgmatic generates the subvolume paths on the fly based on the process ID of the current borgmatic process. You mentioned that there might be an interaction with Snapper here, but presumably Snapper wouldn't be creating conflicting snapshots with "borgmatic" in the name, especially not with the borgmatic process ID.
Could I get a look at the output of
btrfs subvolume list /to see whether there's possibly some conflict? Also what about the output ofls /.borgmatic-snapshot-*as well?Thanks!
FWIW, I've switched the use of findmnt in main to use
--jsonas you suggested.Ah, my bad. Misunderstood. Thanks for clarifying.
Yes, it's weird.
They're always empty after the failure occurs and borgmatic exits.
I'll install the latest version of borgmatic from source and try again, with the
findmnt --json, and let you know what's up.Perhaps the snapshot creation failure is unrelated and we could move this to a different issue.
Everything seems to be working nicely now, thanks.
Installed from source (
pipx install .), commit IDebb3bca4b3.Click for log
Hey, I think I have the same or similar issue, if this would fit a new bug report let me know. I'm using the latest borgmatic version available on Arch Linux (1.9.4) and was confused that I suddenly started to see ERROR and WARNING messages when running my backups -- without borgmatic acknowledging those at all.
The backup finished as always. I then looked into the verbose output and found the reason, which looked quite similar to the issue here:
and so on (no further errors or warnings appear).
In addition to btrfs the lvs command also created a non-parsed message on my system. I have neither a btrfs nor a lvs section in my config. These messages are different than those in this thread, but I think it is related enough to fit in here.
In your case, it seems it's an issue with running
borgmaticas non-root. It wants to do abtrfs subvolume list, which requires root privileges:findmntis working fine it seems, presumably because you only have a root btrfs volume.You're completely right with that one, but I read the initial issue (also from the title) in the way that borgmatic is not parsing the output of the command correctly and thus showing it to us on the command line. This happens here as well (for the btrfs, but also lvs hook). Thus different messages, but the same issue. It's okay if the called program throws an error (e.g. because root permission are required), but this shouldn't be shown to the user, especially if the hooks aren't used.
@bioxz Yeah, borgmatic is trying to cleanup snapshots, and the fact that you're a non-root user is causing those non-fatal errors you're seeing. Refer to this comment from above:
I think though based on the behavior you and others are seeing here, it's probably not worth it to call the cleanup steps of these snapshot hooks unconditionally anymore. So if you wouldn't mind filing a separate ticket for that, I'll look into only running the cleanup steps when the hooks are configured. However I probably won't change the database hooks in particular, because of the real risk of Borg hangs if those cleanups don't run. Thank you!
@achilleas I'm glad to hear everything seems to be working well now. I do still wonder about that potential interaction between borgmatic's Btrfs support and Snapper. So if you do ever enable the Btrfs hook, we may need to revisit that.
I went ahead and implemented the change to the filesystem hooks so their cleanup steps don't trigger if the hooks are not configured:
48dca28c74This is in main and will be part of the next release.
Released in borgmatic 1.9.5!
Thank you, 1.9.5 is now available on Arch Linux and the output clean as it was before!
Glad to hear it!