LVM snapshots (#80). #949
Loading…
x
Reference in New Issue
Block a user
No description provided.
Delete Branch "lvm-snapshots"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This is currently prototype-quality work for snapshotting LVM logical volumes (#80) from borgmatic such that they end up in a Borg archive at the original logical volume path rather than the snapshot path. That means that if your logical volume is mounted at, say,
/my/lvolume
, then the snapshotted data shows up in the Borg archive atmy/lvolume
and not some snapshot directory.Logical volume discovery currently works by expecting that any volumes to backup are listed in borgmatic's
source_directories
by their mount paths.Result
I have run this and it appears to work so far. Here are some logs:
Still to do
@ -0,0 +73,4 @@
lvcreate_command,
'--snapshot',
'--extents',
'1', # The snapshot doesn't need much disk space because it's read-only.
i don't think that works. snapshots take up space even if you don't write to them: writes to the parent volume will write to the snapshot (counter-intuitively, i admit) to keep a copy of the old extent.
at least, that's how i understand it: did you test this to confirm this actually works in production?
i suspect you'll have to make that number of extents configurable, and in fact i would use the
--size
parameter instead of--extents
, unless you want to start parsing units for the user and converting that into extents...Works in production? No. Works on my test machine? Yes. So are you thinking just a configuration option for setting the size of snapshots in MB? Would that really work globally for all logical volumes? Or do you think, at the risk of complicating this code, a percentage of logical volume size would work better?
And thanks for taking the time to look at this and weigh in!
I suspect that works for you because you never write enough to fill that
one extent.
Essentially, yes. It's not necessarily in MB though... You specify the
unit or percentage, so you can do -L 10%ORIGIN to have a snapshot 10% of
the original size, or -L 1G to have a 1G snapshot. I'd pass that
verbatim to lvcreate, essentially.
You could even use 10% as a default, but then the problem is you need
enough free space to cover for that, which is why I think this needs to
be configured by the user.
No problem, thanks for taking a bit on that one!
Sounds good. I'll add that option as you describe, potentially with a default.
Okay, implemented!
@ -148,0 +218,4 @@
Additionally, borgmatic rewrites the snapshot file paths so that they appear
at their original logical volume locations in a Borg archive. For instance, if
your logical volume is mounted at `/mnt/lvolume`, then the snapshotted files
will appear in an archive at `/mnt/lvolume` as well.
i'm a bit confused by this. what are we rewriting here exactly? the files appear in
/mnt/lvolume
, exactly as they are in the logical volume...i would have expected this to say something like:
that would, at least, clarify things for me.
part of the confusion i have, i think, comes from the use of
/mnt
in the example: that's a mountpoint i use for temporary things, not a real partition, so i thought this was where borgmatic would mount the snapshot, temporarily.Got it. I really appreciate the feedback. I will clarify!
i can't figure this out from the code, but another concern i would have here is what happens when you (say) backup
/var/log
and/var/lib
in your configuration, and that the logical volume is in/var
. does it work? does it make multiple snapshots?No, that doesn't work currently. borgmatic would try to find a logical volume mounted at
/var/log
, not find one, try to find a logical volume mounted at/var/lib
, not find one, and then ask Borg to backup those two paths without any snapshotting for them.So at minimum perhaps this should be documented more clearly, so the user knows they need to specify
/var
if they want it snapshotted as part of backups?I could maybe see the case for making borgmatic smart enough to detect this case and snapshot
/var
since it's a common parent directory, but:/var/log
,/var/lib
, and/var/humongous_ever_changing_database
. Just because the user wants/var/log
and/var/lib
backed up doesn't mean they necessarily want to spend snapshot space on their database too.Thoughts?
i would say that's a footgun i'd be happy to provide users: they asked for that snapshot, just give it to them.
then you might be happy to know that Someone already wrote some of that code for you. it's crap old code, and not for borg, but you might find inspiration there.
i think what you want is the find_mountpoint logic (which is good, pure-python code) and perhaps some
find_device
stuff (which is f-ing ugly, if you ask me, parsing/proc/mounts
is probably more reliable, as the kernel won't break that interface).then you need to deduplicate those "snapshot that directory" queries somehow.
but yes, this is the kind of stuff that made me not implement this myself, because i knew how tricky it is.
Okay, thanks for the link. I'll look into what it would take to add this feature. No promises. 😄
Well it actually wasn't too bad. Didn't even have to dedup since everything was already in terms of logical volumes. It'll need lots more testing though. And I'll probably want to implement something similar for ZFS and Btrfs.
fantastic! i'll see if i can find the time to actually test this stuff.. i have a server in the lab here that i'm running borg backups again (not with borgmatic, i'm ashamed to say) that keeps failing because a stupid log file "has changed during backup", hopefully that will knock down that annoyance!
Sounds like a perfect test candidate! I'm sure I'll flush out some more bugs as well with local testing here + automated testing.
WIP: LVM snapshots (#80).to LVM snapshots (#80).