Cap backups by size #25
Labels
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Milestone
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: borgmatic-collective/borgmatic#25
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
It would be nice (although I guess it would be some work without direct support from borg) to have support for capping the number of backups by size.
I'll explain myself better and provide an hypothetical algorithm.
The example config file on the borgmatic website has:
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
Which is lovely! Except for one thing: if I've got plenty of space I'd like to keep the highest possible number of monthly snapshots that fit.
So I'd love to say:
keep_daily: 7
keep_weekly: 4
keep_monthly: up to 300GB, but at least 6 anyway
Now, the algorithm would be: when we go above quota we simply drop old versions until we either reach the required minimum or we get below quota again.
Due to deduplication we can't know in advance how much to drop.
Imported from Taiga issue 24 (to do). Created on 2016-07-10T17:01:19+0000 by Leonardo Taglialegne.
Indeed, that would be a pretty cool feature. A variant of that idea: Just specify the quota separately from all the
keep_
options, and somehow have borgmatic adjust the number of archives kept across all the intervals to stay below that quota.Comment on 2016-07-10T17:44:19+0000 by Dan Helfman.
It seems to be difficult to manage inside borgmatic because it doesn't know which chunks should be removed - except by date, which is a bit rude -, and more importantly can't remove them during
borg create
.As a (not so reliable) workaround, borgmatic could ask (store in a cache for the sake of speed?) the size of the last n backups in order to derive a conservative estimation of the size of the current backup, and prune the oldest ones until a sufficient amount of disk space is available. It seems to be a very important feature, as borg doesn't handle full disks situation well.
Btw, I see the source code of borgmatic calls
create
comes beforeprune
(`command.py:51-54). IMHO they should be swapped, in order to make room with prune before creating the new backup. That would limit the disk space, and potentially avoid conditions that could break Borg.Comment on 2016-09-13T10:24:29+0000 by Aurel G.
The next borgmatic release (1.1.0) will
prune
beforecreate
as you suggested.Comment on 2017-07-23T04:50:49+0000 by Dan Helfman.
Note that there is a
borg init --storage-quota
parameter. It's conceivable that using that underlying Borg quota support could drive the feature described in this ticket. For instance, if borgmatic is trying to create an archive and hits an over-quota error, it could try to opportunistically prune archives until getting back under the quota.Closing due to inactivity, but I'd be happy to reopen if there's still user interest.