borg has support for piped input, great for mysqldump, but I don’t see a way to configure with borgmatic.
Imported from Taiga issue 41 (to do). Created on 2017-09-08T13:41:37+0000 by asdfasdfoaijsdf.
Here’s the documentation on the feature in borg
I got the idea from a similar backup program called restic:
I use it pretty much exactly as in the example of restic, to pipe the output of mysqldump directly into a borg repo.
Comment on 2017-09-09T11:12:36+0000 by jorisjh.
Got it, thanks. That makes sense.. I can envision two different ways that borgmatic could implement that feature:
mysqldumpthere), the output of which borgmatic pipes into the
borg createinvocation. Then, you’d have to invoke borgmatic with the command you want to use (e.g.
mysqldump | borgmatic --stdinor whatever).
The second option would have some challenges, in that it’d be unclear which of the multiple borgmatic config files should consume that stdin. Like, if you’ve got five borgmatic config files and thus five Borg repositories, you probably only want that stdin to go to one of them. So that might mean option one is a more natural fit.
Comment on 2017-09-09T17:01:44+0000 by Dan Helfman.
Now that borgmatic has a hooks system in place, option number 1 may be doable with a hook. For instance, there could be a new
backup_input hook that runs an arbitrary command and pipes its output to Borg’s stdin pretty easily.
Support for the
borg create --stdin-name flag might also be nice to have.
Wow, it looks like this might “just work”. I tried adding a borgmatic source directory of
"-" (with quotes) in the config file to indicate stdin. Then I was able to pipe input in via the command-line:
echo "hi" | borgmatic --create
I did an extract and confirmed that a file named “stdin” with those contents was extracted.
So, work still to-do before calling this feature done:
"-"declared. It appears that the first instance of
"-"consumes all the stdin, and the second instance silently consumes nothing. Which doesn’t seem like great behavior. One idea to deal with this is to guard that
"-"is present at most once in all source directories across all config files. Error if it appears twice or more.
With #258 implemented, adding separate support for stdin streaming should no longer be necessary. That’s because with the new automatic database -> Borg streaming support in #258, you shouldn’t have to manually execute database dumps and stream them into borgmatic.
Having said that, if there’s a use case I’ve missed with stdin support, please feel free to file a new ticket or reopen this one.
Deleting a branch is permanent. It CANNOT be undone. Continue?