borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
10
66
Fork 27
Code Issues 42 Pull Requests 5 Actions Packages Releases 140 Activity

Remove previous named pipes before dumping databases #360

New Issue
Closed
opened 2020-10-14 12:34:59 +00:00 by fallen · 7 comments
fallen commented 2020-10-14 12:34:59 +00:00
Copy Link

What I'm trying to do and why

I had a borg/borgmatic infinite hang during my backups, I opened a ticket on borg repo (https://github.com/borgbackup/borg/issues/5420) then realized the issue was because of a dangling UNIX pipe in /root/.borgmatic/ due to a previously interrupted run with a wrong config (postgresql in the config but no postgresql on the server).
It took me long to understand what was going on.

I would like to avoid this to happen again (or to somebody else).

Steps to reproduce (if a bug)

Tell borgmatic config file you want to backup postgresql database even though you don't have postgresql.
It will create a dangling UNIX pipe and fail.
Then fix your config to tell only to backup mysql database.
Then run your backup, it will hang because of the dangling pipe for postgre is still there.

Actual behavior (if a bug)

It hangs indefinitely

Expected behavior (if a bug)

I would expect either a timeout (I opened a ticket for this idea on the borg repo).
See the ideas section for what is related to borgmatic.

Other notes / implementation ideas

I would propose to always cleanup the .borgmatic directory before backuping.
Or at least removing the mysql and postgresql related pipes.
Then, if in the config there is still a database backup section, re-create the needed pipes.

Environment

borgmatic version: 1.5.10

borgmatic installation method: via documented ansible playbook

Borg version: 1.1.14

Python version: 3.5.3

Database version (if applicable): mysql Ver 15.1 Distrib 10.1.45-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2

operating system and version:
Distributor ID: Debian
Description: Debian GNU/Linux 9.13 (stretch)
Release: 9.13
Codename: stretch

#### What I'm trying to do and why I had a borg/borgmatic infinite hang during my backups, I opened a ticket on borg repo (https://github.com/borgbackup/borg/issues/5420) then realized the issue was because of a dangling UNIX pipe in /root/.borgmatic/ due to a previously interrupted run with a wrong config (postgresql in the config but no postgresql on the server). It took me long to understand what was going on. I would like to avoid this to happen again (or to somebody else). #### Steps to reproduce (if a bug) Tell borgmatic config file you want to backup postgresql database even though you don't have postgresql. It will create a dangling UNIX pipe and fail. Then fix your config to tell only to backup mysql database. Then run your backup, it will hang because of the dangling pipe for postgre is still there. #### Actual behavior (if a bug) It hangs indefinitely #### Expected behavior (if a bug) I would expect either a timeout (I opened a ticket for this idea on the borg repo). See the ideas section for what is related to borgmatic. #### Other notes / implementation ideas I would propose to always cleanup the .borgmatic directory before backuping. Or at least removing the mysql and postgresql related pipes. Then, if in the config there is still a database backup section, re-create the needed pipes. #### Environment **borgmatic version:** 1.5.10 **borgmatic installation method:** via documented ansible playbook **Borg version:** 1.1.14 **Python version:** 3.5.3 **Database version (if applicable):** mysql Ver 15.1 Distrib 10.1.45-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 **operating system and version:** Distributor ID: Debian Description: Debian GNU/Linux 9.13 (stretch) Release: 9.13 Codename: stretch
witten commented 2020-10-14 16:54:18 +00:00
Owner
Copy Link

Fun fact: borgmatic does delete the PostgreSQL named pipe before initiating a backup... but only if the PostgreSQL hook is configured in borgmatic's configuration file! So I think the fix would be for borgmatic to delete the PostgreSQL (and MySQL) named pipes unconditionally before running a create—even if databases are not configured.

You win the award for finding the edge case of the week.🎖️

Thanks so much for bringing this to my attention.

Fun fact: borgmatic *does* delete the PostgreSQL named pipe before initiating a backup... but only if the PostgreSQL hook is configured in borgmatic's configuration file! So I think the fix would be for borgmatic to delete the PostgreSQL (and MySQL) named pipes unconditionally before running a create—even if databases are not configured. You win the award for finding the edge case of the week.🎖️ Thanks so much for bringing this to my attention.
🎉 1
fallen commented 2020-10-14 18:03:04 +00:00
Author
Copy Link

Ahah thanks for the 🎖️!
Yes I agree, that would fix it.
In parallel I am in contact with Borg's people so that they add a timeout and an error message to make this kind of situation more obvious.
(That can also help diagnose issues with people backuping character devices without knowing it with --read-special)
You're welcome!
Thanks for the very quick response :)
And thanks for the good software!

Ahah thanks for the 🎖️! Yes I agree, that would fix it. In parallel I am in contact with Borg's people so that they add a timeout and an error message to make this kind of situation more obvious. (That can also help diagnose issues with people backuping character devices without knowing it with --read-special) You're welcome! Thanks for the very quick response :) And thanks for the good software!
👍 1
witten added the
bug
 label 2020-10-14 18:15:56 +00:00
witten commented 2020-11-20 23:24:13 +00:00
Owner
Copy Link

Relevant Borg ticket: https://github.com/borgbackup/borg/issues/5422

Relevant Borg ticket: https://github.com/borgbackup/borg/issues/5422
witten commented 2022-10-12 17:26:31 +00:00
Owner
Copy Link

At long last, this should be taken care of! borgmatic now unconditionally nukes all ~/.borgmatic database named pipes, whether or not their hooks are configured. This happens automatically before/after create and restore currently. These changes will be part of the next release.

At long last, this should be taken care of! borgmatic now unconditionally nukes all `~/.borgmatic` database named pipes, whether or not their hooks are configured. This happens automatically before/after `create` and `restore` currently. These changes will be part of the next release.
witten commented 2022-10-12 18:03:17 +00:00
Owner
Copy Link

Just released in borgmatic 1.7.3!

Just released in borgmatic 1.7.3!
👍 1 🎉 1
fallen commented 2022-10-13 06:46:52 +00:00
Author
Copy Link

Hey @witten , thanks a lot for the fix :)
I'm still using borgmatic and it works great!

Hey @witten , thanks a lot for the fix :) I'm still using borgmatic and it works great!
witten commented 2022-10-13 16:07:40 +00:00
Owner
Copy Link

Glad to hear it!

Glad to hear it!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
bug
Milestone
No items
No Milestone
Assignees
Clear assignees
diivi import_bot witten
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#360
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?