Why are "database hook" and "one_file_system" are linked together? #527

New Issue

Closed

opened 2022-05-05 19:09:14 +00:00 by qupfer · 3 comments

qupfer commented 2022-05-05 19:09:14 +00:00

Copy Link

Hi,

and yes, if I follow the "RTFM" approach, I would have saved a lot of time :-)

I moved a service (mailcow) from one host to an other and after that, my borgmatic backups are nearly empty (empty folder) but success.

I was clever enough to think thas related to the btrfs filesystem (old host was ext4).
So I looked and found the "one_file_system" option. But it did not change anything. Folders still empty in the borg repo.
After reading all lines of the manual, I saw its always "true" if a database hook is used. Bingo!
Changing that (disasble database hook), it looks like all is copied and the folders contain the data again.

But why is this linked together? And is there an option to have an database dump inside the borg repo and all data - without having all in the same btrfs subvolume?

And yes, I also think there could be some discussion if a btrfs subvolume should be treated as an other filesystem or not scnr

Thanks
a still happy borg(matic) user

Hi, and yes, if I follow the "RTFM" approach, I would have saved a lot of time :-) I moved a service (mailcow) from one host to an other and after that, my borgmatic backups are nearly empty (empty folder) but success. I was clever enough to think thas related to the btrfs filesystem (old host was ext4). So I looked and found the "one_file_system" option. But it did not change anything. Folders still empty in the borg repo. After reading all lines of the manual, I saw its always "true" if a database hook is used. Bingo! Changing that (disasble database hook), it looks like all is copied and the folders contain the data again. But why is this linked together? And is there an option to have an database dump inside the borg repo and all data - without having all in the same btrfs subvolume? And yes, I also think there could be some discussion if a btrfs subvolume should be treated as an other filesystem or not *scnr* Thanks a still happy borg(matic) user

witten commented 2022-05-05 23:30:37 +00:00

Owner

Copy Link

Yikes, nearly empty backups are not good!

But why is this linked together?

You can find the sordid history of that in the comments on #315! But the short of it is that because borgmatic sets read_special when database hooks are enabled (to support streaming databases directly to Borg), it also needs to set one_file_system to prevent hanging in many common use cases like backing up /.

And is there an option to have an database dump inside the borg repo and all data - without having all in the same btrfs subvolume?

Not all in one configuration file, currently. However, if you put your database dumps in one configuration file and your btrfs backups in another, then I believe that would allow borgmatic to run Borg once with one_file_system (for the database backup) and once without (for the btrfs backup). The downside is you'd end up with a separate archive for each.

(Note that as of borgmatic 1.6.0, you can share common settings between the two configuration files pretty easily.)

Let me know your thoughts!

Yikes, nearly empty backups are not good! > But why is this linked together? You can find the sordid history of that in the comments on #315! But the short of it is that because borgmatic sets `read_special` when database hooks are enabled (to support streaming databases directly to Borg), it also needs to set `one_file_system` to prevent hanging in many common use cases like backing up `/`. > And is there an option to have an database dump inside the borg repo and all data - without having all in the same btrfs subvolume? Not all in one configuration file, currently. However, if you put your database dumps in one configuration file and your btrfs backups in another, then I *believe* that would allow borgmatic to run Borg once with `one_file_system` (for the database backup) and once without (for the btrfs backup). The downside is you'd end up with a separate archive for each. (Note that as of borgmatic 1.6.0, you can [share common settings](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#include-merging) between the two configuration files pretty easily.) Let me know your thoughts!

witten added the

question / support

label 2022-05-05 23:30:43 +00:00

qupfer commented 2022-05-20 17:46:03 +00:00

Author

Copy Link

Thanks for your clear answer. That helps to understand.
I solved it for me to make the database dump "outside" borgmatic and put it in the filesystem, I use this also too switch to mariabackup instead mysqldump.
But will try the "two-config-solution" sometimes too.

Thanks for your clear answer. That helps to understand. I solved it for me to make the database dump "outside" borgmatic and put it in the filesystem, I use this also too switch to mariabackup instead mysqldump. But will try the "two-config-solution" sometimes too.

witten commented 2022-05-20 17:48:06 +00:00

Owner

Copy Link

Sounds good! Glad to hear you have a work-around.

Sounds good! Glad to hear you have a work-around.

witten closed this issue 2022-05-20 17:48:06 +00:00

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#527

No description provided.