borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 57
Code Issues 71 Pull requests 3 Releases 162 Packages 1 Activity Actions 1

Add ignore database option, add possibility to configure list of db names #973

New issue
Closed
opened 2025-01-21 09:48:25 +00:00 by powerriegel · 5 comments
powerriegel commented 2025-01-21 09:48:25 +00:00
Copy link

What I'd like to do and why

Currently, It's not possible to simply ignore databases for backup when selecting all databases.

For MariaDB, setting options: --ignore_database foo does not work, as the all option does not hand over --all-databases to m̀ariadb-dump but dumps each DB for it's own. The option works only with the --all-databases parameter.

Other notes / implementation ideas

It would also be nice to be able to configure a list of database names without requiring to configure host, user and password for each DB. This makes borgmatic difficult to use for large database servers.

### What I'd like to do and why Currently, It's not possible to simply ignore databases for backup when selecting `all` databases. For MariaDB, setting `options: --ignore_database foo` does not work, as the `all` option does not hand over `--all-databases` to `m̀ariadb-dump` but dumps each DB for it's own. The option works only with the `--all-databases` parameter. ### Other notes / implementation ideas It would also be nice to be able to configure a list of database names without requiring to configure `host`, `user` and `password` for each DB. This makes borgmatic difficult to use for large database servers.
witten commented 2025-01-24 18:57:57 +00:00
Owner
Copy link

For MariaDB, setting options: --ignore_database foo does not work, as the all option does not hand over --all-databases to m̀ariadb-dump but dumps each DB for it's own. The option works only with the --all-databases parameter.

IIRC, borgmatic dumps each database on its own so that it can filter out system databases from the "all" dump. Otherwise, trying to restore those system databases will fail upon restore.

But to support your use case, I can imagine borgmatic's config supporting an ignore_names list, optionally set for any database with name all. Then borgmatic could do the filtering out of ignored databases on its own.

It would also be nice to be able to configure a list of database names without requiring to configure host, user and password for each DB. This makes borgmatic difficult to use for large database servers.

You could probably do this with a merge include today, but I'm not sure if even that would be convenient enough for you. It might look something like this:

mariadb_databases:
    - name: mydb1
      <<: !include db_common.yaml
    - name: mydb2
      <<: !include db_common.yaml
    ...

And then db_common.yaml could contain:

host: myhost
username: myuser
password: mypass

Another idea (not implemented today) is a syntax like the following:

mariadb_databases:
    - name: [mydb1, mydb2, ...]
      host: myhost
      username: myuser
      password: mypass

Anyway, these host/username/password ideas may be best in a separate ticket.

> For MariaDB, setting options: --ignore_database foo does not work, as the all option does not hand over --all-databases to m̀ariadb-dump but dumps each DB for it's own. The option works only with the --all-databases parameter. IIRC, borgmatic dumps each database on its own so that it can filter out system databases from the "all" dump. Otherwise, trying to restore those system databases will fail upon restore. But to support your use case, I can imagine borgmatic's config supporting an `ignore_names` list, optionally set for any database with name `all`. Then borgmatic could do the filtering out of ignored databases on its own. > It would also be nice to be able to configure a list of database names without requiring to configure host, user and password for each DB. This makes borgmatic difficult to use for large database servers. You could probably do this with a [merge include](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#include-merging) today, but I'm not sure if even that would be convenient enough for you. It might look something like this: ```yaml mariadb_databases: - name: mydb1 <<: !include db_common.yaml - name: mydb2 <<: !include db_common.yaml ... ``` And then `db_common.yaml` could contain: ```yaml host: myhost username: myuser password: mypass ``` Another idea (not implemented today) is a syntax like the following: ```yaml mariadb_databases: - name: [mydb1, mydb2, ...] host: myhost username: myuser password: mypass ``` Anyway, these `host`/`username`/`password` ideas may be best in a separate ticket.
powerriegel commented 2025-01-27 09:17:24 +00:00
Author
Copy link

Thanks, I will try the inlude solution. It makes the configuration much shorter.

Thanks, I will try the inlude solution. It makes the configuration much shorter.
👍 1
witten commented 2025-10-16 21:50:31 +00:00
Owner
Copy link

This is implemented in main and will be part of the next release! The new option is called skip_names and is implemented right now for both MariaDB and MySQL. Just add it to a given database config, and the value is a sequence of database names to ignore when dumping "all".

This is implemented in main and will be part of the next release! The new option is called `skip_names` and is implemented right now for both MariaDB and MySQL. Just add it to a given database config, and the value is a sequence of database names to ignore when dumping "all".
👍 1
powerriegel commented 2025-10-23 08:18:48 +00:00
Author
Copy link

Really happy to hear and cant wait to run the new release! Thanks a lot!

Really happy to hear and cant wait to run the new release! Thanks a lot!
👍 1
witten commented 2025-10-24 04:52:55 +00:00
Owner
Copy link

Released in borgmatic 2.0.10!

Released in borgmatic 2.0.10!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
parallel-borgmatic
command-hook-steps
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#973
No description provided.