Specify path for database dump/restore commands #311

New Issue

Closed

opened 2020-05-15 17:52:51 +00:00 by witten · 4 comments

witten commented 2020-05-15 17:52:51 +00:00

Owner

Copy Link

What I'm trying to do and why

As a user doing database dumps and restores, I want to be able to specify the path to the various dump/restore binaries (pg_dump, pg_restore, mysql, etc.) in the borgmatic configuration file. The idea is that these binaries may not be on the default path, or I may want to select different ones than those on the default path.

Work-around: Manipulate the shell PATH before invoking borgmatic.

Other notes / implementation ideas

This could work similarly to the existing Borg local_path / remote_path options. One challenge is that there are multiple different binaries, not just one. So maybe specifying the folder containing all of them (per database hook) would be better.

#### What I'm trying to do and why As a user doing database dumps and restores, I want to be able to specify the path to the various dump/restore binaries (`pg_dump`, `pg_restore`, `mysql`, etc.) in the borgmatic configuration file. The idea is that these binaries may not be on the default path, or I may want to select different ones than those on the default path. Work-around: Manipulate the shell `PATH` before invoking borgmatic. #### Other notes / implementation ideas This could work similarly to the existing Borg `local_path` / `remote_path` options. One challenge is that there are multiple different binaries, not just one. So maybe specifying the folder containing all of them (per database hook) would be better.

level420 commented 2021-02-12 09:31:35 +00:00

Copy Link

@witten this feature would be very helpful for our use case, where we have different mariadb (versions 5 and 10) and firebirdsql (versions 2.5 and 3) instances running in docker containers which are not reachable from the host but only within their isolated docker network.

In our home brew backup scripts we define these commands for database dumping:

CMD_MYSQLDUMP="/usr/bin/docker exec -t stack-somestack_mariadb_1 /usr/bin/mysqldump"

This dumps the database by executing the appropriate mysqldump within the running database container, dumping it to stdout.

The restore command works similarly by piping the dump via stdin to the mysql command within the container.

And it would be great to allow for indidvidual database configurations like

    other_databases:
        - name: someotherdatabase
          before_dump: <here my individual action to prepare for dumping>
          dump: <here my individual dump command to stdout>
          after_dump: <here my individual action to cleanup dumping>
          before_restore: <here my individual action to prepare for restoration>
          restore: <here my individual restore command to stdin>
          after_restore: <here my individual action to cleanup restoration>

This would allow for very individual configurations, not only tied to postgresql or mysql, as long as there are command line tools available which could be used to dump and restore from/to stdin/stdout. Or, if filenames have to be involved, some filename expansion like ${filename} or similar where borgmatic could create some temporary usable filename.

@witten this feature would be very helpful for our use case, where we have different mariadb (versions 5 and 10) and firebirdsql (versions 2.5 and 3) instances running in docker containers which are not reachable from the host but only within their isolated docker network. In our home brew backup scripts we define these commands for database dumping: CMD_MYSQLDUMP="/usr/bin/docker exec -t stack-somestack_mariadb_1 /usr/bin/mysqldump" This dumps the database by executing the appropriate mysqldump within the running database container, dumping it to stdout. The restore command works similarly by piping the dump via stdin to the mysql command within the container. And it would be great to allow for indidvidual database configurations like ``` other_databases: - name: someotherdatabase before_dump: <here my individual action to prepare for dumping> dump: <here my individual dump command to stdout> after_dump: <here my individual action to cleanup dumping> before_restore: <here my individual action to prepare for restoration> restore: <here my individual restore command to stdin> after_restore: <here my individual action to cleanup restoration> ``` This would allow for very individual configurations, not only tied to postgresql or mysql, as long as there are command line tools available which could be used to dump and restore from/to stdin/stdout. Or, if filenames have to be involved, some filename expansion like ${filename} or similar where borgmatic could create some temporary usable filename.

witten referenced this issue from a commit 2022-11-18 16:35:11 +00:00

specify pg dump/restore commands (#311)

witten referenced this issue from a commit 2022-11-18 16:35:11 +00:00

Override PostgreSQL dump/restore commands via configuration options (#311).

witten commented 2022-11-27 17:05:05 +00:00

Author

Owner

Copy Link

Just released for PostgreSQL in borgmatic 1.7.5! I'll leave this ticket open to cover the other database hooks.

Just released for PostgreSQL in borgmatic 1.7.5! I'll leave this ticket open to cover the other database hooks.

witten added the

good first issue

label 2023-03-29 18:49:09 +00:00

witten referenced this issue from a commit 2024-03-03 22:33:06 +00:00

Add custom dump/restore command options for MySQL and MariaDB (#311).

witten commented 2024-03-03 22:36:37 +00:00

Author

Owner

Copy Link

shivansh02 added custom command options for MariaDB and MySQL. So now all that remains is adding custom command support for the SQLite and MongoDB hooks. Rather than leave this ticket open even longer, I'll file separate tickets for that work.

[shivansh02](https://github.com/shivansh02) added [custom command options for MariaDB and MySQL](https://github.com/borgmatic-collective/borgmatic/pull/81). So now all that remains is adding custom command support for the SQLite and MongoDB hooks. Rather than leave this ticket open even longer, I'll file separate tickets for that work.

witten closed this issue 2024-03-03 22:36:37 +00:00

witten referenced this issue 2024-03-03 22:39:44 +00:00

Custom command option for SQLite hook #836

witten referenced this issue 2024-03-03 22:42:11 +00:00

Custom command options for MongoDB hook #837

witten commented 2024-03-11 20:29:51 +00:00

Author

Owner

Copy Link

Released in borgmatic 1.8.9!

Released in borgmatic 1.8.9!

🎉 1

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

config-command-line

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels

blocked

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

No Label

good first issue

Milestone

No items

No Milestone

Assignees

Clear assignees

diivi import_bot witten

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#311

No description provided.