borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 53 Pull Requests 1 Actions Packages Releases 121 Activity

mysql restore does not allow extra options #630

New Issue
Closed
opened 2023-01-19 18:56:57 +00:00 by LordRalex · 5 comments
LordRalex commented 2023-01-19 18:56:57 +00:00
Copy Link

What I'm trying to do and why

I have my db passwords stored in Docker secrets, which I use a before hook to write to a config file for mysql to read from.

This is due to the limitations that borgmatic doesn't permit passwords to be stored in files, so this was the workaround I had.

This worked to make backups, and I've confirmed said backups are functional when doing extracts.

However, when I actually needed to restore the databases, I tried to use the built-in restore to perform this, only for it to fail to get to the database with it complaining that the password was not specified.

Steps to reproduce (if a bug)

Config contains following:

hooks:
    before_everything:
        - printf "[client]\npassword=" | tee db.cnf >/dev/null
        - cat /run/secrets/core_db_pw >> db.cnf
        
    mysql_databases:
        - name: database
          hostname: database
          username: user
          list_options: --defaults-extra-file=db.cnf
          options: --defaults-extra-file=db.cnf --skip-comments

borgmatic creates backups of this database as expected.
Attempt to use borgmatic restore to restore the latest version of this database

Actual behavior (if a bug)

borgmatic fails to connect to database on restore. Command output does not show any options being passed in.

Expected behavior (if a bug)

It actually does the restore using the config without manual intervention.

Other notes / implementation ideas

Per the docs, it only indicates there are 2 options supported: List and dump. It does not indicate the actual restore allows for setting properties. In this scenario, it's not possible to use borgmatic to do the full restore. Rather, I have to extract the backup, and then do it via command-line to complete the restoration.

Environment

borgmatic version: 1.7.5

borgmatic installation method: Docker swarm container, based on b3vis/borgmatic

Borg version: 1.2.2

Python version: 3.10.8

Use python3 --version

Database version (if applicable):
(client) mysql Ver 15.1 Distrib 10.6.11-MariaDB, for Linux (aarch64) using readline 5.1
(server) mysql Ver 15.1 Distrib 10.10.2-MariaDB, for debian-linux-gnu (aarch64) using EditLine wrapper

Use psql --version or mysql --version on client and server.

operating system and version: Linux 5.11.0-1028-oracle #31~20.04.1-Ubuntu SMP Wed Jan 26 14:20:52 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

#### What I'm trying to do and why I have my db passwords stored in Docker secrets, which I use a before hook to write to a config file for mysql to read from. This is due to the limitations that borgmatic doesn't permit passwords to be stored in files, so this was the workaround I had. This worked to make backups, and I've confirmed said backups are functional when doing extracts. However, when I actually needed to restore the databases, I tried to use the built-in restore to perform this, only for it to fail to get to the database with it complaining that the password was not specified. #### Steps to reproduce (if a bug) Config contains following: ``` hooks: before_everything: - printf "[client]\npassword=" | tee db.cnf >/dev/null - cat /run/secrets/core_db_pw >> db.cnf mysql_databases: - name: database hostname: database username: user list_options: --defaults-extra-file=db.cnf options: --defaults-extra-file=db.cnf --skip-comments ``` borgmatic creates backups of this database as expected. Attempt to use borgmatic restore to restore the latest version of this database #### Actual behavior (if a bug) borgmatic fails to connect to database on restore. Command output does not show any options being passed in. #### Expected behavior (if a bug) It actually does the restore using the config without manual intervention. #### Other notes / implementation ideas Per the docs, it only indicates there are 2 options supported: List and dump. It does not indicate the actual restore allows for setting properties. In this scenario, it's not possible to use borgmatic to do the full restore. Rather, I have to extract the backup, and then do it via command-line to complete the restoration. #### Environment **borgmatic version:** 1.7.5 **borgmatic installation method:** Docker swarm container, based on b3vis/borgmatic **Borg version:** 1.2.2 **Python version:** 3.10.8 Use `python3 --version` **Database version (if applicable):** (client) mysql Ver 15.1 Distrib 10.6.11-MariaDB, for Linux (aarch64) using readline 5.1 (server) mysql Ver 15.1 Distrib 10.10.2-MariaDB, for debian-linux-gnu (aarch64) using EditLine wrapper Use `psql --version` or `mysql --version` on client and server. **operating system and version:** Linux 5.11.0-1028-oracle #31~20.04.1-Ubuntu SMP Wed Jan 26 14:20:52 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
witten commented 2023-01-19 19:05:16 +00:00
Owner
Copy Link

Thanks for taking the time to file this! It sounds like a restore_options option would solve your need here?

Thanks for taking the time to file this! It sounds like a `restore_options` option would solve your need here?
LordRalex commented 2023-01-19 19:06:10 +00:00
Author
Copy Link

Yep, that indeed does sound like it would solve my needs.

Yep, that indeed does sound like it would solve my needs.
witten commented 2023-01-19 19:07:10 +00:00
Owner
Copy Link

Awesome! That should hopefully be pretty straight-forward to add.

Awesome! That should hopefully be pretty straight-forward to add.
witten added the
design finalized
 label 2023-01-19 19:07:20 +00:00
witten commented 2023-01-26 23:05:02 +00:00
Owner
Copy Link

This is implemented in master with new configuration options for database command customization: list_options, restore_options, and analyze_options for PostgreSQL, restore_options for MySQL, and restore_options for MongoDB. It should hopefully be released soon! Thanks again for suggesting it.

This is implemented in master with new configuration options for database command customization: `list_options`, `restore_options`, and `analyze_options` for PostgreSQL, `restore_options` for MySQL, and `restore_options` for MongoDB. It should hopefully be released soon! Thanks again for suggesting it.
witten commented 2023-01-26 23:24:26 +00:00
Owner
Copy Link

Just released in borgmatic 1.7.6!

Just released in borgmatic 1.7.6!
Sign in to join this conversation.
Branches Tags
Clear current reference
main
spot-check
Clear current reference
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#630
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?