MySQL Mariadb: is it possible to have an alternative to pipe for authentication? #1033

New issue

Closed

opened 2025-03-18 14:57:19 +00:00 by paspo · 3 comments

paspo commented

2025-03-18 14:57:19 +00:00

What I'd like to do and why

Even if the use of pipes for passing credentials is good practice, sometimes this is a disadvantage.

Reference:

def make_defaults_file_options(username=None, password=None, defaults_extra_filename=None):

In a scenario with some containers (managed with docker compose, but that's not important), I am backing up a postgresql db of an application, by using this configuration:

postgresql_databases:
  - name: TheDatabaseName
    username: TheUser
    password: AStrongPassword
    pg_dump_command: docker exec postgresql-container-name-1 pg_dump

This works and has several advantages:

the borgmatic container is not on the same network with potentially a lot of different db
I am always using the correct postgres client as the server

I tried to do the same with mariadb but that's not working with a similar configuration:

mariadb_databases:
  - name: TheDatabaseName
    username: TheUser
    password: AStrongPassword
    options: "--single-transaction --quick"
    mariadb_dump_command: docker exec mariadb-container-name-1 mariadb-dump

This configuration is NOT working and borgmatic is exiting with this message:

Could not open required defaults file: /dev/fd/4

This is not directly a borgmatic problem: mariadb-dump is complaining about not being able to load the defaults file: this happens because the pipe is created in borgmatic's container and this is not accessible by the target db container.

docker image: ghcr.io/borgmatic-collective/borgmatic:1.9.14

Other notes / implementation ideas

I know this is an edge case, but it would be great to have a way for the make_defaults_file_options to return an empty string, so that I can use something like:

mariadb_databases:
  - name: TheDatabaseName
    options: "--single-transaction --quick"
    mariadb_dump_command: docker exec mariadb-container-name-1 mariadb-dump -u TheUser -PAStrongPassword

Or a flag to disable the pipe, and use something else (env vars, arguments, ...)

### What I'd like to do and why Even if the use of pipes for passing credentials is good practice, sometimes this is a disadvantage. Reference: https://projects.torsion.org/borgmatic-collective/borgmatic/src/commit/c926f0bd5d75095ac4b70852279c38b68d9a7d9c/borgmatic/hooks/data_source/mariadb.py#L55 In a scenario with some containers (managed with `docker compose`, but that's not important), I am backing up a postgresql db of an application, by using this configuration: ```yaml postgresql_databases: - name: TheDatabaseName username: TheUser password: AStrongPassword pg_dump_command: docker exec postgresql-container-name-1 pg_dump ``` This works and has several advantages: - the borgmatic container is not on the same network with potentially a lot of different db - I am always using the correct postgres client as the server I tried to do the same with mariadb but that's not working with a similar configuration: ```yaml mariadb_databases: - name: TheDatabaseName username: TheUser password: AStrongPassword options: "--single-transaction --quick" mariadb_dump_command: docker exec mariadb-container-name-1 mariadb-dump ``` This configuration is NOT working and borgmatic is exiting with this message: ```sh Could not open required defaults file: /dev/fd/4 ``` This is not directly a borgmatic problem: `mariadb-dump` is complaining about not being able to load the defaults file: this happens because the pipe is created in borgmatic's container and this is not accessible by the target db container. docker image: `ghcr.io/borgmatic-collective/borgmatic:1.9.14` ### Other notes / implementation ideas I know this is an edge case, but it would be great to have a way for the `make_defaults_file_options` to return an empty string, so that I can use something like: ```yaml mariadb_databases: - name: TheDatabaseName options: "--single-transaction --quick" mariadb_dump_command: docker exec mariadb-container-name-1 mariadb-dump -u TheUser -PAStrongPassword ``` Or a flag to disable the pipe, and use something else (env vars, arguments, ...)

witten commented

2025-03-18 17:05:27 +00:00

Owner

mariadb_databases:
  - name: TheDatabaseName
    options: "--single-transaction --quick"
    mariadb_dump_command: docker exec --env MYSQL_PWD mariadb-container-name-1 mariadb-dump

Thanks for bringing this to my attention! And yeah, I think the right solution here is probably a borgmatic configuration option that, when enabled, reverts to the old borgmatic behavior of sending passwords to MariaDB via environment variable and username via CLI flags. Then you would be able to do something like this: ``` mariadb_databases: - name: TheDatabaseName options: "--single-transaction --quick" mariadb_dump_command: docker exec --env MYSQL_PWD mariadb-container-name-1 mariadb-dump ```

👍 1

witten referenced this issue from a commit

2025-04-29 21:19:41 +00:00

Add a "password_transport" option to the MariaDB and MySQL database hooks for customizing how borgmatic transmits passwords to the database client (#1033).

witten referenced this issue from a commit

2025-04-29 21:22:19 +00:00

Fix some outdated config (section headers) in documentation (#1033).

witten commented

2025-04-29 21:23:46 +00:00

Owner

Okay, this is implemented in main and will be part of the next release! The new database option is called password_transport and is implemented for both MariaDB and MySQL database hooks. Example usage:

mariadb_databases:
     - name: users
       hostname: 127.0.0.1
       username: example
       password: trustsome1
       password_transport: environment
       mariadb_dump_command: docker exec --env MYSQL_PWD my_mariadb_container mariadb-dump

Thanks again for suggesting this!

Okay, this is implemented in main and will be part of the next release! The new database option is called `password_transport` and is implemented for both MariaDB and MySQL database hooks. Example usage: ```yaml mariadb_databases: - name: users hostname: 127.0.0.1 username: example password: trustsome1 password_transport: environment mariadb_dump_command: docker exec --env MYSQL_PWD my_mariadb_container mariadb-dump ``` Thanks again for suggesting this!

witten closed this issue

2025-04-29 21:23:46 +00:00

witten commented

2025-04-29 22:23:30 +00:00

Owner

Released in borgmatic 2.0.5!

Documentation is here: https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#database-client-in-a-running-container

Released in borgmatic 2.0.5! Documentation is here: https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#database-client-in-a-running-container