WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login.

mrclschstr commented

2025-02-24 21:51:08 +00:00

What I'm trying to do and why

I use borgmatic in a container to back up a containerized MariaDB 10.6.X database. I am currently still using borgmatic 1.8.X and am currently testing the update to borgmatic 1.9.X. During the tests I noticed the following warning during the backup:

WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login.

I can't explain where this warning comes from, but I have the assumption that it is from “mariadb-dump”. Should I be worried? Can I get rid of the warning somehow?

Steps to reproduce

Well, nothing special if you ask me, but die database part looks like this:

mariadb_databases:
  - name: ${MARIADB_DATABASE}
    hostname: mariadb
    username: ${MARIADB_USER}
    password: ${MARIADB_PASSWORD}
    options: "--single-transaction --default-character-set=utf8mb4"

Do you need the full config file?

Actual behavior

I create the backups with the following command:

docker compose exec -T borgmatic borgmatic --stats --verbosity 1

The warning mentioned above can be found between the normal backup output. The backup itself does not throw any further errors.

Expected behavior

The backup function is not affected.

Other notes / implementation ideas

No response

borgmatic version

1.9.12

borgmatic installation method

Container

Borg version

1.4.0

Python version

3.13.2

Database version (if applicable)

MariaDB 10.6.21

Operating system and version

Ubuntu 22.04.5 LTS

### What I'm trying to do and why I use borgmatic in a container to back up a containerized MariaDB 10.6.X database. I am currently still using borgmatic 1.8.X and am currently testing the update to borgmatic 1.9.X. During the tests I noticed the following warning during the backup: ``` WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login. ``` I can't explain where this warning comes from, but I have the assumption that it is from “mariadb-dump”. Should I be worried? Can I get rid of the warning somehow? ### Steps to reproduce Well, nothing special if you ask me, but die database part looks like this: ```yaml mariadb_databases: - name: ${MARIADB_DATABASE} hostname: mariadb username: ${MARIADB_USER} password: ${MARIADB_PASSWORD} options: "--single-transaction --default-character-set=utf8mb4" ``` Do you need the full config file? ### Actual behavior I create the backups with the following command: ``` docker compose exec -T borgmatic borgmatic --stats --verbosity 1 ``` The warning mentioned above can be found between the normal backup output. The backup itself does not throw any further errors. ### Expected behavior The backup function is not affected. ### Other notes / implementation ideas _No response_ ### borgmatic version 1.9.12 ### borgmatic installation method Container ### Borg version 1.4.0 ### Python version 3.13.2 ### Database version (if applicable) MariaDB 10.6.21 ### Operating system and version Ubuntu 22.04.5 LTS

mrclschstr referenced this issue

2025-02-24 22:08:55 +00:00

[Errno 20] Not a directory #1007

witten commented

2025-02-25 00:53:02 +00:00

Owner

This warning is coming from either the mariadb-dump or the mariadb command (which borgmatic uses for listing databases). My guess is that the warning is showing up because you're making a non-TLS connection to your containerized database, and not anything to do with the password. But IMO you shouldn't need to use TLS for that internal connection—assuming everything's running on the same host. So my recommendation would be to suppress the warning with --skip-ssl-verify-server-cert. Unfortunately you'll need to add that to three different options, because borgmatic runs three different MariaDB commands and each can have different flags:

mariadb_databases:
  - name: ${MARIADB_DATABASE}
    ...
    options: "--single-transaction --default-character-set=utf8mb4 --skip-ssl-verify-server-cert"
    list_options: "--skip-ssl-verify-server-cert"
    restore_options: "--skip-ssl-verify-server-cert"

As an enhancement, it may be a good idea to support this as a "native" borgmatic option like verify_tls or similar. That way you wouldn't need to modify the flags manually.

This warning is coming from either the `mariadb-dump` or the `mariadb` command (which borgmatic uses for listing databases). My guess is that the warning is showing up because you're making a non-TLS connection to your containerized database, and not anything to do with the password. But IMO you shouldn't need to use TLS for that internal connection—assuming everything's running on the same host. So my recommendation would be to suppress the warning with `--skip-ssl-verify-server-cert`. Unfortunately you'll need to add that to three different options, because borgmatic runs three different MariaDB commands and each can have different flags: ``` mariadb_databases: - name: ${MARIADB_DATABASE} ... options: "--single-transaction --default-character-set=utf8mb4 --skip-ssl-verify-server-cert" list_options: "--skip-ssl-verify-server-cert" restore_options: "--skip-ssl-verify-server-cert" ``` As an enhancement, it may be a good idea to support this as a "native" borgmatic option like `verify_tls` or similar. That way you wouldn't need to modify the flags manually.

witten added the

question / support

label

2025-02-25 00:53:12 +00:00

mrclschstr commented

2025-02-25 08:12:43 +00:00

Author

Thanks again for the quick reply and I will definitely test the flags. However, I think I need to do some more research here. I find it very strange that the warning only appears during backup and not during restore. In addition, the warning is always displayed between the stats:

------------------------------------------------------------------------------
Repository: ssh://XXX@XXX/./borg_repo_test
Archive name: XXX-2025-02-25T08:44:18.606729
Archive fingerprint: XXX
Time (start): Tue, 2025-02-25 08:44:19
Time (end):   Tue, 2025-02-25 08:44:26
Duration: 7.00 seconds
Number of files: 4318
Utilization of max. archive size: 0%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              240.33 MB            117.03 MB             68.95 kB
All archives:              769.60 MB            509.63 MB            116.77 MB
WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login.
                       Unique chunks         Total chunks
Chunk index:                    4313                 9610
------------------------------------------------------------------------------
RemoteRepository: 371.03 kB bytes sent, 377.13 kB bytes received, 57 messages sent

Maybe it has nothing to do with mariadb-dump? I'll let you know if I can find out anything else.

Thanks again for the quick reply and I will definitely test the flags. However, I think I need to do some more research here. I find it very strange that the warning only appears during backup and not during restore. In addition, the warning is always displayed between the stats: ``` ------------------------------------------------------------------------------ Repository: ssh://XXX@XXX/./borg_repo_test Archive name: XXX-2025-02-25T08:44:18.606729 Archive fingerprint: XXX Time (start): Tue, 2025-02-25 08:44:19 Time (end): Tue, 2025-02-25 08:44:26 Duration: 7.00 seconds Number of files: 4318 Utilization of max. archive size: 0% ------------------------------------------------------------------------------ Original size Compressed size Deduplicated size This archive: 240.33 MB 117.03 MB 68.95 kB All archives: 769.60 MB 509.63 MB 116.77 MB WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login. Unique chunks Total chunks Chunk index: 4313 9610 ------------------------------------------------------------------------------ RemoteRepository: 371.03 kB bytes sent, 377.13 kB bytes received, 57 messages sent ``` Maybe it has nothing to do with `mariadb-dump`? I'll let you know if I can find out anything else.

mrclschstr commented

2025-02-25 09:25:52 +00:00

Author

A quick and dirty test has shown that --skip-ssl-verify-server-cert is sufficient for options:, so that no more warnings are displayed during both backup and restore:

mariadb_databases:
  - name: ${MARIADB_DATABASE}
    ...
    options: "--single-transaction --default-character-set=utf8mb4 --skip-ssl-verify-server-cert"

However, I don't quite understand where the message has been coming from recently...

A quick and dirty test has shown that `--skip-ssl-verify-server-cert` is sufficient for `options:`, so that no more warnings are displayed during both backup and restore: ``` mariadb_databases: - name: ${MARIADB_DATABASE} ... options: "--single-transaction --default-character-set=utf8mb4 --skip-ssl-verify-server-cert" ``` However, I don't quite understand where the message has been coming from recently...

witten commented

2025-02-25 17:36:24 +00:00

Owner

If adding --skip-ssl-verify-server-cert to options: suppresses the warning, then it's almost certainly coming from mariadb-dump. And yes, it's odd that MariaDB doesn't similarly complain during restore. But my guess as to why the warning is interleaved with the Borg output is because borgmatic runs mariadb-dump and Borg simultaneously so as to stream MariaDB dump output directly to Borg without taking up any disk space. The part I don't understand is why the warning appears so late in the process; I would perhaps expect it to show up earlier, before the stats are displayed.

FYI here's the code that issues that warning in MariaDB: https://github.com/MariaDB/server/blob/main/include/sslopt-vars.h#L61

And here's where it's "called': https://github.com/search?q=repo%3AMariaDB%2Fserver%20SET_SSL_OPTS_WITH_CHECK&type=code

If you wanted to try to reproduce the warning without borgmatic, you could first run borgmatic with --verbosity 2, copy the mariadb-dump command, and then paste it manually into a shell—taking care to set your password into the MYSQL_PWD environment variable and maybe changing the dump filename so as not to interfere with actual borgmatic dumps.

If adding `--skip-ssl-verify-server-cert` to `options:` suppresses the warning, then it's almost certainly coming from `mariadb-dump`. And yes, it's odd that MariaDB doesn't similarly complain during restore. But my guess as to why the warning is interleaved with the Borg output is because borgmatic runs `mariadb-dump` and Borg *simultaneously* so as to stream MariaDB dump output directly to Borg without taking up any disk space. The part I don't understand is why the warning appears so late in the process; I would perhaps expect it to show up earlier, before the stats are displayed. FYI here's the code that issues that warning in MariaDB: https://github.com/MariaDB/server/blob/main/include/sslopt-vars.h#L61 And here's where it's "called': https://github.com/search?q=repo%3AMariaDB%2Fserver%20SET_SSL_OPTS_WITH_CHECK&type=code If you wanted to try to reproduce the warning without borgmatic, you could first run borgmatic with `--verbosity 2`, copy the `mariadb-dump` command, and then paste it manually into a shell—taking care to set your password into the `MYSQL_PWD` environment variable and maybe changing the dump filename so as not to interfere with actual borgmatic dumps.

mrclschstr commented

2025-02-25 20:53:20 +00:00

Author

I followed your suggestion and extracted the mariadb-dump command. As the output shows, this is definitely where the warning comes from. It is still a mystery to me why the message appears so late in the output, but we can leave that aside for now.

testcloud:/tmp# MYSQL_PWD=123456 mariadb-dump --single-transaction --default-character-set=utf8mb4 --add-drop-database --host mariadb --protocol tcp --user nextcloud --databases nextcloud --result-file /tmp/nextcloud
WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login.

If I add the option --skip-ssl-verify-server-cert, the warning no longer appears. I think this clearly shows that the error is not with borgmatic. I will continue to investigate whether this is a bug.

Many thanks for your help!

I followed your suggestion and extracted the `mariadb-dump` command. As the output shows, this is definitely where the warning comes from. It is still a mystery to me why the message appears so late in the output, but we can leave that aside for now. ```bash testcloud:/tmp# MYSQL_PWD=123456 mariadb-dump --single-transaction --default-character-set=utf8mb4 --add-drop-database --host mariadb --protocol tcp --user nextcloud --databases nextcloud --result-file /tmp/nextcloud WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login. ``` If I add the option `--skip-ssl-verify-server-cert`, the warning no longer appears. I think this clearly shows that the error is not with borgmatic. I will continue to investigate whether this is a bug. Many thanks for your help!

mrclschstr commented

2025-02-25 21:15:18 +00:00

Author

Small addition: I have also tried the same command with other MariaDB containers in version 10.11 and 11.4, but here the same warning appears. The message is therefore not related to the MariaDB version used.

My current working theory: The command mariadb-dump has a bug and does not recognize that the password was passed via the environment variable MYSQL_PWD.

testcloud:/# mariadb-dump --version
mariadb-dump from 11.4.4-MariaDB, client 10.19 for Linux (x86_64)

EDIT: If I execute the command in the following form (with --password option), no warning is thrown:

mariadb-dump --single-transaction --default-character-set=utf8mb4 --add-drop-database --host mariadb --protocol tcp --user nextcloud --password=123456 --databases nextcloud --result-file /tmp/nextcloud

Looks like I'm on the right track...

Small addition: I have also tried the same command with other MariaDB containers in version 10.11 and 11.4, but here the same warning appears. The message is therefore not related to the MariaDB version used. My current working theory: The command `mariadb-dump` has a bug and does not recognize that the password was passed via the environment variable `MYSQL_PWD`. ```bash testcloud:/# mariadb-dump --version mariadb-dump from 11.4.4-MariaDB, client 10.19 for Linux (x86_64) ``` **EDIT:** If I execute the command in the following form (with `--password` option), no warning is thrown: ``` mariadb-dump --single-transaction --default-character-set=utf8mb4 --add-drop-database --host mariadb --protocol tcp --user nextcloud --password=123456 --databases nextcloud --result-file /tmp/nextcloud ``` Looks like I'm on the right track...

witten commented

2025-02-25 21:51:50 +00:00

Owner

My current working theory: The command mariadb-dump has a bug and does not recognize that the password was passed via the environment variable MYSQL_PWD

I think you are probably right. Looking at the MariaDB code linked above, it appears that opt_password comes from a tty (so, interactive) password.

Side note: I would be happy to enhance the MariaDB hook to use a mechanism more secure than an environment variable to get the password from borgmatic to MariaDB. I recently made a related change for the Borg encryption passphrase.

> My current working theory: The command mariadb-dump has a bug and does not recognize that the password was passed via the environment variable MYSQL_PWD I think you are probably right. Looking at the [MariaDB code](https://github.com/MariaDB/server/blob/main/include/sslopt-vars.h#L61) linked above, it appears that `opt_password` comes from a tty (so, interactive) password. Side note: I would be happy to enhance the MariaDB hook to use a mechanism more secure than an environment variable to get the password from borgmatic to MariaDB. I recently made a related change for the Borg encryption passphrase.

mrclschstr commented

2025-02-25 22:13:31 +00:00

Author

I think I've got a bingo: github.com/MariaDB/server@853bdf576f and github.com/MariaDB/server@e951edd80b

For now I'm happy with the --skip-ssl-verify-server-cert option, but apparently passing passwords via environment variables is no longer considered secure by the MariaDB community. Establishing a new mechanism certainly makes sense.

Thanks again for your help. I think this issue can be closed.

I think I've got a bingo: https://github.com/MariaDB/server/commit/853bdf576fab92a8cfc9205a60154c6848aa7133 and https://github.com/MariaDB/server/commit/e951edd80bcb8bfc1ec6734848590c6ea0bdd00a For now I'm happy with the `--skip-ssl-verify-server-cert` option, but apparently passing passwords via environment variables is no longer considered secure by the MariaDB community. Establishing a new mechanism certainly makes sense. Thanks again for your help. I think this issue can be closed.

witten commented

2025-02-25 22:28:45 +00:00

Owner

I've created a ticket to come up with a better approach than environment variables for that: #1009.

Glad to hear the workaround is sufficient for now!

I've created a ticket to come up with a better approach than environment variables for that: #1009. Glad to hear the workaround is sufficient for now!

❤️ 1

witten closed this issue

2025-02-25 22:28:48 +00:00

Rows
Columns

WARNING: option --ssl-verify-server-cert is disabled, because of an insecure passwordless login. #1006

What I'm trying to do and why

Steps to reproduce

Actual behavior

Expected behavior

Other notes / implementation ideas

borgmatic version

borgmatic installation method

Borg version

Python version

Database version (if applicable)

Operating system and version