Database backup with e. g. MariaDB to Hetzner StorageBox (Borg) fails

smartinventure commented

2026-04-09 08:01:15 +00:00

What I'm trying to do and why

I ran into a problem with borgmatic 2.1.4 to backup MariaDB databases directly and it may be the same for e. g. Postgresql, etc.

That is the bug I get from the log and below that the analysis from Claude about the issue.

2026-04-08 14:10:38	[2026-04-08 14:10:38,348] CRITICAL: Need some help? https://torsion.org/borgmatic/#issues

2026-04-08 14:10:38	[2026-04-08 14:10:38,348] CRITICAL:
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: Error running configuration
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: Expecting value: line 1 column 1 (char 0)
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: Error running actions for repository
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: An error occurred
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: summary:
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL:
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: An error occurred
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: Error running configuration
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Expecting value: line 1 column 1 (char 0)
2026-04-08 14:10:38	[2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Error running actions for repository
2026-04-08 14:10:37	[2026-04-08 14:10:37,878] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html

2026-04-08 14:10:37	[2026-04-08 14:10:37,878] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks.
2026-04-08 14:10:37	[2026-04-08 14:10:37,878] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm.
2026-04-08 14:10:36	[2026-04-08 14:10:36,516] INFO: Creating archive at "ssh://u533353@u533353.your-storagebox.de:23/./backupspeedbits::d436e9ea4263-2026-04-08T14:10:35.checkpoint"

2026-04-08 14:10:36	[2026-04-08 14:10:36,147] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html

2026-04-08 14:10:36	[2026-04-08 14:10:36,146] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks.
2026-04-08 14:10:36	[2026-04-08 14:10:36,146] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm.
2026-04-08 14:10:35	[2026-04-08 14:10:35,374] INFO: Infinity Tools Repo: Dumping MariaDB databases
2026-04-08 14:10:35	[2026-04-08 14:10:35,349] INFO: Infinity Tools Repo: Creating archive

Steps to reproduce

No response

Actual behavior

Here is the analysis of Claude, what could be the culprit:

What's Actually Happening
Borgmatic's native mariadb_databases: hook does not simply run mariadb-dump to a file and then back it up. It uses a sophisticated FIFO (named pipe) + streaming mechanism:

Borgmatic creates a named pipe (FIFO) at a temp path
It spawns mariadb-dump --result-file=<fifo_path> as a background process
It tells Borg to run with --read-special, which makes Borg read from the FIFO directly
The database dump streams through the pipe straight into the Borg archive -- it never hits disk as a normal file
Borgmatic writes a dumps.json metadata file alongside for use during restore
This streaming approach is clever for large databases (no disk space needed for the dump), but it changes borgmatic's internal control flow significantly. During and after the archive creation, borgmatic does internal JSON parsing of Borg's output for checkpoints, archive renaming, and especially during prune/compact. When database hook stream processes are active, Borg's output can contain mixed text (SSH warnings, checkpoint messages, rename info) that trips up borgmatic's JSON parser -- resulting in the Expecting value: line 1 column 1 (char 0) error.
Why It Still Fails Without --json / --stats
The fix we applied (removing --json and --stats for database backups) addressed the external flags we pass. But borgmatic 2.1.4 also parses borg output internally in several places -- borgmatic/actions/json.py:parse_json() and within prune/compact modules that use --log-json internally. The SSH post-quantum warnings from Hetzner's StorageBox are being emitted by Borg and are polluting those internal JSON streams. Version 2.1.2 partially fixed this (#1260), but clearly not completely for the database hook code path.
The Right Fix (Not a Downgrade)
You're absolutely right that downgrading is not a good approach. And here's the thing -- we already have a working pattern in our codebase for exactly this: the MSSQL database type. Since borgmatic has no native MSSQL support, we handle it via commands: hooks that dump to a temp directory and include that directory in source_directories. This approach:
Completely bypasses borgmatic's internal FIFO/streaming/JSON machinery
Works with any borgmatic version (past, present, future)
Is a standard file-based backup (no --read-special needed)
Gives us full control over the dump process

Expected behavior

No response

Other notes / implementation ideas

No response

borgmatic version

2.1.4

borgmatic installation method

Debian package

Borg version

borg 2.0.0b20

Python version

No response

Database version (if applicable)

No response

Operating system and version

Ubuntu 22

### What I'm trying to do and why I ran into a problem with borgmatic 2.1.4 to backup MariaDB databases directly and it may be the same for e. g. Postgresql, etc. That is the bug I get from the log and below that the analysis from Claude about the issue. 2026-04-08 14:10:38 [2026-04-08 14:10:38,348] CRITICAL: Need some help? https://torsion.org/borgmatic/#issues 2026-04-08 14:10:38 [2026-04-08 14:10:38,348] CRITICAL: 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: Error running configuration 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: Expecting value: line 1 column 1 (char 0) 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: Error running actions for repository 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: An error occurred 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: summary: 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: An error occurred 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: Error running configuration 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Expecting value: line 1 column 1 (char 0) 2026-04-08 14:10:38 [2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Error running actions for repository 2026-04-08 14:10:37 [2026-04-08 14:10:37,878] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html 2026-04-08 14:10:37 [2026-04-08 14:10:37,878] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks. 2026-04-08 14:10:37 [2026-04-08 14:10:37,878] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm. 2026-04-08 14:10:36 [2026-04-08 14:10:36,516] INFO: Creating archive at "ssh://u533353@u533353.your-storagebox.de:23/./backupspeedbits::d436e9ea4263-2026-04-08T14:10:35.checkpoint" 2026-04-08 14:10:36 [2026-04-08 14:10:36,147] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html 2026-04-08 14:10:36 [2026-04-08 14:10:36,146] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks. 2026-04-08 14:10:36 [2026-04-08 14:10:36,146] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm. 2026-04-08 14:10:35 [2026-04-08 14:10:35,374] INFO: Infinity Tools Repo: Dumping MariaDB databases 2026-04-08 14:10:35 [2026-04-08 14:10:35,349] INFO: Infinity Tools Repo: Creating archive ### Steps to reproduce _No response_ ### Actual behavior Here is the analysis of Claude, what could be the culprit: What's Actually Happening Borgmatic's native mariadb_databases: hook does not simply run mariadb-dump to a file and then back it up. It uses a sophisticated FIFO (named pipe) + streaming mechanism: 1. Borgmatic creates a named pipe (FIFO) at a temp path 2. It spawns mariadb-dump --result-file=<fifo_path> as a background process 3. It tells Borg to run with --read-special, which makes Borg read from the FIFO directly 4. The database dump streams through the pipe straight into the Borg archive -- it never hits disk as a normal file 5. Borgmatic writes a dumps.json metadata file alongside for use during restore This streaming approach is clever for large databases (no disk space needed for the dump), but it changes borgmatic's internal control flow significantly. During and after the archive creation, borgmatic does internal JSON parsing of Borg's output for checkpoints, archive renaming, and especially during prune/compact. When database hook stream processes are active, Borg's output can contain mixed text (SSH warnings, checkpoint messages, rename info) that trips up borgmatic's JSON parser -- resulting in the Expecting value: line 1 column 1 (char 0) error. Why It Still Fails Without --json / --stats The fix we applied (removing --json and --stats for database backups) addressed the external flags we pass. But borgmatic 2.1.4 also parses borg output internally in several places -- borgmatic/actions/json.py:parse_json() and within prune/compact modules that use --log-json internally. The SSH post-quantum warnings from Hetzner's StorageBox are being emitted by Borg and are polluting those internal JSON streams. Version 2.1.2 partially fixed this (#1260), but clearly not completely for the database hook code path. The Right Fix (Not a Downgrade) You're absolutely right that downgrading is not a good approach. And here's the thing -- we already have a working pattern in our codebase for exactly this: the MSSQL database type. Since borgmatic has no native MSSQL support, we handle it via commands: hooks that dump to a temp directory and include that directory in source_directories. This approach: 1. Completely bypasses borgmatic's internal FIFO/streaming/JSON machinery 2. Works with any borgmatic version (past, present, future) 3. Is a standard file-based backup (no --read-special needed) 4. Gives us full control over the dump process ### Expected behavior _No response_ ### Other notes / implementation ideas _No response_ ### borgmatic version 2.1.4 ### borgmatic installation method Debian package ### Borg version borg 2.0.0b20 ### Python version _No response_ ### Database version (if applicable) _No response_ ### Operating system and version Ubuntu 22

witten commented

2026-04-09 16:49:51 +00:00

Owner

Thanks for taking the time to file a ticket on this. So far I haven't been able to reproduce the problem. A few questions:

Can I get a look at your borgmatic configuration? Feel free to redact as necessary.
Does this problem go away when you disable the database hooks?
Does the problem occur for other non-MariaDB database hooks?
What does the borgmatic command-line look like when you're running borgmatic to get this error?
Can you include the full logs from one of these borgmatic runs when you get the error and not just the end of the log? Feel free to redact, and you can do a file upload if that's easier for you.
What version of MariaDB are you using?

Thanks for taking the time to file a ticket on this. So far I haven't been able to reproduce the problem. A few questions: - Can I get a look at your borgmatic configuration? Feel free to redact as necessary. - Does this problem go away when you disable the database hooks? - Does the problem occur for other non-MariaDB database hooks? - What does the borgmatic command-line look like when you're running borgmatic to get this error? - Can you include the full logs from one of these borgmatic runs when you get the error and not just the end of the log? Feel free to redact, and you can do a file upload if that's easier for you. - What version of MariaDB are you using?

smartinventure commented

2026-04-10 04:54:46 +00:00

Author

Thank you for the fast reply!

I have gotten the answer with Claude Opus 4.6 (God, way better than I could ever have written it myself!!).

One addition though: If you want to, I can create you a Hetzner Storagebox and send you the credentials in a PM, if you want to test. Hetzner Storageboxes are quite popular, not only in Germany as they are dirt cheap compared to e. g. Azure, AWS, etc AND they support Borg on their side (SSH configuration).

1. Borgmatic configuration (redacted credentials)

local_path: /usr/bin/borg1
source_directories: []
repositories:
  - path: ssh://xxxx@xxxx.your-storagebox.de:23/./backupspeedbits
    label: Infinity Tools Repo
keep_daily: 7
keep_weekly: 4
keep_monthly: 6
keep_yearly: 1
exclude_caches: true
archive_name_format: '{hostname}-{now}'
upload_rate_limit: 0
checks:
  - name: repository
    frequency: 2 weeks
mariadb_databases:
  - name: all
    hostname: wp-db
    port: 3306
    username: xxxx
    password: ${BORGMATIC_UI_DB_PASS_REDACTED_0}
    tls: false
  - name: all
    hostname: bookstack-db
    port: 3306
    username: xxxx
    password: ${BORGMATIC_UI_DB_PASS_REDACTED_1}
    tls: false
  - name: all
    hostname: matomo-db
    port: 3306
    username: xxxx
    password: ${BORGMATIC_UI_DB_PASS_REDACTED_2}
    tls: false

2. Does the problem go away when you disable the database hooks?

Yes, completely. We have a second backup job (file-only, no database hooks) that targets the exact same Hetzner StorageBox repository via the same SSH connection and it always succeeds. Only the database backup job that uses mariadb_databases: fails with the JSON parse error. We also confirmed that switching these same MariaDB sources to a commands: hook approach (dump to temp file, then include that directory in source_directories) works perfectly against the same repo — so the issue is specific to the native FIFO/streaming code path.

3. Does the problem occur for other non-MariaDB database hooks?

We have not tested with PostgreSQL or MySQL hooks against this specific repository, so we cannot confirm. However, given that the error occurs during the prune/compact phase (not during the dump itself), and the suspected cause is SSH warnings polluting Borg's JSON output, we believe it would affect any native database hook when the remote emits non-JSON stderr/stdout during those phases.

4. What does the borgmatic command-line look like?

Our UI runs borgmatic as:

borgmatic --config /app/config/borgmatic.d/SpeedBits\ Database\ Backup.yaml --verbosity 1 --stats --json

We initially used --stats --json for all backups. After seeing the error, we tried removing --stats --json for database backup jobs (running with just --verbosity 1), but the error still occurred. This suggests borgmatic is parsing JSON internally (e.g., during prune/compact or archive rename operations) regardless of whether we pass --json externally.

5. Full logs from one run (chronological order)

[2026-04-08 14:10:35,349] INFO: Infinity Tools Repo: Creating archive
[2026-04-08 14:10:35,374] INFO: Infinity Tools Repo: Dumping MariaDB databases
[2026-04-08 14:10:36,146] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm.
[2026-04-08 14:10:36,146] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks.
[2026-04-08 14:10:36,147] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html
[2026-04-08 14:10:36,516] INFO: Creating archive at "ssh://xxxxx@xxxx.your-storagebox.de:23/./backupspeedbits::d436e9ea4263-2026-04-08T14:10:35.checkpoint"
[2026-04-08 14:10:37,878] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm.
[2026-04-08 14:10:37,878] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks.
[2026-04-08 14:10:37,878] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html
[2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Expecting value: line 1 column 1 (char 0)
[2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Error running actions for repository
[2026-04-08 14:10:38,347] CRITICAL: An error occurred
[2026-04-08 14:10:38,347] CRITICAL: summary:
[2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: An error occurred
[2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: Error running configuration
[2026-04-08 14:10:38,347] CRITICAL: Expecting value: line 1 column 1 (char 0)
[2026-04-08 14:10:38,347] CRITICAL: Error running actions for repository
[2026-04-08 14:10:38,347] CRITICAL: Error running configuration
[2026-04-08 14:10:38,348] CRITICAL: Need some help? https://torsion.org/borgmatic/#issues

Note the pattern: the SSH post-quantum warnings appear twice (once during the initial connection for archive creation, once during a subsequent SSH session — likely prune or compact), and then immediately the JSON parse error fires. The json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) is a Python error that occurs when json.loads() receives an empty string or non-JSON text.

Earlier runs showed the same pattern with additional context — the SSH warnings appear during both the create and prune/compact phases, and the error consistently fires right after warnings during the post-create phase.

6. MariaDB version

The MariaDB servers are running in Docker containers (images based on mariadb:10.11 or mariadb:11). The exact version shouldn't matter though, because mariadb-dump itself completes successfully — the dump data reaches Borg. The JSON parse error occurs later, during borgmatic's internal processing of Borg's output.

Additional context that may help reproduce:

The key ingredient is the Hetzner StorageBox (or any SSH remote whose server emits warnings). Hetzner's restricted SSH shell cannot be upgraded to support post-quantum key exchange, so OpenSSH 9.x+ clients always emit these 3-line warnings on every connection.
Borg version: 1.4.0 (accessed as borg1, configured via local_path: /usr/bin/borg1). Note: my original bug report incorrectly mentioned borg 2.0.0b20 — that's also installed in our container but not used for this backup job.
Borgmatic version: 2.1.4
OS: Alpine Linux 3.23.3 (Docker container)
The error is 100% reproducible on every run against this Hetzner StorageBox with native mariadb_databases: hooks. It has never succeeded.
A file-only backup to the exact same repo with the same SSH key always succeeds.

One important correction for you to make in the original GitHub issue: I had listed Borg version as 2.0.0b20, but the actual backup job uses local_path: /usr/bin/borg1 which is Borg 1.4.0.

Thank you for the fast reply! I have gotten the answer with Claude Opus 4.6 (God, way better than I could ever have written it myself!!). One addition though: If you want to, I can create you a Hetzner Storagebox and send you the credentials in a PM, if you want to test. Hetzner Storageboxes are quite popular, not only in Germany as they are dirt cheap compared to e. g. Azure, AWS, etc AND they support Borg on their side (SSH configuration). **1. Borgmatic configuration (redacted credentials)** ```yaml local_path: /usr/bin/borg1 source_directories: [] repositories: - path: ssh://xxxx@xxxx.your-storagebox.de:23/./backupspeedbits label: Infinity Tools Repo keep_daily: 7 keep_weekly: 4 keep_monthly: 6 keep_yearly: 1 exclude_caches: true archive_name_format: '{hostname}-{now}' upload_rate_limit: 0 checks: - name: repository frequency: 2 weeks mariadb_databases: - name: all hostname: wp-db port: 3306 username: xxxx password: ${BORGMATIC_UI_DB_PASS_REDACTED_0} tls: false - name: all hostname: bookstack-db port: 3306 username: xxxx password: ${BORGMATIC_UI_DB_PASS_REDACTED_1} tls: false - name: all hostname: matomo-db port: 3306 username: xxxx password: ${BORGMATIC_UI_DB_PASS_REDACTED_2} tls: false ``` **2. Does the problem go away when you disable the database hooks?** Yes, completely. We have a second backup job (file-only, no database hooks) that targets the **exact same Hetzner StorageBox repository** via the same SSH connection and it always succeeds. Only the database backup job that uses `mariadb_databases:` fails with the JSON parse error. We also confirmed that switching these same MariaDB sources to a `commands:` hook approach (dump to temp file, then include that directory in `source_directories`) works perfectly against the same repo — so the issue is specific to the native FIFO/streaming code path. **3. Does the problem occur for other non-MariaDB database hooks?** We have not tested with PostgreSQL or MySQL hooks against this specific repository, so we cannot confirm. However, given that the error occurs during the prune/compact phase (not during the dump itself), and the suspected cause is SSH warnings polluting Borg's JSON output, we believe it would affect any native database hook when the remote emits non-JSON stderr/stdout during those phases. **4. What does the borgmatic command-line look like?** Our UI runs borgmatic as: ``` borgmatic --config /app/config/borgmatic.d/SpeedBits\ Database\ Backup.yaml --verbosity 1 --stats --json ``` We initially used `--stats --json` for all backups. After seeing the error, we tried removing `--stats --json` for database backup jobs (running with just `--verbosity 1`), but the error still occurred. This suggests borgmatic is parsing JSON internally (e.g., during prune/compact or archive rename operations) regardless of whether we pass `--json` externally. **5. Full logs from one run (chronological order)** ``` [2026-04-08 14:10:35,349] INFO: Infinity Tools Repo: Creating archive [2026-04-08 14:10:35,374] INFO: Infinity Tools Repo: Dumping MariaDB databases [2026-04-08 14:10:36,146] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm. [2026-04-08 14:10:36,146] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks. [2026-04-08 14:10:36,147] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html [2026-04-08 14:10:36,516] INFO: Creating archive at "ssh://xxxxx@xxxx.your-storagebox.de:23/./backupspeedbits::d436e9ea4263-2026-04-08T14:10:35.checkpoint" [2026-04-08 14:10:37,878] WARNING: Remote: ** WARNING: connection is not using a post-quantum key exchange algorithm. [2026-04-08 14:10:37,878] WARNING: Remote: ** This session may be vulnerable to "store now, decrypt later" attacks. [2026-04-08 14:10:37,878] WARNING: Remote: ** The server may need to be upgraded. See https://openssh.com/pq.html [2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Expecting value: line 1 column 1 (char 0) [2026-04-08 14:10:38,347] CRITICAL: Infinity Tools Repo: Error running actions for repository [2026-04-08 14:10:38,347] CRITICAL: An error occurred [2026-04-08 14:10:38,347] CRITICAL: summary: [2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: An error occurred [2026-04-08 14:10:38,347] CRITICAL: /app/config/borgmatic.d/SpeedBits Database Backup.yaml: Error running configuration [2026-04-08 14:10:38,347] CRITICAL: Expecting value: line 1 column 1 (char 0) [2026-04-08 14:10:38,347] CRITICAL: Error running actions for repository [2026-04-08 14:10:38,347] CRITICAL: Error running configuration [2026-04-08 14:10:38,348] CRITICAL: Need some help? https://torsion.org/borgmatic/#issues ``` Note the pattern: the SSH post-quantum warnings appear twice (once during the initial connection for archive creation, once during a subsequent SSH session — likely prune or compact), and then immediately the JSON parse error fires. The `json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)` is a Python error that occurs when `json.loads()` receives an empty string or non-JSON text. Earlier runs showed the same pattern with additional context — the SSH warnings appear during both the create and prune/compact phases, and the error consistently fires right after warnings during the post-create phase. **6. MariaDB version** The MariaDB servers are running in Docker containers (images based on `mariadb:10.11` or `mariadb:11`). The exact version shouldn't matter though, because `mariadb-dump` itself completes successfully — the dump data reaches Borg. The JSON parse error occurs later, during borgmatic's internal processing of Borg's output. **Additional context that may help reproduce:** - The key ingredient is the **Hetzner StorageBox** (or any SSH remote whose server emits warnings). Hetzner's restricted SSH shell cannot be upgraded to support post-quantum key exchange, so OpenSSH 9.x+ clients always emit these 3-line warnings on every connection. - Borg version: **1.4.0** (accessed as `borg1`, configured via `local_path: /usr/bin/borg1`). *Note: my original bug report incorrectly mentioned borg 2.0.0b20 — that's also installed in our container but not used for this backup job.* - Borgmatic version: **2.1.4** - OS: Alpine Linux 3.23.3 (Docker container) - The error is **100% reproducible** on every run against this Hetzner StorageBox with native `mariadb_databases:` hooks. It has never succeeded. - A file-only backup to the exact same repo with the same SSH key always succeeds. --- One important correction for you to make in the original GitHub issue: I had listed Borg version as `2.0.0b20`, but the actual backup job uses `local_path: /usr/bin/borg1` which is **Borg 1.4.0**.

smartinventure commented

2026-04-10 05:06:07 +00:00

Author

PS: I am working on the "Borgmatic (Director) UI", a project for a great Web UI for Borgmatic. It seems to be working quite well (still in testing phase myself).
I made it open source and free for personal use (community edition). There is a commercial version for added "Director Mode" to orchestrate multiple servers.

Dan was so nice to put a link on the borgmatic website to the project:
https://github.com/SpeedbitsInfinityTools/borgmatic-ui-community

PS: I am working on the "Borgmatic (Director) UI", a project for a great Web UI for Borgmatic. It seems to be working quite well (still in testing phase myself). I made it open source and free for personal use (community edition). There is a commercial version for added "Director Mode" to orchestrate multiple servers. Dan was so nice to put a link on the borgmatic website to the project: https://github.com/SpeedbitsInfinityTools/borgmatic-ui-community

witten commented

2026-04-10 05:52:08 +00:00

Owner

Thanks for including all of these details. The good news is I got a repro! I appreciate the offer of Hetzner Storage Box credentials, but it turned out I have access to another server that gives the same post-quantum SSH warning, and the error reproduces when using a remote repo there. Interestingly, it reproduces even without the MariaDB hook enabled. So hopefully this is indeed the same issue and not just one that appears very similar.

I'll dig into this and hopefully figure out what's going on.

This suggests borgmatic is parsing JSON internally (e.g., during prune/compact or archive rename operations) regardless of whether we pass --json externally.

That's correct. borgmatic gets logs from Borg in JSON (unconditionally) and then displays it to the user as either JSON or plaintext depending on what they asked for.

Also, in the future please provide --verbosity 2 logs in tickets. I should've mentioned that to begin with. No need to do that now, since I have a local repro.

Thanks for including all of these details. The good news is I got a repro! I appreciate the offer of Hetzner Storage Box credentials, but it turned out I have access to another server that gives the same post-quantum SSH warning, and the error reproduces when using a remote repo there. Interestingly, it reproduces even without the MariaDB hook enabled. So hopefully this is indeed the same issue and not just one that appears very similar. I'll dig into this and hopefully figure out what's going on. > This suggests borgmatic is parsing JSON internally (e.g., during prune/compact or archive rename operations) regardless of whether we pass --json externally. That's correct. borgmatic gets logs from Borg in JSON (unconditionally) and then displays it to the user as either JSON or plaintext depending on what they asked for. Also, in the future please provide `--verbosity 2` logs in tickets. I should've mentioned that to begin with. No need to do that now, since I have a local repro.

witten added the

bug

label

2026-04-10 05:52:34 +00:00

witten commented

2026-04-10 05:54:48 +00:00

Owner

Actually, I take that last part back. If you could provide --verbosity 2 logs, that would help me confirm that I've indeed reproduced the same issue you're experiencing rather than some related issue. Thanks!

Actually, I take that last part back. If you could provide `--verbosity 2` logs, that would help me confirm that I've indeed reproduced the same issue you're experiencing rather than some related issue. Thanks!

witten referenced this issue from a commit

2026-04-11 03:28:35 +00:00

Fix a regression in which SSH warnings from remote repositories broke the "spot" check (#1294).

witten commented

2026-04-11 03:29:26 +00:00

Owner

Looks like this was a regression caused by changes from #1264. I have a fix in main that will be part of the next release. Thanks for your patience here!

witten closed this issue

2026-04-11 03:29:26 +00:00

smartinventure commented

2026-04-11 04:58:03 +00:00

Author

Ok, so you dont need the --verbosity 2?!
Last question: how often do you release, i. e. when appr. would you expect the next release?
Thank you for your great work!!!

Ok, so you dont need the --verbosity 2?! Last question: how often do you release, i. e. when appr. would you expect the next release? Thank you for your great work!!!

witten commented

2026-04-11 05:25:44 +00:00

Owner

The --verbosity 2 log would still be useful to confirm. And releases are usually every couple of weeks, although that can vary. If you need a workaround in the meantime... I suspect that the problem for you is occurring within the check action, so you could try skipping checks.

The `--verbosity 2` log would still be useful to confirm. And releases are usually every couple of weeks, although that can vary. If you need a workaround in the meantime... I suspect that the problem for you is occurring within the `check` action, so you could try skipping checks.