Borgmatic fails without giving much information #754

New Issue

wxgeo · 2023-09-12T20:20:15Z

wxgeo commented

2023-09-12 20:20:15 +00:00

What I'm trying to do and why

Hi!

I use borgmatic 1.8.2 on top on borg 1.1.15, on a Yunohost 11 (Debian 11) server.

However, backup fail without much explanation, even if I set verbosity to 2.

Steps to reproduce

This is my configuration file (/etc/borgmatic/config.yaml) :

https://www.toptal.com/developers/paste-gd/tiNGgOb4

Actual behavior

When running borgmatic --syslog-verbosity 2, I get the following message:

https://www.toptal.com/developers/paste-gd/I0C11zbS

Expected behavior

I expect backup to work, or at least to get some useful information for debugging.

Other notes / implementation ideas

No response

borgmatic version

1.8.2

borgmatic installation method

pip install

Borg version

1.1.15

Python version

3.9.2

Database version (if applicable)

No response

Operating system and version

Raspbian GNU/Linux 11 (bullseye)

### What I'm trying to do and why Hi! I use borgmatic 1.8.2 on top on borg 1.1.15, on a Yunohost 11 (Debian 11) server. However, backup fail without much explanation, even if I set verbosity to 2. ### Steps to reproduce This is my configuration file (`/etc/borgmatic/config.yaml`) : https://www.toptal.com/developers/paste-gd/tiNGgOb4 ### Actual behavior When running `borgmatic --syslog-verbosity 2`, I get the following message: https://www.toptal.com/developers/paste-gd/I0C11zbS ### Expected behavior I expect backup to work, or at least to get some useful information for debugging. ### Other notes / implementation ideas _No response_ ### borgmatic version 1.8.2 ### borgmatic installation method pip install ### Borg version 1.1.15 ### Python version 3.9.2 ### Database version (if applicable) _No response_ ### Operating system and version Raspbian GNU/Linux 11 (bullseye)

wxgeo commented

2023-09-12 20:29:17 +00:00

OK, I created an exclude_file to directly test borg command.

The problem seems to be the ssh:// syntax for location, which is not supported by borg 1.1.15.

However, it would be nice from borgmatic to display borg message, without having to to create an exclude_file and then to launch the borg command manually.

OK, I created an exclude_file to directly test borg command. The problem seems to be the `ssh://` syntax for location, which is not supported by borg 1.1.15. However, it would be nice from borgmatic to display borg message, without having to to create an exclude_file and then to launch the borg command manually.

wxgeo commented

2023-09-12 20:39:18 +00:00

The last line of borg error message was in fact the most important for debugging, and unfortunately was missing from borgmatic output:

borg create: error: argument ARCHIVE: Invalid location format: "ssh://borg@192.168.1.20:/mnt/backup/data::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}"

(Complete borg message: https://www.toptal.com/developers/paste-gd/DyuwhgFw)

The last line of borg error message was in fact the most important for debugging, and unfortunately was missing from borgmatic output: ```borg create: error: argument ARCHIVE: Invalid location format: "ssh://borg@192.168.1.20:/mnt/backup/data::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}"``` (Complete borg message: https://www.toptal.com/developers/paste-gd/DyuwhgFw)

witten commented

2023-09-12 20:41:10 +00:00

I'm glad you figured out the issue—even if borgmatic wasn't making it easy to do so. I'm a little surprised though that ssh:// repository syntax doesn't work in Borg 1.1.15, as it is listed in the documentation for that version.

In any case, I'll try to reproduce this locally and see if I can make the Borg error message pass through more clearly. Thanks for filing this!

I'm glad you figured out the issue—even if borgmatic wasn't making it easy to do so. I'm a little surprised though that `ssh://` repository syntax doesn't work in Borg 1.1.15, as it is listed in the [documentation for that version](https://borgbackup.readthedocs.io/en/1.1.15/usage/general.html#repository-urls). In any case, I'll try to reproduce this locally and see if I can make the Borg error message pass through more clearly. Thanks for filing this!

witten commented

2023-09-13 16:12:29 +00:00

At least one of these mysteries is cleared up! The ssh:// syntax you were using was incorrect! There should be no colon separating the host address from the path.

That still doesn't solve the error reporting issue you discovered though. And I have been able to reproduce it.. Although the Invalid location format message does show up in the full logs, just not in the borgmatic summary at the bottom.

At least one of these mysteries is cleared up! The `ssh://` syntax you were using was incorrect! There should be no colon separating the host address from the path. That still doesn't solve the error reporting issue you discovered though. And I have been able to reproduce it.. Although the `Invalid location format` message _does_ show up in the full logs, just not in the borgmatic summary at the bottom.

wxgeo commented

2023-09-14 20:10:48 +00:00

Thanks for your investigations !

Concerning the ssh://syntax, maybe borg's documentation should insist there is no colon anymore before the path, since it was mandatory in the previous syntax ?

Although the Invalid location format message does show up in the full logs, just not in the borgmatic summary at the bottom.

Where can I find the full logs ? I thought journactl -xe would help, since it was recommended in borgmatic's documentation, yet I couldn't find any information there. Maybe I missed something ?

Anyway, when borgmatic is launched in the console, the failure reason should directly appear in the console too.

Thanks for your investigations ! Concerning the `ssh://`syntax, maybe borg's documentation should insist there is no colon anymore before the path, since it was mandatory in the previous syntax ? > Although the Invalid location format message does show up in the full logs, just not in the borgmatic summary at the bottom. Where can I find the full logs ? I thought `journactl -xe` would help, since it was recommended in borgmatic's documentation, yet I couldn't find any information there. Maybe I missed something ? Anyway, when borgmatic is launched in the console, the failure reason should directly appear in the console too.

witten commented

2023-09-14 21:23:35 +00:00

Concerning the ssh://syntax, maybe borg's documentation should insist there is no colon anymore before the path, since it was mandatory in the previous syntax ?

That seems like a good idea. It is implied here, but not said explicitly: https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls

You might consider filing a Borg ticket if you feel strongly about it.

Where can I find the full logs ? I thought journactl -xe would help, since it was recommended in borgmatic's documentation, yet I couldn't find any information there. Maybe I missed something ?

In the same place you got the original logs—above the "summary:" part. And the failure reason does appear in the console, just not in the summary. The reason the summary doesn't include everything is because its log entries are intentionally truncated! I'm looking into what options we have there.

> Concerning the ssh://syntax, maybe borg's documentation should insist there is no colon anymore before the path, since it was mandatory in the previous syntax ? That seems like a good idea. It is implied here, but not said explicitly: https://borgbackup.readthedocs.io/en/stable/usage/general.html#repository-urls You might consider filing a Borg ticket if you feel strongly about it. > Where can I find the full logs ? I thought journactl -xe would help, since it was recommended in borgmatic's documentation, yet I couldn't find any information there. Maybe I missed something ? In the same place you got the original logs—above the "summary:" part. And the failure reason does appear in the console, just not in the summary. The reason the summary doesn't include everything is because its log entries are intentionally truncated! I'm looking into what options we have there.

witten added the

bug

label 2023-09-15 04:05:50 +00:00

witten referenced this issue from a commit

2023-09-15 04:11:33 +00:00

Fix error handling to log command output as one record per line (#754).

witten commented

2023-09-15 04:15:05 +00:00

Okay, this is fixed in main! The fix is to log command output as one record per line instead of truncating too-long output and swallowing the end of some Borg error messages. The truncation was there to begin with because many logging backends (like systemd) impose log record size limits, and will themselves truncate long log records or refuse to log. So I couldn't just remove the truncation without coming up with a replacement.

I'm pretty sure this fixes the problem you were seeing, and It'll be part of the next release so you can try it out then. Thanks again for reporting this!

Okay, this is fixed in main! The fix is to log command output as one record per line instead of truncating too-long output and swallowing the end of some Borg error messages. The truncation was there to begin with because many logging backends (like systemd) impose log record size limits, and will themselves truncate long log records or refuse to log. So I couldn't just remove the truncation without coming up with a replacement. I'm pretty sure this fixes the problem you were seeing, and It'll be part of the next release so you can try it out then. Thanks again for reporting this!

witten closed this issue

2023-09-15 04:15:05 +00:00

witten commented

2023-09-30 20:40:56 +00:00

This has been released as part of borgmatic 1.8.3!

Sign in to join this conversation.