Many borgmatic commands do not return any information #748

New Issue

Wqrld · 2023-08-28T15:25:20Z

Wqrld commented

2023-08-28 15:25:20 +00:00

What I'm trying to do and why

trying to run commands like borgmatic info or borgmatic list

Steps to reproduce

borgbase over ssh://
borg 1.2.4
borgmatic 1.8.2

Actual behavior

running borgmatic info returns no information.

/usr/local/bin/borg --version --debug --show-rc
borgbase: Running actions for repository
/etc/borgmatic/config.yaml: No commands to run for pre-actions hook
borgbase: Displaying archive summary information
/usr/local/bin/borg info --debug --show-rc --glob-archives {hostname}-* ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo
using builtin fallback logging configuration
33 self tests completed in 0.06 seconds
SSH command line: ['ssh', 'q31fkzgr@q31fkzgr.repo.borgbase.com', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./repo'
Remote: Resolved repository path to '/srv/repos/q31fkzgr/repo'
Remote: Verified integrity of /srv/repos/q31fkzgr/repo/index.823
TAM-verified manifest
security: read previous location 'ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo'
security: read manifest timestamp '2023-08-23T19:35:28.880076'
security: determined newest manifest timestamp as 2023-08-23T19:35:28.880076
security: repository checks ok, allowing access
Verified integrity of /root/.cache/borg/0ed277bb80cac771cd58e3c8dc3d2642632658a37bba08290b98f5819f32c0dc/chunks
security: read previous location 'ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo'
security: read manifest timestamp '2023-08-23T19:35:28.880076'
security: determined newest manifest timestamp as 2023-08-23T19:35:28.880076
security: repository checks ok, allowing access
RemoteRepository: 216 B bytes sent, 3.38 kB bytes received, 5 messages sent
terminating with success status, rc 0
/etc/borgmatic/config.yaml: No commands to run for post-actions hook

summary:
/etc/borgmatic/config.yaml: Loading configuration file
/etc/borgmatic/config.yaml: Successfully ran configuration file

weirdly enough, borgmatic info with --archive latest works fine. running borg info manually also works fine.

Expected behavior

should get output but get nothing.

root@ash-bm-1142-ey2:~# borgmatic info
borgbase: Displaying archive summary information

Other notes / implementation ideas

did the archive name fetching system break on newer borg versions?

borgmatic version

1.8.2

borgmatic installation method

pip

Borg version

1.2.4

Python version

3.10.12

Database version (if applicable)

No response

Operating system and version

Ubuntu 22.04.3

### What I'm trying to do and why trying to run commands like borgmatic info or borgmatic list ### Steps to reproduce - borgbase over ssh:// - borg 1.2.4 - borgmatic 1.8.2 ### Actual behavior running borgmatic info returns no information. ``` /usr/local/bin/borg --version --debug --show-rc borgbase: Running actions for repository /etc/borgmatic/config.yaml: No commands to run for pre-actions hook borgbase: Displaying archive summary information /usr/local/bin/borg info --debug --show-rc --glob-archives {hostname}-* ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo using builtin fallback logging configuration 33 self tests completed in 0.06 seconds SSH command line: ['ssh', 'q31fkzgr@q31fkzgr.repo.borgbase.com', 'borg', 'serve', '--debug'] Remote: using builtin fallback logging configuration Remote: borg selftest disabled via BORG_SELFTEST env variable Remote: using builtin fallback logging configuration Remote: Initialized logging system for JSON-based protocol Remote: Resolving repository path b'/./repo' Remote: Resolved repository path to '/srv/repos/q31fkzgr/repo' Remote: Verified integrity of /srv/repos/q31fkzgr/repo/index.823 TAM-verified manifest security: read previous location 'ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo' security: read manifest timestamp '2023-08-23T19:35:28.880076' security: determined newest manifest timestamp as 2023-08-23T19:35:28.880076 security: repository checks ok, allowing access Verified integrity of /root/.cache/borg/0ed277bb80cac771cd58e3c8dc3d2642632658a37bba08290b98f5819f32c0dc/chunks security: read previous location 'ssh://q31fkzgr@q31fkzgr.repo.borgbase.com/./repo' security: read manifest timestamp '2023-08-23T19:35:28.880076' security: determined newest manifest timestamp as 2023-08-23T19:35:28.880076 security: repository checks ok, allowing access RemoteRepository: 216 B bytes sent, 3.38 kB bytes received, 5 messages sent terminating with success status, rc 0 /etc/borgmatic/config.yaml: No commands to run for post-actions hook summary: /etc/borgmatic/config.yaml: Loading configuration file /etc/borgmatic/config.yaml: Successfully ran configuration file ``` weirdly enough, borgmatic info with --archive latest works fine. running `borg info` manually also works fine. ### Expected behavior should get output but get nothing. root@ash-bm-1142-ey2:~# borgmatic info borgbase: Displaying archive summary information ### Other notes / implementation ideas did the archive name fetching system break on newer borg versions? ### borgmatic version 1.8.2 ### borgmatic installation method pip ### Borg version 1.2.4 ### Python version 3.10.12 ### Database version (if applicable) _No response_ ### Operating system and version Ubuntu 22.04.3

Wqrld changed title from ~~Many borgmatic do not return any information~~ to Many borgmatic commands do not return any information

2023-08-28 15:37:39 +00:00

Wqrld commented

2023-08-28 15:40:14 +00:00

Update (luckily you are opensource, yay!):

the code looks for match_archives, but that breaks if the hostname is changed (which in my case happened because we were restoring on a new install). Setting match_archives to '*' fixes the issue.
I would personally think it would be better to default match_archives to * instead of having it depend on the archive_name_format or hostname so this does not scare people trying to do restores out. I am not sure if there are any workflows that this would break, but it is something that i'd consider changing.

Update (luckily you are opensource, yay!): the code looks for match_archives, but that breaks if the hostname is changed (which in my case happened because we were restoring on a new install). Setting match_archives to '*' fixes the issue. I would personally think it would be better to default match_archives to * instead of having it depend on the archive_name_format or hostname so this does not scare people trying to do restores out. I am not sure if there are any workflows that this would break, but it is something that i'd consider changing.

witten commented

2023-08-28 17:15:59 +00:00

Thanks for taking the time to file this one—and debug it yourself! 😄 The current match_archives filtering behavior is actually designed to support a common use case where a user has multiple borgmatic configuration files, each with a different archive_name_format. For that, deriving the match_archives value from the archive_name_format ensures that any given borgmatic info or list command doesn't output the same archives multiple times, because each config file's output gets filtered down to just its own archives. So changing the match_archives default to * would break that use case among others. You can read more about this automatic archive filtering here: https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#archive-filtering

However I can see how this behavior is kind of surprising and obnoxious for your use case, where you've only got a single configuration file and you just wanted to restore your archives to a different host. So one idea is to try to detect this case and display a warning to the user. I'm not sure how feasible this is, since borgmatic doesn't normally parse Borg output, but maybe it could do a separate JSON list/info first and see if there are any archives found matching the current filtering. If not, borgmatic could display a prominent warning that says something like: No matching archives found. Maybe some archive filtering is going on? Try setting --match-archives * if you want to see all archives.

And/or maybe this could just be documented around.

Thoughts?

Thanks for taking the time to file this one—and debug it yourself! 😄 The current `match_archives` filtering behavior is actually designed to support a common use case where a user has [multiple borgmatic configuration files](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/), each with a different `archive_name_format`. For that, deriving the `match_archives` value from the `archive_name_format` ensures that any given `borgmatic info` or `list` command doesn't output the same archives multiple times, because each config file's output gets filtered down to just its own archives. So changing the `match_archives` default to `*` would break that use case among others. You can read more about this automatic archive filtering here: https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#archive-filtering However I can see how this behavior is kind of surprising and obnoxious for your use case, where you've only got a single configuration file and you just wanted to restore your archives to a different host. So one idea is to try to detect this case and display a warning to the user. I'm not sure how feasible this is, since borgmatic doesn't normally parse Borg output, but maybe it could do a separate JSON `list`/`info` first and see if there are any archives found matching the current filtering. If not, borgmatic could display a prominent warning that says something like: `No matching archives found. Maybe some archive filtering is going on? Try setting --match-archives * if you want to see all archives.` And/or maybe this could just be documented around. Thoughts?

witten added the

question / support

label 2023-08-28 17:16:13 +00:00

Wqrld commented

2023-08-28 18:34:47 +00:00

A warning like that is a good idea. I'm not sure how you derive match-archives from archive_name_format, but another idea is to filter out the hostname like you do with the timestamp and only match hardcoded (non-variable) strings in match-archives? In that way a given config file would have a deterministic output independent of the host, but you can still separate your backups by adding a prefix. Parsing the hostname as ephemeral would also not break your usecase of having multiple files.

witten commented

2023-08-28 18:52:32 +00:00

I like that idea as a solution. It stretches the definition of "ephemeral data placeholders" a bit, but presumably auto-converting {hostname} in archive_name_format to * in match_archives wouldn't break the multiple-config use case, because they should all have the same hostname if they're on the same host! In fact, if making this change I would probably change it so all {placeholder}s get replaced with *.

I like that idea as a solution. It stretches the definition of "ephemeral data placeholders" a bit, but presumably auto-converting `{hostname}` in `archive_name_format` to `*` in `match_archives` wouldn't break the multiple-config use case, because they *should* all have the same hostname if they're on the same host! In fact, if making this change I would probably change it so _all_ [`{placeholder}`s](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-placeholders) get replaced with `*`.

witten referenced this issue

2023-09-07 06:19:06 +00:00

borgmatic prune unexpectedly selects all hostnames on default settings #753

witten commented

2023-09-07 06:29:32 +00:00

So I discovered a problem with this solution: It breaks the use case in #753, conveniently filed just recently. Specifically, the user in #753 wanted borgmatic's default archive_name_format of {hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} to be applied to filter archives (as {hostname}-*), so only the archives for a single hostname get pruned on a particular host. Which seemed like a totally reasonable expectation to me.

But if this ticket gets implemented as described above, the archive filter will become *-* instead of {hostname}-*—and pruning will no longer be host-specific. So that makes me think maybe the changes to how archive filtering is derived won't work, and maybe we're back to the warning-based solution. Unless there's some other solution I'm missing...

So I discovered a problem with this solution: It breaks the use case in #753, conveniently filed just recently. Specifically, the user in #753 wanted borgmatic's default `archive_name_format` of `{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}` to be applied to filter archives (as `{hostname}-*`), so only the archives for a single hostname get pruned on a particular host. Which seemed like a totally reasonable expectation to me. But if _this_ ticket gets implemented as described above, the archive filter will become `*-*` instead of `{hostname}-*`—and pruning will no longer be host-specific. So that makes me think maybe the changes to how archive filtering is derived won't work, and maybe we're back to the warning-based solution. Unless there's some other solution I'm missing...

gemarcano commented

2023-09-11 06:41:46 +00:00

As the reporter for #753 who already lost old archives to a prune that was effectively matching all (due to the issue reported in #753), I would prefer that prune's filter not be overly aggressive by default. Prior to 1.8 at some point, borgmatic leveraged the prefix option to control the filtering of pruning, and if archive_name_format is its replacement, I'd hope its semantics are somewhat similar-- else you end up with data loss as what happened to me.

I've seen other software report warnings in cases like this one. A very rough pseudocode for logic for the warnings might be:

get relevant archive data
if (not a custom filter) and (relevant archive data list is empty) and (all archive data is not empty):
        print warning

This way if there truly is nothing in the backup, there's no warning, but otherwise, it's at least a notice to let the user know that there are other archives that the current filter option does not cover. And if the user is explicitly asking for a non-default filter option, maybe don't print the warning, I think.

As the reporter for #753 who already lost old archives to a prune that was effectively matching all (due to the issue reported in #753), I would prefer that prune's filter not be overly aggressive by default. Prior to 1.8 at some point, borgmatic leveraged the `prefix` option to control the filtering of pruning, and if `archive_name_format` is its replacement, I'd hope its semantics are somewhat similar-- else you end up with data loss as what happened to me. I've seen other software report warnings in cases like this one. A very rough pseudocode for logic for the warnings might be: ``` get relevant archive data if (not a custom filter) and (relevant archive data list is empty) and (all archive data is not empty): print warning ``` This way if there truly is nothing in the backup, there's no warning, but otherwise, it's at least a notice to let the user know that there are other archives that the current filter option does not cover. And if the user is explicitly asking for a non-default filter option, maybe don't print the warning, I think.

👍 1

witten commented

2023-09-11 15:54:30 +00:00

That makes sense to me. Thanks for weighing in! Also note that match_archives is more of the replacement for prefix. But archive_name_format is used to auto-populate match_archives if its value is not set.

That makes sense to me. Thanks for weighing in! Also note that `match_archives` is more of the replacement for `prefix`. But `archive_name_format` is used to auto-populate `match_archives` if its value is not set.

witten referenced this issue

2023-10-16 22:08:36 +00:00

Retention policy appears to have no effect #772

kalsan referenced this issue

2023-10-17 06:56:11 +00:00

Retention policy appears to have no effect #772

witten referenced this issue from a commit

2023-10-27 05:12:37 +00:00

When an archive filter causes no matching archives for the "rlist" or "info" actions, warn (#748).

witten commented

2023-10-27 05:16:38 +00:00

This warning has been implemented in main for the "rlist" and "info" actions and will be part of the next release. The only caveat is that it doesn't check that all archive data is not empty as suggested.

Here's an example of the warning in action:

[root@flux tmp]# borgmatic -c test.yaml info --verbosity 1 --match-archives nothing
backupserver: Displaying archive summary information
An archive filter was applied, but no matching archives were found.
Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration.

summary:
/root/tmp/test.yaml: Successfully ran configuration file

This warning has been implemented in main for the "rlist" and "info" actions and will be part of the next release. The only caveat is that it doesn't check that all archive data is not empty as suggested. Here's an example of the warning in action: ``` [root@flux tmp]# borgmatic -c test.yaml info --verbosity 1 --match-archives nothing backupserver: Displaying archive summary information An archive filter was applied, but no matching archives were found. Try adding --match-archives "*" or adjusting archive_name_format/match_archives in configuration. summary: /root/tmp/test.yaml: Successfully ran configuration file ```

witten closed this issue

2023-10-27 05:16:38 +00:00

witten commented

2023-10-27 05:22:22 +00:00

Released in borgmatic 1.8.4!

Sign in to join this conversation.