borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 54 Pull Requests 1 Actions Packages Releases 121 Activity

borgmatic seems to be confused re --glob-archives and --match-archives #666

New Issue
Closed
opened 2023-04-04 01:52:26 +00:00 by lingfish ยท 9 comments
lingfish commented 2023-04-04 01:52:26 +00:00
Copy Link

What I'm trying to do and why

Trying to run borgmatic info with the -a argument.

Steps to reproduce (if a bug)

Run borgmatic -v2 -c /etc/borgmatic/config.yaml info -a '*a*'

Actual behavior (if a bug)

Ensuring legacy configuration is upgraded
borg --version --debug --show-rc
/etc/borgmatic/config.yaml: No commands to run for pre-actions hook
ssh://host/./NAS: Displaying archive summary information
borg info --debug --show-rc --match-archives *a* ssh://host/./NAS
usage: borg [-V] [-h] [--critical] [--error] [--warning] [--info] [--debug]
            [--debug-topic TOPIC] [-p] [--iec] [--log-json]
            [--lock-wait SECONDS] [--bypass-lock] [--show-version] [--show-rc]
            [--umask M] [--remote-path PATH] [--remote-ratelimit RATE]
            [--upload-ratelimit RATE] [--remote-buffer UPLOAD_BUFFER]
            [--upload-buffer UPLOAD_BUFFER] [--consider-part-files]
            [--debug-profile FILE] [--rsh RSH]
            <command> ...
borg: error: unrecognized arguments: --match-archives ssh://host/./NAS
ssh://host/./NAS: Error running actions for repository
Command 'borg info --debug --show-rc --match-archives *a* ssh://host/./NAS' returned non-zero exit status 2.
/etc/borgmatic/config.yaml: Error running configuration file

As you can see, I'm using borg 1, but the newer --match-archives is being supplied.

Expected behavior (if a bug)

Normal listing.

Other notes / implementation ideas

When run in an identical PyCharm environment, the feature args get even weirder:

borg info --debug --show-rc --glob-archives {hostname}-nas-* --match-archives '*a*' ssh://host/./NAS

Environment

borgmatic version: 1.7.9 and 1.7.11

borgmatic installation method: Python venv

Borg version: 1.2.2

Python version: 3.9.2

Database version (if applicable): [version here]

operating system and version: Debian 11

#### What I'm trying to do and why Trying to run borgmatic info with the `-a` argument. #### Steps to reproduce (if a bug) Run `borgmatic -v2 -c /etc/borgmatic/config.yaml info -a '*a*'` #### Actual behavior (if a bug) ``` Ensuring legacy configuration is upgraded borg --version --debug --show-rc /etc/borgmatic/config.yaml: No commands to run for pre-actions hook ssh://host/./NAS: Displaying archive summary information borg info --debug --show-rc --match-archives *a* ssh://host/./NAS usage: borg [-V] [-h] [--critical] [--error] [--warning] [--info] [--debug] [--debug-topic TOPIC] [-p] [--iec] [--log-json] [--lock-wait SECONDS] [--bypass-lock] [--show-version] [--show-rc] [--umask M] [--remote-path PATH] [--remote-ratelimit RATE] [--upload-ratelimit RATE] [--remote-buffer UPLOAD_BUFFER] [--upload-buffer UPLOAD_BUFFER] [--consider-part-files] [--debug-profile FILE] [--rsh RSH] <command> ... borg: error: unrecognized arguments: --match-archives ssh://host/./NAS ssh://host/./NAS: Error running actions for repository Command 'borg info --debug --show-rc --match-archives *a* ssh://host/./NAS' returned non-zero exit status 2. /etc/borgmatic/config.yaml: Error running configuration file ``` As you can see, I'm using borg 1, but the newer --match-archives is being supplied. #### Expected behavior (if a bug) Normal listing. #### Other notes / implementation ideas When run in an identical PyCharm environment, the feature args get even weirder: `borg info --debug --show-rc --glob-archives {hostname}-nas-* --match-archives '*a*' ssh://host/./NAS` #### Environment **borgmatic version:** 1.7.9 and 1.7.11 **borgmatic installation method:** Python venv **Borg version:** 1.2.2 **Python version:** 3.9.2 **Database version (if applicable):** [version here] **operating system and version:** Debian 11
witten commented 2023-04-04 03:22:16 +00:00
Owner
Copy Link

Thank you so much for reporting this! I've confirmed the behavior with Borg 1.2. I'm not quite sure what's going on yet, because the code is supposed to check the local Borg version and use --match-archives or --glob-archives as appropriate. I'll dig into this and let you know what I find.

Thank you so much for reporting this! I've confirmed the behavior with Borg 1.2. I'm not quite sure what's going on yet, because the code is *supposed* to check the local Borg version and use `--match-archives` or `--glob-archives` as appropriate. I'll dig into this and let you know what I find.
witten added the
bug
 label 2023-04-04 03:22:22 +00:00
lingfish commented 2023-04-04 03:37:31 +00:00
Author
Copy Link

Yep, agreed... I did see the cool stuff being done in borg/feature.py etc, and even ran tests via tox -- even made another test as below and it strangely passed!

def test_version_checker():
    assert not module.available(module.Feature.MATCH_ARCHIVES, '1.2.2')
Yep, agreed... I did see the cool stuff being done in `borg/feature.py` etc, and even ran tests via `tox` -- even made another test as below and it strangely passed! ``` def test_version_checker(): assert not module.available(module.Feature.MATCH_ARCHIVES, '1.2.2') ```
diivi commented 2023-04-04 04:10:31 +00:00
Collaborator
Copy Link

I Was able to debug this:

The problem is with this line in info.py::display_archives_info() (line55):

flags.make_flags_from_arguments(
            info_arguments, excludes=('repository', 'archive', 'prefix')
        )

When I added match_prefix to the excludes argument for make_flags_from_arguments, the error did not occur anymore.

I Was able to debug this: The problem is with this line in `info.py::display_archives_info()` (line55): ```python flags.make_flags_from_arguments( info_arguments, excludes=('repository', 'archive', 'prefix') ) ``` When I added `match_prefix` to the `excludes` argument for `make_flags_from_arguments`, the error did not occur anymore.
witten commented 2023-04-04 04:42:36 +00:00
Owner
Copy Link

Aha! Nice catch. I'll make that fix and see if there's a way to catch this sort of issue in an integration test.

Aha! Nice catch. I'll make that fix and see if there's a way to catch this sort of issue in an integration test.
๐Ÿ‘ 1
lingfish commented 2023-04-04 04:51:51 +00:00
Author
Copy Link

I Was able to debug this:

The problem is with this line in info.py::display_archives_info() (line55):

flags.make_flags_from_arguments(
            info_arguments, excludes=('repository', 'archive', 'prefix')
        )

When I added match_prefix to the excludes argument for make_flags_from_arguments, the error did not occur anymore.

Not being fully immersed in the code, would this fix not need to be in other lines too, wherever --archive is accepted?

> I Was able to debug this: > > The problem is with this line in `info.py::display_archives_info()` (line55): > > ```python > flags.make_flags_from_arguments( > info_arguments, excludes=('repository', 'archive', 'prefix') > ) > ``` > > When I added `match_prefix` to the `excludes` argument for `make_flags_from_arguments`, the error did not occur anymore. Not being fully immersed in the code, would this fix not need to be in other lines too, wherever `--archive` is accepted?
witten commented 2023-04-04 05:41:50 +00:00
Owner
Copy Link

Yes, this is likely a bug in multiple different actions. And what's worse, it looks like even once fixed, the -a/--match-archives/--glob-archives flag on the command-line isn't correctly overriding any of the other places that value can come from. Fortunately the fix should be pretty straightforward.

Yes, this is likely a bug in multiple different actions. And what's worse, it looks like even once fixed, the `-a`/`--match-archives`/`--glob-archives` flag on the command-line isn't correctly overriding any of the [other places](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#archive-naming) that value can come from. Fortunately the fix should be pretty straightforward.
๐Ÿ‘ 2
witten commented 2023-04-05 04:29:31 +00:00
Owner
Copy Link

This has been fixed in master for all relevant actions. I also fixed the related --match-archives override problem mentioned above. Lastly, I added a few integration tests that lightly "fuzz" some of the the functions that produce Borg arguments to hopefully catch related issues in the future.

These fixes will be part of the next release. Thanks again for taking the time to file this. And thanks to @diivi for diagnosing!

This has been fixed in master for all relevant actions. I also fixed the related `--match-archives` override problem mentioned above. Lastly, I added a few integration tests that lightly "fuzz" some of the the functions that produce Borg arguments to hopefully catch related issues in the future. These fixes will be part of the next release. Thanks again for taking the time to file this. And thanks to @diivi for diagnosing!
๐Ÿš€ 1
lingfish commented 2023-04-05 04:41:19 +00:00
Author
Copy Link

You're awesome Dan, always a pleasure dealing with you :)

You're awesome Dan, always a pleasure dealing with you :)
๐Ÿ‘ 2
witten commented 2023-04-13 22:18:13 +00:00
Owner
Copy Link

This fix has been released as part of borgmatic 1.7.12!

This fix has been released as part of borgmatic 1.7.12!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#666
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?