Feature Request: support for --strip-components all (or similar)

sebastianhaberey commented

2023-03-09 06:33:34 +00:00

Hey guys,

first off thanks for borgmatic, it's awesome! After trying it out I've decided to replace all three of my current backup solutions (Duplicati, Time Machine, self-written shell script) with the borg / borgmatic combination. I really love it - it's so lean and simple.

I have one use case that's a bit clunky though. Usually I'll search the archives for that one file, then go through its versions to see if any of them contains what I'm looking for.

When I run borgmatic extract, the file will be downloaded to my current working directory, but with all its parent directories. I can fix this by counting the parent directories and using --strip-components 4 for example, but it would be much nicer to have something like --strip-components all or --remove-parent-directories.

borgmatic version: 1.7.8

borgmatic installation method: brew

Borg version: 1.2.3

operating system and version: macOS Ventura 13.2.1

Hey guys, first off thanks for borgmatic, it's awesome! After trying it out I've decided to replace all three of my current backup solutions (Duplicati, Time Machine, self-written shell script) with the borg / borgmatic combination. I really love it - it's so lean and simple. I have one use case that's a bit clunky though. Usually I'll search the archives for that _one_ file, then go through its versions to see if any of them contains what I'm looking for. When I run `borgmatic extract`, the file will be downloaded to my current working directory, but with all its parent directories. I can fix this by counting the parent directories and using `--strip-components 4` for example, but it would be much nicer to have something like `--strip-components all` or `--remove-parent-directories`. **borgmatic version:** 1.7.8 **borgmatic installation method:** brew **Borg version:** 1.2.3 **operating system and version:** macOS Ventura 13.2.1

witten commented

2023-03-09 06:48:05 +00:00

Owner

Thanks for using borgmatic and taking the time to write up this feature! It sounds like a great idea to me, especially since borgmatic already has a --destination flag. But given that such a feature isn't (currently) in Borg and would have to be done at the borgmatic level, the only issue I can see is that there could potentially be collisions during extraction. For instance, if you're extracting multiple files like foo/file.txt and bar/file.txt and you use --remove-parent-directories or whatever, then what should borgmatic do? Error? Maybe this isn't something for you to worry about now, but just an edge case for the implementer to deal with.

When you do use borgmatic extract today, do you use the --path flag with the path for a single file? And then do you mv it to wherever you want it to go? Or did you cd to its desired parent directory first (or use --destination)?

In any case, this removal of parent directories would have to be a post-processing step done by borgmatic after Borg performs its usual extraction into nested directories.

Usually I'll search the archives for that one file ...

I assume you're aware of the --find feature, which makes this process a little more streamlined. However it won't show you the contents of the file.

Thanks for using borgmatic and taking the time to write up this feature! It sounds like a great idea to me, especially since borgmatic already has a `--destination` flag. But given that such a feature isn't (currently) in Borg and would have to be done at the borgmatic level, the only issue I can see is that there could potentially be collisions during extraction. For instance, if you're extracting multiple files like `foo/file.txt` and `bar/file.txt` and you use `--remove-parent-directories` or whatever, then what should borgmatic do? Error? Maybe this isn't something for you to worry about now, but just an edge case for the implementer to deal with. When you do use `borgmatic extract` today, do you use the `--path` flag with the path for a single file? And then do you `mv` it to wherever you want it to go? Or did you `cd` to its desired parent directory first (or use `--destination`)? In any case, this removal of parent directories would have to be a post-processing step done by borgmatic after Borg performs its usual extraction into nested directories. > Usually I'll search the archives for that one file ... I assume you're aware of the [`--find` feature](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/#searching-for-a-file), which makes this process a little more streamlined. However it won't show you the contents of the file.

sebastianhaberey commented

2023-03-09 08:13:02 +00:00

Author

You are right, of course.

I was using the --find feature which is great by the way, and one of the reasons I switched from Time Machine. I love that I can see all the versions of the file, including modification time and file size. That helps a lot when looking for a specific change.

Then I would do cd to my download directory (~/Desktop/test for example) and do a borgmatic extract with a single argument to the --path flag. For that scenario, a --remove-parent-directories would be convenient. But I agree this feature isn't so straightforward once there are multiple arguments to --path. One could

output an error: --remove-parent-directories allows only one argument to --path
keep first file, skip subsequent files of same name
overwrite existing file of same name
ask user
rename subsequent files like macOS does, like "Test (1).txt"

All of these options feel a bit icky though, so unless you have a better (cleaner) idea, I suggest we put this idea on the "nice to have" stack 🙂

You are right, of course. I was using the `--find` feature which is great by the way, and one of the reasons I switched from Time Machine. I love that I can see all the versions of the file, including modification time and file size. That helps a lot when looking for a specific change. Then I would do cd to my download directory (`~/Desktop/test` for example) and do a `borgmatic extract` with a single argument to the `--path` flag. For that scenario, a `--remove-parent-directories` would be convenient. But I agree this feature isn't so straightforward once there are multiple arguments to `--path`. One could * output an error: `--remove-parent-directories` allows only one argument to `--path` * keep first file, skip subsequent files of same name * overwrite existing file of same name * ask user * rename subsequent files like macOS does, like "Test (1).txt" All of these options feel a bit icky though, so unless you have a better (cleaner) idea, I suggest we put this idea on the "nice to have" stack 🙂

witten commented

2023-03-09 17:29:23 +00:00

Owner

Any of those options make sense to me. Thanks for the details and the brainstorming. Honestly I'd probably look at what "native" borg extract does and try to mimic its overwriting behavior.

Any of those options make sense to me. Thanks for the details and the brainstorming. Honestly I'd probably look at what "native" `borg extract` does and try to mimic its overwriting behavior.

witten commented

2023-03-09 17:32:23 +00:00

Owner

Borg's "native" extract behavior: Silently overwrite. 😄

Borg's "native" `extract` behavior: Silently overwrite. 😄

sebastianhaberey commented

2023-03-09 17:49:25 +00:00

Author

I mean that would be just fine for my use case. And if Borg already does it, it would be consistent for the user, too.

witten referenced this issue from a commit

2023-03-09 18:09:51 +00:00

Add "--strip-components all" on the "extract" action to remove leading path components (#647).

witten commented

2023-03-09 18:13:50 +00:00

Owner

I ended up going with your original --strip-components all idea, in large part because it lets me lean on Borg to do the heavy lifting. What borgmatic does is simply count the leading path components in any specified --paths, take the maximum, and pass that number to Borg's --strip-components. Seems to work!

$ borgmatic -c test.yaml -v 2 extract --archive flux-2023-01-25T23:18:52.207930 \
  --strip-components all --path root/tmp/foo.yaml
...
$ ls foo.yaml 
foo.yaml

The main downside is that this won't work when no --paths are specified. In that case, borgmatic just errors.

Thanks again for suggesting this. It'll be part of the next release!

I ended up going with your original `--strip-components all` idea, in large part because it lets me lean on Borg to do the heavy lifting. What borgmatic does is simply count the leading path components in any specified `--path`s, take the maximum, and pass that number to Borg's `--strip-components`. Seems to work! ``` $ borgmatic -c test.yaml -v 2 extract --archive flux-2023-01-25T23:18:52.207930 \ --strip-components all --path root/tmp/foo.yaml ... $ ls foo.yaml foo.yaml ``` The main downside is that this won't work when no `--path`s are specified. In that case, borgmatic just errors. Thanks again for suggesting this. It'll be part of the next release!

witten closed this issue

2023-03-09 18:13:50 +00:00

sebastianhaberey commented

2023-03-09 18:21:50 +00:00

Author

Wow nice, thanks so much 😊 looking forward!