borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
10
66
Fork 27
Code Issues 42 Pull Requests 5 Actions Packages Releases 140 Activity

Borgmatic delete? #298

New Issue
Closed
opened 2020-03-09 09:49:23 +00:00 by jucor_ · 22 comments
jucor_ commented 2020-03-09 09:49:23 +00:00
Copy Link

What I'm trying to do and why

Delete specific archives within a repository

Other notes / implementation ideas

Would it be possible to have a delete command, similar to borg delete, to destroy specific archives within the repositories? I could do it with borg delete, but that would lose all the nice setup specified in borgmatic :)

Environment

borgmatic version: 1.5.1
borgmatic installation method: Pip
Borg version: 1.1.5
Python version: 3.6.9
operating system and version: Ubuntu 18.04.3 LTS

#### What I'm trying to do and why Delete specific archives within a repository #### Other notes / implementation ideas Would it be possible to have a `delete` command, similar to `borg delete`, to destroy specific archives within the repositories? I could do it with `borg delete`, but that would lose all the nice setup specified in borgmatic :) #### Environment **borgmatic version:** 1.5.1 **borgmatic installation method:** Pip **Borg version:** 1.1.5 **Python version:** 3.6.9 **operating system and version:** Ubuntu 18.04.3 LTS
👍 1
witten commented 2020-03-10 04:41:55 +00:00
Owner
Copy Link

Seems like a reasonable addition. Could you talk a little about the use case where borg delete works better for you than automatic pruning? That would help me understand the motivation a bit. Thanks for writing up the idea!

Seems like a reasonable addition. Could you talk a little about the use case where `borg delete` works better for you than automatic pruning? That would help me understand the motivation a bit. Thanks for writing up the idea!
jucor_ commented 2020-03-19 11:50:08 +00:00
Author
Copy Link

I needed to delete a range of backups that contained a set of files.
But I realise this is tricky, because each repository has different archive name with using {utcnow}.

So alternatively, and instead of having to reimplement every single borg command, how about a borgmatic command that just outputs a pre-filled borg command-line with the right setup (environment variables, repos, maybe repo passphrase in a securefile sourced with "PASSCOMMAND", etc). Then the user would just have to add the final borg commands they need.

Would that make more sense?

I needed to delete a range of backups that contained a set of files. But I realise this is tricky, because each repository has different archive name with using {utcnow}. So alternatively, and instead of having to reimplement every single borg command, how about a borgmatic command that just outputs a pre-filled borg command-line with the right setup (environment variables, repos, maybe repo passphrase in a securefile sourced with "PASSCOMMAND", etc). Then the user would just have to add the final borg commands they need. Would that make more sense?
witten commented 2020-04-29 20:49:44 +00:00
Owner
Copy Link

So alternatively, and instead of having to reimplement every single borg command, how about a borgmatic command that just outputs a pre-filled borg command-line with the right setup (environment variables, repos, maybe repo passphrase in a securefile sourced with “PASSCOMMAND”, etc). Then the user would just have to add the final borg commands they need.

You can kind of get that now by using --dry-run with -v 2. But yeah, it doesn't enclude all environment variables, etc.

I'm fine adding borgmatic delete, and indeed borgmatic is slowly growing to wrap more and more Borg actions. I just wanted to better understand your use case first. Makes sense to me.

> So alternatively, and instead of having to reimplement every single borg command, how about a borgmatic command that just outputs a pre-filled borg command-line with the right setup (environment variables, repos, maybe repo passphrase in a securefile sourced with “PASSCOMMAND”, etc). Then the user would just have to add the final borg commands they need. You can *kind of* get that now by using `--dry-run` with `-v 2`. But yeah, it doesn't enclude all environment variables, etc. I'm fine adding `borgmatic delete`, and indeed borgmatic is slowly growing to wrap more and more Borg actions. I just wanted to better understand your use case first. Makes sense to me.
lingfish commented 2020-09-19 00:38:19 +00:00
Contributor
Copy Link

To add my 2c here; I'd like this too. What I've noticed is that, I like to use a different borg_cache_directory than default in borgmatic... When using borg to do the delete, I always forget to point at that different cache, and I think the default ~/.cache/borg gets used, confusing things.

To add my 2c here; I'd like this too. What I've noticed is that, I like to use a different `borg_cache_directory` than default in borgmatic... When using `borg` to do the delete, I always forget to point at that different cache, and I think the default `~/.cache/borg` gets used, confusing things.
kaysond commented 2021-08-12 05:02:38 +00:00
Copy Link

Bump. I'd also love a delete feature. Honestly, it would be nice if you could just run any arbitrary borg command using the relevant config from borgmatic (e.g. ssh command, encryption keys, etc).

My particular need is that I want to delete a whole set of backups. I can prune down to 1, but then I manually have to delete that one anyways

Bump. I'd also love a delete feature. Honestly, it would be nice if you could just run any arbitrary borg command using the relevant config from borgmatic (e.g. ssh command, encryption keys, etc). My particular need is that I want to delete a whole set of backups. I can prune down to 1, but then I manually have to delete that one anyways
witten commented 2021-08-12 05:04:19 +00:00
Owner
Copy Link

Have you tried borgmatic borg delete? 😄

Have you tried [`borgmatic borg delete`](https://torsion.org/borgmatic/docs/how-to/run-arbitrary-borg-commands/)? 😄
👍 1
kaysond commented 2021-08-12 05:30:16 +00:00
Copy Link

Oooh! Awesome! I thought there might be something like that, and I checked borgmatic --help but didn't see anything because I was on an old version. Just updated

Oooh! Awesome! I thought there might be something like that, and I checked `borgmatic --help` but didn't see anything because I was on an old version. Just updated
witten commented 2021-08-12 06:07:32 +00:00
Owner
Copy Link

Cool! I agree that a native borgmatic delete would be helpful too.

Cool! I agree that a native `borgmatic delete` would be helpful too.
elhananjair commented 2022-01-26 04:54:33 +00:00
Copy Link

Have you tried borgmatic borg delete? 😄

I tried borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171 Tue, 2022-01-25 17:20:35

and it did delete the archive but there is an errr/warning showing up.

Archive Tue, not found (2/4).
Archive 2022-01-25 not found (3/4).
Archive 17:20:35 not found (4/4).

Is that ok? @witten

> Have you tried [`borgmatic borg delete`](https://torsion.org/borgmatic/docs/how-to/run-arbitrary-borg-commands/)? 😄 I tried `borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171 Tue, 2022-01-25 17:20:35` and it did delete the archive but there is an errr/warning showing up. > Archive Tue, not found (2/4). Archive 2022-01-25 not found (3/4). Archive 17:20:35 not found (4/4). Is that ok? @witten
witten commented 2022-01-26 05:40:45 +00:00
Owner
Copy Link

The archive name looks like it's just MY-hostname-2022-01-25T17:20:35.335171. The rest of it (Tue, ...) is not part of the archive name—that's just additional information about the archive that Borg provides. So the full command would look more like:

borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171
The archive name looks like it's just `MY-hostname-2022-01-25T17:20:35.335171`. The rest of it (`Tue, ...`) is not part of the archive name—that's just additional information about the archive that Borg provides. So the full command would look more like: ```bash borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171 ```
👍 1
elhananjair commented 2022-01-26 05:43:16 +00:00
Copy Link

The archive name looks like it's just MY-hostname-2022-01-25T17:20:35.335171. The rest of it (Tue, ...) is not part of the archive name—that's just additional information about the archive that Borg provides. So the full command would look more like:

borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171

My bad, thanks as always!

> The archive name looks like it's just `MY-hostname-2022-01-25T17:20:35.335171`. The rest of it (`Tue, ...`) is not part of the archive name—that's just additional information about the archive that Borg provides. So the full command would look more like: > > ```bash > borgmatic borg delete MY-hostname-2022-01-25T17:20:35.335171 > ``` My bad, thanks as always!
👍 1
witten added the
good first issue
 label 2023-02-04 22:06:40 +00:00
Soumik_Dutta commented 2023-03-21 02:22:33 +00:00
Contributor
Copy Link

I want to work on this issue if this is available 😄

I want to work on this issue if this is available 😄
witten commented 2023-03-21 02:39:57 +00:00
Owner
Copy Link

It's still available! Thanks for checking, and let me know if I can help in any way.

It's still available! Thanks for checking, and let me know if I can help in any way.
👍 1
Soumik_Dutta commented 2023-03-22 21:40:27 +00:00
Contributor
Copy Link

I was looking into the documentation for Borg command and I have a question in general that between Borg 1 and Borg 2, a lot of commands have changed. For example between these versions,

  • latest borg delete
  • 1.2.3 borg delete
    the latest version of borg delete has more archive filters.
    We are targeting full coverage for Borg 1 commands ... but at some point we will have to support Borg 2 commands.

So, I was wondering that is there any approach that we follow right now so that we might support Borg 2 commands with minimal change?

I was looking into the documentation for Borg command and I have a question in general that between Borg 1 and Borg 2, a lot of commands have changed. For example between these versions, - [latest `borg delete`](https://borgbackup.readthedocs.io/en/latest/usage/delete.html) - [1.2.3 `borg delete`](https://borgbackup.readthedocs.io/en/1.2.3/usage/delete.html) the latest version of `borg delete` has more archive filters. We are targeting full coverage for Borg 1 commands ... but at some point we will have to support Borg 2 commands. So, I was wondering that is there any approach that we follow right now so that we might support Borg 2 commands with minimal change?
Soumik_Dutta commented 2023-03-22 22:33:54 +00:00
Contributor
Copy Link

Also, if both --repository and --archive are not passed, should delete all repositories?

Also, if both `--repository` and `--archive` are not passed, should delete all repositories?
witten commented 2023-03-22 23:03:54 +00:00
Owner
Copy Link

So, I was wondering that is there any approach that we follow right now so that we might support Borg 2 commands with minimal change?

The approach I've been taking to that has varied based on the specific Borg flags. Some examples relevant to this ticket:

  • borgmatic list accepts both --match-archives and --glob-archives, and in fact they are a single Python argument with an alias. See list_group in borgmatic/commands/argument.py:make_parsers() for where that happens.
  • For borgmatic check, the flags passed to Borg vary based on the version of Borg found. See borgmatic/borg/check.py:make_check_flags(), specifically the call to feature.available().
  • For certain Borg sub-commands or flags that are only present in Borg 2, I'll simply add them to borgmatic and note that they're only available in Borg 2+. An example borgmatic action like this is borgmatic transfer, and an example flag like this is borgmatic init/rcreate --source-repository.

For the new Borg 2 archive filters you're referring to (--oldest, --newest, etc.), I'd be fine with leaving those unimplemented for now and filing them in another ticket. Nothing else in borgmatic implements them since they're so new.

both --repository and --archive are not passed, it should delete all repositories,

Is that how plain borg delete works? I couldn't tell from the documentation. It might be safer not to do this unless an explicit archive filter is provided that matches all archives.

only --archive is passed, it should delete that archive from all configured repositories,

That makes sense, although in practice there's probably not going to be an archive with the same name in multiple repositories. It might be safer and more sensible just to require --repository on the delete action if multiple repositories are configured. (There are existing borgmatic actions that do this.)

only --repository is passed, should it delete that specific repository or only the latest archive from that repository?

I'd say it should error. After all, Borg 2 has a separate rdelete sub-command for this purpose. I'm personally not fussed if we don't support doing this in Borg 1.x.

Hope this helps!

> So, I was wondering that is there any approach that we follow right now so that we might support Borg 2 commands with minimal change? The approach I've been taking to that has varied based on the specific Borg flags. Some examples relevant to this ticket: * `borgmatic list` accepts both `--match-archives` and `--glob-archives`, and in fact they are a single Python argument with an alias. See `list_group` in `borgmatic/commands/argument.py:make_parsers()` for where that happens. * For `borgmatic check`, the flags passed to Borg vary based on the version of Borg found. See `borgmatic/borg/check.py:make_check_flags()`, specifically the call to `feature.available()`. * For certain Borg sub-commands or flags that are only present in Borg 2, I'll simply add them to borgmatic and note that they're only available in Borg 2+. An example borgmatic action like this is `borgmatic transfer`, and an example flag like this is `borgmatic init/rcreate --source-repository`. For the new Borg 2 archive filters you're referring to (`--oldest`, `--newest`, etc.), I'd be fine with leaving those unimplemented for now and filing them in another ticket. Nothing else in borgmatic implements them since they're so new. > both --repository and --archive are not passed, it should delete all repositories, Is that how plain `borg delete` works? I couldn't tell from the documentation. It might be safer *not* to do this unless an explicit archive filter is provided that matches all archives. > only --archive is passed, it should delete that archive from all configured repositories, That makes sense, although in practice there's probably not going to be an archive with the same name in multiple repositories. It might be safer and more sensible just to require `--repository` on the `delete` action if multiple repositories are configured. (There are existing borgmatic actions that do this.) > only --repository is passed, should it delete that specific repository or only the latest archive from that repository? I'd say it should error. After all, Borg 2 has a separate `rdelete` sub-command for this purpose. I'm personally not fussed if we don't support doing this in Borg 1.x. Hope this helps!
Soumik_Dutta commented 2023-03-22 23:10:09 +00:00
Contributor
Copy Link

In the arguments, is it necessary to mention default=False if action='store_true' is passed? From the docs it seems unnecessary to mention it.

In the `arguments`, is it necessary to mention `default=False` if `action='store_true'` is passed? From the [docs](https://docs.python.org/3/library/argparse.html#action) it seems unnecessary to mention it.
witten commented 2023-03-23 04:02:51 +00:00
Owner
Copy Link

I would trust the docs on this one. 😄

I would trust the docs on this one. 😄
witten added the
new feature area
 label 2023-06-28 18:52:59 +00:00
witten commented 2024-06-25 22:05:32 +00:00
Owner
Copy Link

@Soumik_Dutta, are you still planning on working on this? No worries either way.. I just want to know if this ticket is available to work on. Thanks!

@Soumik_Dutta, are you still planning on working on this? No worries either way.. I just want to know if this ticket is available to work on. Thanks!
Soumik_Dutta commented 2024-06-28 09:06:26 +00:00
Contributor
Copy Link

No, I am not working on this code anymore. Please feel free to assign this to someone else @witten

No, I am not working on this code anymore. Please feel free to assign this to someone else @witten
👍 1
witten removed the
good first issue
 label 2024-06-29 01:51:56 +00:00
witten commented 2024-07-04 06:17:48 +00:00
Owner
Copy Link

Implemented in main and will be in the next release!

Implemented in main and will be in the next release!
witten referenced this issue from a commit 2024-07-04 15:42:27 +00:00
Add additional tests (#298).
witten commented 2024-07-04 15:57:33 +00:00
Owner
Copy Link

Released in borgmatic 1.8.13!

Released in borgmatic 1.8.13!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
new feature area
Milestone
No items
No Milestone
Assignees
Clear assignees
diivi import_bot witten
No Assignees
6 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#298
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?