Archive not being created more than 2 times #646

New Issue

admin.epicaman.in · 2023-03-06T21:42:24Z

admin.epicaman.in commented

2023-03-06 21:42:24 +00:00

i runned brogmatic command 4 times and all 4 times i have changed my data but only i can see only 2 archives how can i rectify this .
i have run 4 times brogmatic soi should see 4 archives but it stuck at 2 only.

i runned brogmatic command 4 times and all 4 times i have changed my data but only i can see only 2 archives how can i rectify this . i have run 4 times brogmatic soi should see 4 archives but it stuck at 2 only.

witten commented

2023-03-06 21:43:53 +00:00

Can I get a look at your configuration? For instance, what is your configured retention policy? Perhaps borgmatic is pruning the oldest archives as new ones are created, and that's why you're seeing fewer archives than you expect?

witten added the

question / support

label 2023-03-06 21:44:01 +00:00

admin.epicaman.in commented

2023-03-07 02:54:46 +00:00

yes sure

location:
    source_directories:
        - ~/Desktop
        - ~/Documents
        - ~/Pictures

    one_file_system: true

    repositories:
        - ssh://myrepohere

    exclude_caches: true

storage:
    compression: auto,zstd
    encryption_passphrase: mypassishere
    archive_name_format: '{hostname}-{now}'

retention:
    keep_daily: 3
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 2
    prefix: '{hostname}-'

consistency:
    checks:
        # uncomment to always do integrity checks. (takes long time for large repos)
        #- repository
        - disabled

    prefix: '{hostname}-'

yes sure ``` location: source_directories: - ~/Desktop - ~/Documents - ~/Pictures one_file_system: true repositories: - ssh://myrepohere exclude_caches: true storage: compression: auto,zstd encryption_passphrase: mypassishere archive_name_format: '{hostname}-{now}' retention: keep_daily: 3 keep_weekly: 4 keep_monthly: 12 keep_yearly: 2 prefix: '{hostname}-' consistency: checks: # uncomment to always do integrity checks. (takes long time for large repos) #- repository - disabled prefix: '{hostname}-' ```

witten commented

2023-03-07 03:13:58 +00:00

Thanks for posting your configuration. I'm pretty sure that the keep_daily option is what's causing your archives to get pruned. From the Borg prune documentation:

As an example, --keep-daily 7 means to keep the latest backup on each day, up to 7 most recent days with backups (days without backups do not count).

Therefore your keep_daily: 3 means Borg retains three backups, one made each day. When you run borgmatic repeatedly, it runs borg prune, which prunes all backups except one because a day hasn't passed since the last backup.

So why are you seeing two archives? Because borgmatic currently prunes first—before creating a new backup. It prunes down to one backup and then immediately creates another one.

I hope that makes sense! This all can be a little tricky. There are several examples in the Borg docs that should hopefully provide inspiration if you'd like to tweak your prune settings.

Thanks for posting your configuration. I'm pretty sure that the `keep_daily` option is what's causing your archives to get pruned. From the [Borg prune documentation](https://borgbackup.readthedocs.io/en/stable/usage/prune.html): > As an example, --keep-daily 7 means to keep the latest backup on each day, up to 7 most recent days with backups (days without backups do not count). Therefore your `keep_daily: 3` means Borg retains three backups, *one* made each day. When you run borgmatic repeatedly, it runs `borg prune`, which prunes all backups except *one* because a day hasn't passed since the last backup. So why are you seeing two archives? Because borgmatic currently prunes *first*—before creating a new backup. It prunes down to one backup and then immediately creates another one. I hope that makes sense! This all can be a little tricky. There are several examples in the [Borg docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html#examples) that should hopefully provide inspiration if you'd like to tweak your prune settings.

admin.epicaman.in commented

2023-03-07 03:18:19 +00:00

So what value should i write so that it doesnt prune ?

location:
    source_directories:
        - /var/www
    one_file_system: true

    repositories:
        - ssh:///./repo

    exclude_caches: true

storage:
    compression: auto,zstd
    encryption_passphrase:
    archive_name_format: '{hostname}-{now}'

retention:
    keep_daily: 1000
    keep_weekly: 1000
    keep_monthly: 1000
    keep_yearly: 1000
    prefix: '{hostname}-'

consistency:
    checks:
        # uncomment to always do integrity checks. (takes long time for large repos)
        #- repository
        - disabled

    prefix: '{hostname}-'

i have set this but still i see 2 archives . pls help

So what value should i write so that it doesnt prune ? ``` location: source_directories: - /var/www one_file_system: true repositories: - ssh:///./repo exclude_caches: true storage: compression: auto,zstd encryption_passphrase: archive_name_format: '{hostname}-{now}' retention: keep_daily: 1000 keep_weekly: 1000 keep_monthly: 1000 keep_yearly: 1000 prefix: '{hostname}-' consistency: checks: # uncomment to always do integrity checks. (takes long time for large repos) #- repository - disabled prefix: '{hostname}-' ``` i have set this but still i see 2 archives . pls help

witten commented

2023-03-07 03:21:09 +00:00

Currently, the way to do that is to run borgmatic on the command-line without an explicit or implicit prune action. So for instance: borgmatic create or borgmatic create check depending on which actions you want to run. See the documentation for more information.

But be advised that without any pruning (or compacting), your repository will grow without bound!

Currently, the way to do that is to run borgmatic on the command-line without an explicit or implicit `prune` action. So for instance: `borgmatic create` or `borgmatic create check` depending on which actions you want to run. See [the documentation](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups/#a-la-carte-actions) for more information. But be advised that without any pruning (or compacting), your repository will grow without bound!

admin.epicaman.in commented

2023-03-07 03:24:57 +00:00

Thnaks it worked . Thanks

May i know how to connect to sftp so that i can manage archives fro thier itself

Thnaks it worked . Thanks May i know how to connect to sftp so that i can manage archives fro thier itself

witten commented

2023-03-07 03:30:42 +00:00

Borg doesn't support sftp directly as far as I know, but the details of connecting to your repository server via sftp would depend on your hosting provider. However you wouldn't see very much if you did that; the Borg repository isn't intended to be viewable without using Borg to view it.

If you just want to browse your archives though, you can use borgmatic rlist to list your archives or borgmatic list to list the contents of an archive or even borgmatic mount to get a file-level view into one of your archives.

Borg doesn't support sftp directly as far as I know, but the details of connecting to your repository server via sftp would depend on your hosting provider. However you wouldn't see very much if you did that; the Borg repository isn't intended to be viewable without using Borg to view it. If you just want to browse your archives though, you can use `borgmatic rlist` to list your archives or `borgmatic list` to list the contents of an archive or even `borgmatic mount` to get a file-level view into one of your archives.

admin.epicaman.in commented

2023-03-07 03:34:11 +00:00

So if i want to extract a file or folder from archive how this can be done specifically.

So if i want to extract a `file` or `folder` from archive how this can be done specifically.

witten commented

2023-03-07 03:46:26 +00:00

Check out the documentation on that topic and let me know if anything's not clear!

Check out [the documentation](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) on that topic and let me know if anything's not clear!

admin.epicaman.in commented

2023-03-07 09:06:46 +00:00

yeh thanks
another question is it possible that like i have archive but i want that the test folder in that archive should replace my other test folder in the /home/ubuntu directory .
how can i achive this

yeh thanks another question is it possible that like i have archive but i want that the `test` folder in that archive should replace my other `test` folder in the /home/ubuntu directory . how can i achive this

witten commented

2023-03-07 18:06:40 +00:00

Here's an example that will do that:

borgmatic extract --archive latest --path home/ubuntu --destination /home/ubuntu

Note however that it will overwrite /home/ubuntu on file-by-file basis. Meaning it won't delete any files already in that directory. If you want that behavior, I recommend manually deleting /home/ubuntu first before extracting.

Here's an example that will do that: ```bash borgmatic extract --archive latest --path home/ubuntu --destination /home/ubuntu ``` Note however that it will overwrite `/home/ubuntu` on file-by-file basis. Meaning it won't delete any files already in that directory. If you want that behavior, I recommend manually deleting `/home/ubuntu` first before extracting.

camdenorrb commented

2023-03-08 17:00:32 +00:00

@witten Can you elaborate on this issue a bit, I'm also experiencing it but I have keep_daily set to 30

Shouldn't this keep up to 30 backups rather than 3?

If so, wouldn't this be a bug?

@witten Can you elaborate on this issue a bit, I'm also experiencing it but I have `keep_daily` set to `30` Shouldn't this keep up to 30 backups rather than 3? If so, wouldn't this be a bug?

witten commented

2023-03-08 17:35:41 +00:00

Nope! keep_daily doesn't mean Borg will keep 30 archives per day. It means it'll keep 30 archives taken at a cadence of once per day. So if there are more archives than one per day, they'll get pruned down to the last archive taken each day. But if you're looking to keep archives more frequently than that, check out the keep_hourly or keep_within options. For more information, see Borg's prune documentation.

Nope! `keep_daily` doesn't mean Borg will keep 30 archives per day. It means it'll keep 30 archives taken at a cadence of *once per day*. So if there are *more* archives than one per day, they'll get pruned down to the last archive taken each day. But if you're looking to keep archives more frequently than that, check out the `keep_hourly` or `keep_within` options. For more information, see [Borg's prune documentation](https://borgbackup.readthedocs.io/en/stable/usage/prune.html).

👍 1

camdenorrb commented

2023-03-08 17:40:55 +00:00

Oh interesting, thank you @witten

👍 1

witten commented

2023-03-08 22:20:44 +00:00

I'm going to call this original questions addressed for now, but please feel free to file any follow-up questions as additional tickets. You may also be interested in #304, which changes the default ordering to prune second rather than first.

witten closed this issue

2023-03-08 22:20:44 +00:00

Sign in to join this conversation.