borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 57
Code Issues 71 Pull requests 3 Releases 162 Packages 1 Activity Actions 1

Archive not being created more than 2 times #646

New issue
Closed
opened 2023-03-06 21:42:24 +00:00 by admin.epicaman.in · 15 comments
admin.epicaman.in commented 2023-03-06 21:42:24 +00:00
Copy link

i runned brogmatic command 4 times and all 4 times i have changed my data but only i can see only 2 archives how can i rectify this .
i have run 4 times brogmatic soi should see 4 archives but it stuck at 2 only.

i runned brogmatic command 4 times and all 4 times i have changed my data but only i can see only 2 archives how can i rectify this . i have run 4 times brogmatic soi should see 4 archives but it stuck at 2 only.
witten commented 2023-03-06 21:43:53 +00:00
Owner
Copy link

Can I get a look at your configuration? For instance, what is your configured retention policy? Perhaps borgmatic is pruning the oldest archives as new ones are created, and that's why you're seeing fewer archives than you expect?

Can I get a look at your configuration? For instance, what is your configured retention policy? Perhaps borgmatic is pruning the oldest archives as new ones are created, and that's why you're seeing fewer archives than you expect?
admin.epicaman.in commented 2023-03-07 02:54:46 +00:00
Author
Copy link

yes sure

location:
    source_directories:
        - ~/Desktop
        - ~/Documents
        - ~/Pictures

    one_file_system: true

    repositories:
        - ssh://myrepohere

    exclude_caches: true

storage:
    compression: auto,zstd
    encryption_passphrase: mypassishere
    archive_name_format: '{hostname}-{now}'

retention:
    keep_daily: 3
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 2
    prefix: '{hostname}-'

consistency:
    checks:
        # uncomment to always do integrity checks. (takes long time for large repos)
        #- repository
        - disabled

    prefix: '{hostname}-'
yes sure ``` location: source_directories: - ~/Desktop - ~/Documents - ~/Pictures one_file_system: true repositories: - ssh://myrepohere exclude_caches: true storage: compression: auto,zstd encryption_passphrase: mypassishere archive_name_format: '{hostname}-{now}' retention: keep_daily: 3 keep_weekly: 4 keep_monthly: 12 keep_yearly: 2 prefix: '{hostname}-' consistency: checks: # uncomment to always do integrity checks. (takes long time for large repos) #- repository - disabled prefix: '{hostname}-' ```
witten commented 2023-03-07 03:13:58 +00:00
Owner
Copy link

Thanks for posting your configuration. I'm pretty sure that the keep_daily option is what's causing your archives to get pruned. From the Borg prune documentation:

As an example, --keep-daily 7 means to keep the latest backup on each day, up to 7 most recent days with backups (days without backups do not count).

Therefore your keep_daily: 3 means Borg retains three backups, one made each day. When you run borgmatic repeatedly, it runs borg prune, which prunes all backups except one because a day hasn't passed since the last backup.

So why are you seeing two archives? Because borgmatic currently prunes first—before creating a new backup. It prunes down to one backup and then immediately creates another one.

I hope that makes sense! This all can be a little tricky. There are several examples in the Borg docs that should hopefully provide inspiration if you'd like to tweak your prune settings.

Thanks for posting your configuration. I'm pretty sure that the `keep_daily` option is what's causing your archives to get pruned. From the [Borg prune documentation](https://borgbackup.readthedocs.io/en/stable/usage/prune.html): > As an example, --keep-daily 7 means to keep the latest backup on each day, up to 7 most recent days with backups (days without backups do not count). Therefore your `keep_daily: 3` means Borg retains three backups, *one* made each day. When you run borgmatic repeatedly, it runs `borg prune`, which prunes all backups except *one* because a day hasn't passed since the last backup. So why are you seeing two archives? Because borgmatic currently prunes *first*—before creating a new backup. It prunes down to one backup and then immediately creates another one. I hope that makes sense! This all can be a little tricky. There are several examples in the [Borg docs](https://borgbackup.readthedocs.io/en/stable/usage/prune.html#examples) that should hopefully provide inspiration if you'd like to tweak your prune settings.
admin.epicaman.in commented 2023-03-07 03:18:19 +00:00
Author
Copy link

So what value should i write so that it doesnt prune ?

location:
    source_directories:
        - /var/www
    one_file_system: true

    repositories:
        - ssh:///./repo

    exclude_caches: true

storage:
    compression: auto,zstd
    encryption_passphrase:
    archive_name_format: '{hostname}-{now}'

retention:
    keep_daily: 1000
    keep_weekly: 1000
    keep_monthly: 1000
    keep_yearly: 1000
    prefix: '{hostname}-'

consistency:
    checks:
        # uncomment to always do integrity checks. (takes long time for large repos)
        #- repository
        - disabled

    prefix: '{hostname}-'

i have set this but still i see 2 archives . pls help

So what value should i write so that it doesnt prune ? ``` location: source_directories: - /var/www one_file_system: true repositories: - ssh:///./repo exclude_caches: true storage: compression: auto,zstd encryption_passphrase: archive_name_format: '{hostname}-{now}' retention: keep_daily: 1000 keep_weekly: 1000 keep_monthly: 1000 keep_yearly: 1000 prefix: '{hostname}-' consistency: checks: # uncomment to always do integrity checks. (takes long time for large repos) #- repository - disabled prefix: '{hostname}-' ``` i have set this but still i see 2 archives . pls help
witten commented 2023-03-07 03:21:09 +00:00
Owner
Copy link

Currently, the way to do that is to run borgmatic on the command-line without an explicit or implicit prune action. So for instance: borgmatic create or borgmatic create check depending on which actions you want to run. See the documentation for more information.

But be advised that without any pruning (or compacting), your repository will grow without bound!

Currently, the way to do that is to run borgmatic on the command-line without an explicit or implicit `prune` action. So for instance: `borgmatic create` or `borgmatic create check` depending on which actions you want to run. See [the documentation](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups/#a-la-carte-actions) for more information. But be advised that without any pruning (or compacting), your repository will grow without bound!
admin.epicaman.in commented 2023-03-07 03:24:57 +00:00
Author
Copy link

Thnaks it worked . Thanks

May i know how to connect to sftp so that i can manage archives fro thier itself

Thnaks it worked . Thanks May i know how to connect to sftp so that i can manage archives fro thier itself
witten commented 2023-03-07 03:30:42 +00:00
Owner
Copy link

Borg doesn't support sftp directly as far as I know, but the details of connecting to your repository server via sftp would depend on your hosting provider. However you wouldn't see very much if you did that; the Borg repository isn't intended to be viewable without using Borg to view it.

If you just want to browse your archives though, you can use borgmatic rlist to list your archives or borgmatic list to list the contents of an archive or even borgmatic mount to get a file-level view into one of your archives.

Borg doesn't support sftp directly as far as I know, but the details of connecting to your repository server via sftp would depend on your hosting provider. However you wouldn't see very much if you did that; the Borg repository isn't intended to be viewable without using Borg to view it. If you just want to browse your archives though, you can use `borgmatic rlist` to list your archives or `borgmatic list` to list the contents of an archive or even `borgmatic mount` to get a file-level view into one of your archives.
admin.epicaman.in commented 2023-03-07 03:34:11 +00:00
Author
Copy link

So if i want to extract a file or folder from archive how this can be done specifically.

So if i want to extract a `file` or `folder` from archive how this can be done specifically.
witten commented 2023-03-07 03:46:26 +00:00
Owner
Copy link

Check out the documentation on that topic and let me know if anything's not clear!

Check out [the documentation](https://torsion.org/borgmatic/docs/how-to/extract-a-backup/) on that topic and let me know if anything's not clear!
admin.epicaman.in commented 2023-03-07 09:06:46 +00:00
Author
Copy link

yeh thanks
another question is it possible that like i have archive but i want that the test folder in that archive should replace my other test folder in the /home/ubuntu directory .
how can i achive this

yeh thanks another question is it possible that like i have archive but i want that the `test` folder in that archive should replace my other `test` folder in the /home/ubuntu directory . how can i achive this
witten commented 2023-03-07 18:06:40 +00:00
Owner
Copy link

Here's an example that will do that:

borgmatic extract --archive latest --path home/ubuntu --destination /home/ubuntu

Note however that it will overwrite /home/ubuntu on file-by-file basis. Meaning it won't delete any files already in that directory. If you want that behavior, I recommend manually deleting /home/ubuntu first before extracting.

Here's an example that will do that: ```bash borgmatic extract --archive latest --path home/ubuntu --destination /home/ubuntu ``` Note however that it will overwrite `/home/ubuntu` on file-by-file basis. Meaning it won't delete any files already in that directory. If you want that behavior, I recommend manually deleting `/home/ubuntu` first before extracting.
camdenorrb commented 2023-03-08 17:00:32 +00:00
Copy link

@witten Can you elaborate on this issue a bit, I'm also experiencing it but I have keep_daily set to 30

Shouldn't this keep up to 30 backups rather than 3?

If so, wouldn't this be a bug?

@witten Can you elaborate on this issue a bit, I'm also experiencing it but I have `keep_daily` set to `30` Shouldn't this keep up to 30 backups rather than 3? If so, wouldn't this be a bug?
witten commented 2023-03-08 17:35:41 +00:00
Owner
Copy link

Nope! keep_daily doesn't mean Borg will keep 30 archives per day. It means it'll keep 30 archives taken at a cadence of once per day. So if there are more archives than one per day, they'll get pruned down to the last archive taken each day. But if you're looking to keep archives more frequently than that, check out the keep_hourly or keep_within options. For more information, see Borg's prune documentation.

Nope! `keep_daily` doesn't mean Borg will keep 30 archives per day. It means it'll keep 30 archives taken at a cadence of *once per day*. So if there are *more* archives than one per day, they'll get pruned down to the last archive taken each day. But if you're looking to keep archives more frequently than that, check out the `keep_hourly` or `keep_within` options. For more information, see [Borg's prune documentation](https://borgbackup.readthedocs.io/en/stable/usage/prune.html).
👍 1
camdenorrb commented 2023-03-08 17:40:55 +00:00
Copy link

Oh interesting, thank you @witten

Oh interesting, thank you @witten
👍 1
witten commented 2023-03-08 22:20:44 +00:00
Owner
Copy link

I'm going to call this original questions addressed for now, but please feel free to file any follow-up questions as additional tickets. You may also be interested in #304, which changes the default ordering to prune second rather than first.

I'm going to call this original questions addressed for now, but please feel free to file any follow-up questions as additional tickets. You may also be interested in #304, which changes the default ordering to prune second rather than first.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
parallel-borgmatic
command-hook-steps
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#646
No description provided.