borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 53 Pull Requests 1 Actions Packages Releases 121 Activity

borgmatic fails with borg prune when backing up to an rclone mounted drive #344

New Issue
Closed
opened 2020-07-28 23:12:36 +00:00 by anaxonda · 3 comments
anaxonda commented 2020-07-28 23:12:36 +00:00
Copy Link

What I'm trying to do and why

I'm trying to backup to a network drive mounted with rclone.

Steps to reproduce (if a bug)

Relevant config file:

location:
# List of source directories to backup (required). Globs and
source_directories:
- /home/test.borg

        repositories:
  • /mnt/test/test1.borg

retention:
# Keep all archives within this time interval.

Number of hourly archives to keep.
keep_hourly: 24

Number of daily archives to keep.
keep_daily: 7

Number of weekly archives to keep.
keep_weekly: 4

Number of monthly archives to keep.
keep_monthly: 6

I run the command:

borgmatic --config ~/.config/borgmatic/config.yaml --verbosity 2

Actual behavior (if a bug)

Ensuring legacy configuration is upgraded
/home/.config/borgmatic/config.yaml: No commands to run for pre-everything hook
/home/.config/borgmatic/config.yaml: No commands to run for pre-prune hook
/home/.config/borgmatic/config.yaml: No commands to run for pre-backup hook
/home/.config/borgmatic/config.yaml: No commands to run for pre-check hook
/mnt/test/test1.borg: Pruning archives
borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg
using builtin fallback logging configuration
35 self tests completed in 0.12 seconds
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 4565, in main
/mnt/test/test1.borg: Error running actions for repository
Command 'borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg' returned non-zero exit status 2.
/home/.config/borgmatic/config.yaml: No commands to run for on-error hook
/home/.config/borgmatic/config.yaml: Error running configuration file
/home/.config/borgmatic/config.yaml: No commands to run for post-everything hook

summary:
/home/.config/borgmatic/config.yaml: Error running configuration file
/mnt/test/test1.borg: Error running actions for repository
using builtin fallback logging configuration
35 self tests completed in 0.12 seconds
Local Exception
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 4565, in main
Command 'borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg' returned non-zero exit status 2.

Expected behavior (if a bug)

borgmatic to run successfully

Other notes / implementation ideas

I can run borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 successfully on the repository, but it seems like borgmatic is failing with it.

I assume this has something to do with rclone, because borgmatic runs successfuly for a locally mounted drive.

Environment

borgmatic version: [1.5.8]

borgmatic installation method: [Arch AUR]

Borg version: [1.1.13]

Python version: [3.8.3]

Database version (if applicable): [12.3]

operating system and version: [Arch Linux]

#### What I'm trying to do and why I'm trying to backup to a network drive mounted with rclone. #### Steps to reproduce (if a bug) Relevant config file: location: # List of source directories to backup (required). Globs and source_directories: - /home/test.borg repositories: - /mnt/test/test1.borg retention: # Keep all archives within this time interval. Number of hourly archives to keep. keep_hourly: 24 Number of daily archives to keep. keep_daily: 7 Number of weekly archives to keep. keep_weekly: 4 Number of monthly archives to keep. keep_monthly: 6 I run the command: borgmatic --config ~/.config/borgmatic/config.yaml --verbosity 2 #### Actual behavior (if a bug) Ensuring legacy configuration is upgraded /home/.config/borgmatic/config.yaml: No commands to run for pre-everything hook /home/.config/borgmatic/config.yaml: No commands to run for pre-prune hook /home/.config/borgmatic/config.yaml: No commands to run for pre-backup hook /home/.config/borgmatic/config.yaml: No commands to run for pre-check hook /mnt/test/test1.borg: Pruning archives borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg using builtin fallback logging configuration 35 self tests completed in 0.12 seconds Local Exception Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 4565, in main /mnt/test/test1.borg: Error running actions for repository Command 'borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg' returned non-zero exit status 2. /home/.config/borgmatic/config.yaml: No commands to run for on-error hook /home/.config/borgmatic/config.yaml: Error running configuration file /home/.config/borgmatic/config.yaml: No commands to run for post-everything hook summary: /home/.config/borgmatic/config.yaml: Error running configuration file /mnt/test/test1.borg: Error running actions for repository using builtin fallback logging configuration 35 self tests completed in 0.12 seconds Local Exception Traceback (most recent call last): File "/usr/lib/python3.8/site-packages/borg/archiver.py", line 4565, in main Command 'borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg' returned non-zero exit status 2. #### Expected behavior (if a bug) borgmatic to run successfully #### Other notes / implementation ideas I can run `borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6` successfully on the repository, but it seems like borgmatic is failing with it. I assume this has something to do with rclone, because borgmatic runs successfuly for a locally mounted drive. #### Environment **borgmatic version:** [1.5.8] **borgmatic installation method:** [Arch AUR] **Borg version:** [1.1.13] **Python version:** [3.8.3] **Database version (if applicable):** [12.3] **operating system and version:** [Arch Linux]
witten commented 2020-07-29 02:47:10 +00:00
Owner
Copy Link

Thank you for taking the time to file this and include all the detailed information. Based on the log from running borgmatic, the error is occurring within Borg itself:

Local Exception
Traceback (most recent call last):
File “/usr/lib/python3.8/site-packages/borg/archiver.py”, line 4565, in main

So it's super odd that the problem doesn't reproduce when you run Borg by itself. A couple of thoughts:

  • Is the Borg command you're running identical to what borgmatic's invoking? It's: borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg ... The addition of --debug --show-rc shouldn't make a difference, but good to check anyway.
  • Have you tried running borgmatic prune --config ~/.config/borgmatic/config.yaml --verbosity 2. (Note the addition of prune.) It's possible that one thing that's differen't between borgmatic and Borg is that borgmatic is also doing a create and a check. But if you just specify borgmatic prune, then it'll be more similiar to running Borg by itself.

Hopefully that'll help us figure out what's going on here!

Thank you for taking the time to file this and include all the detailed information. Based on the log from running borgmatic, the error is occurring within Borg itself: ```bash Local Exception Traceback (most recent call last): File “/usr/lib/python3.8/site-packages/borg/archiver.py”, line 4565, in main ``` So it's super odd that the problem doesn't reproduce when you run Borg by itself. A couple of thoughts: * Is the Borg command you're running identical to what borgmatic's invoking? It's: `borg prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --debug --show-rc /mnt/test/test1.borg` ... The addition of `--debug --show-rc` *shouldn't* make a difference, but good to check anyway. * Have you tried running `borgmatic prune --config ~/.config/borgmatic/config.yaml --verbosity 2`. (Note the addition of `prune`.) It's possible that one thing that's differen't between borgmatic and Borg is that borgmatic is also doing a `create` and a `check`. But if you just specify `borgmatic prune`, then it'll be more similiar to running Borg by itself. Hopefully that'll help us figure out what's going on here!
witten added the
bug
 label 2020-07-29 02:47:17 +00:00
anaxonda commented 2020-07-30 03:59:31 +00:00
Author
Copy Link

So, today I came back to it, and did as you suggested. I ran borgmatic prune --config ~/.config/borgmatic/config.yaml --verbosity 2, also borgmatic create and borgmatic check. All suceeded without error.

I went back to my original borgmatic command: borgmatic --config ~/.config/borgmatic/config.yaml --verbosity 2, and now that worked as well.

What is different now, I cannot tell you, but everything seems ok on my end now. Fingers crossed its smooth sailing from here.

Thanks for the help though, really appreciate it!

So, today I came back to it, and did as you suggested. I ran `borgmatic prune --config ~/.config/borgmatic/config.yaml --verbosity 2`, also `borgmatic create` and `borgmatic check`. All suceeded without error. I went back to my original borgmatic command: `borgmatic --config ~/.config/borgmatic/config.yaml --verbosity 2`, and now that worked as well. What is different now, I cannot tell you, but everything seems ok on my end now. Fingers crossed its smooth sailing from here. Thanks for the help though, really appreciate it!
witten commented 2020-07-30 04:18:04 +00:00
Owner
Copy Link

I'm glad to hear it's working now! Please let me know if the problem occurs again.

I'm glad to hear it's working now! Please let me know if the problem occurs again.
witten added
question / support
 and removed
bug
 labels 2020-07-30 04:18:16 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#344
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?