File count spot check error #981

New Issue

Closed

opened 2025-01-27 10:00:27 +00:00 by carl-westerlund · 5 comments

carl-westerlund commented 2025-01-27 10:00:27 +00:00

Copy Link

What I'm trying to do and why

I recently updated borgmatic to version 1.9.7, and then all my spot checks started failing. It seems that borgmatic doesn't find the files in the archive. However, if I extract the archive everything looks fine. Furthermore, if I use version 1.9.6 all the spot checks pass. So something seems to have happened between version 1.9.6 and 1.9.7.

Error message:

Spot check failed: 100.00% file count delta between source paths and latest archive (tolerance is 10%)

Steps to reproduce

Config file:

source_directories:
    - XXX

repositories:
    - path: XXX
      label: local

encryption_passcommand: "XXX"

archive_name_format: 'XXX'

keep_daily: 30

checks:
    - name: spot
      count_tolerance_percentage: 10
      data_sample_percentage: 1
      data_tolerance_percentage: 0

Actual behavior

I can recreate the issue by creating a new repo using borgmatic repo-create --encryption repokey-blake2 and then running borgmatic --verbosity 2.
I think the relevant section of the output is:

...
security: repository checks ok, allowing access
Archive authentication DISABLED.
TAM-verified archive
terminating with success status, rc 0
local: 0 total archive paths for spot check
local: Paths in source paths but not latest archive: ...

followed by a list of every file I'm trying to backup.

Then after that list it continues:

local: Paths in latest archive but not source paths: none
local: Error running actions for repository
Spot check failed: 100.00% file count delta between source paths and latest archive (tolerance is 10%)

Expected behavior

Since everything else seems to be working correctly and I can extract the archive, I would expect the file count spot check to pass.

Other notes / implementation ideas

No response

borgmatic version

1.9.7

borgmatic installation method

pipx

Borg version

1.4.0

Python version

3.13.1

Database version (if applicable)

No response

Operating system and version

macOS Sonoma 14.6.1

### What I'm trying to do and why I recently updated borgmatic to version 1.9.7, and then all my spot checks started failing. It seems that borgmatic doesn't find the files in the archive. However, if I extract the archive everything looks fine. Furthermore, if I use version 1.9.6 all the spot checks pass. So something seems to have happened between version 1.9.6 and 1.9.7. Error message: ``` Spot check failed: 100.00% file count delta between source paths and latest archive (tolerance is 10%) ``` ### Steps to reproduce Config file: ``` source_directories: - XXX repositories: - path: XXX label: local encryption_passcommand: "XXX" archive_name_format: 'XXX' keep_daily: 30 checks: - name: spot count_tolerance_percentage: 10 data_sample_percentage: 1 data_tolerance_percentage: 0 ``` ### Actual behavior I can recreate the issue by creating a new repo using `borgmatic repo-create --encryption repokey-blake2` and then running `borgmatic --verbosity 2`. I think the relevant section of the output is: ``` ... security: repository checks ok, allowing access Archive authentication DISABLED. TAM-verified archive terminating with success status, rc 0 local: 0 total archive paths for spot check local: Paths in source paths but not latest archive: ... ``` followed by a list of every file I'm trying to backup. Then after that list it continues: ``` local: Paths in latest archive but not source paths: none local: Error running actions for repository Spot check failed: 100.00% file count delta between source paths and latest archive (tolerance is 10%) ``` ### Expected behavior Since everything else seems to be working correctly and I can extract the archive, I would expect the file count spot check to pass. ### Other notes / implementation ideas _No response_ ### borgmatic version 1.9.7 ### borgmatic installation method pipx ### Borg version 1.4.0 ### Python version 3.13.1 ### Database version (if applicable) _No response_ ### Operating system and version macOS Sonoma 14.6.1

witten commented 2025-01-27 17:24:15 +00:00

Owner

Copy Link

Thanks for the detailed bug report! Please try upgrading to 1.9.8 and give your repro steps another shot. While 1.9.7 fixed several bugs, it also created others (fixed by #979). I believe 1.9.8 may solve the problem for you.

Thanks for the detailed bug report! Please try upgrading to 1.9.8 and give your repro steps another shot. While 1.9.7 fixed several bugs, it also created others (fixed by #979). I believe 1.9.8 may solve the problem for you.

witten added the question / support label 2025-01-27 17:24:23 +00:00

carl-westerlund commented 2025-01-27 17:58:01 +00:00

Author

Copy Link

Sorry, same issue with 1.9.8

It was actually when I upgraded from some older version (<=1.9.6) to 1.9.8 that encountered this issue, and then in the debugging process discovered that it was between 1.9.6 and 1.9.7 that it appeared.

Sorry, same issue with 1.9.8 It was actually when I upgraded from some older version (<=1.9.6) to 1.9.8 that encountered this issue, and then in the debugging process discovered that it was between 1.9.6 and 1.9.7 that it appeared.

witten commented 2025-01-27 18:12:50 +00:00

Owner

Copy Link

Okay, thanks for checking. Looks like I have a repro locally, so I'll dig into what exactly went wrong.

Okay, thanks for checking. Looks like I have a repro locally, so I'll dig into what exactly went wrong.

witten added bug and removed question / support labels 2025-01-27 18:13:02 +00:00

witten referenced this issue from a commit 2025-01-27 18:51:28 +00:00

Fix "spot" check file count delta error (#981).

witten commented 2025-01-27 18:51:37 +00:00

Owner

Copy Link

Okay, this was caused by the fix for #968, which internally changed archive listing captures to be null-separated instead of newline-separated. And it looks like the spot check wasn't updated accordingly. Now it is.

The fix is in main and will be part of the next release. Thanks again!

Okay, this was caused by the fix for #968, which internally changed archive listing captures to be null-separated instead of newline-separated. And it looks like the spot check wasn't updated accordingly. Now it is. The fix is in main and will be part of the next release. Thanks again!

witten closed this issue 2025-01-27 18:51:38 +00:00

witten referenced this issue 2025-01-27 18:52:42 +00:00

Spot check crashes #968

witten commented 2025-02-03 19:28:44 +00:00

Owner

Copy Link

Released in borgmatic 1.9.9!

Released in borgmatic 1.9.9!

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

command-hook-steps

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

No Label bug

Milestone

No items

No Milestone

Assignees

Clear assignees

witten

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#981