Q: Spot check schedule with config.yml & requesting clarification of spot check parameters #868

New Issue

Open

opened 2024-05-15 17:29:47 +00:00 by michaeltoohig · 0 comments

michaeltoohig commented 2024-05-15 17:29:47 +00:00

Copy Link

What I'm trying to do and why

I want to confirm spot checks are working as intended. I am defining how to run spot checks inside config.yml. Once a month it should spot check an archive. Given issue #866 I am seeing an error occur due to xxh64sum which sends me a fail alert; however, new archives are still being created on my remote repo. Is this the expected behavior?

I already restarted the container and lost the current log information but I recall seeing the output of --stats in the logs before the xxh6sum error which added to the fact I have new archives in my repo means an archive is created then checks are run which is counter to the documentation which says to run spot check on a separate schedule to create. Is this expected behavior too or should I manually add spot check to my crontab.txt instead?

Also for clarification here is an example spot check.

checks:
  - name: spot
    frequency: 1 week
    count_tolerance_percentage: 10
    data_sample_percentage: 10
    data_tolerance_percentage: 0.5

My second question is to clarify the meaning of the parameters as I find it a little unclear to me. If I have 1000 files, 100 files will be spot checked and from that 100 if 5 files are found to be different the check will fail? Is that the correct way to read this? In addition if there is more than a 10% change in the total number of files the check will fail given count_tolerance_percentage?

Steps to reproduce

No response

Actual behavior

No response

Expected behavior

No response

Other notes / implementation ideas

No response

borgmatic version

1.8.11

borgmatic installation method

container

Borg version

1.2.8

Python version

3.12.3

Database version (if applicable)

mysql Ver 15.1 Distrib 10.11.6-MariaDB, for Linux (x86_64) using readline 5.1

Operating system and version

Alpine 3.19

### What I'm trying to do and why I want to confirm `spot` checks are working as intended. I am defining how to run `spot` checks inside `config.yml`. Once a month it should spot check an archive. Given issue #866 I am seeing an error occur due to `xxh64sum` which sends me a `fail` alert; however, new archives are still being created on my remote repo. Is this the expected behavior? I already restarted the container and lost the current log information but I recall seeing the output of `--stats` in the logs before the `xxh6sum` error which added to the fact I have new archives in my repo means an archive is created then checks are run which is counter to the documentation which says to run `spot` check on a separate schedule to `create`. Is this expected behavior too or should I manually add `spot` check to my `crontab.txt` instead? Also for clarification here is an example spot check. ``` checks: - name: spot frequency: 1 week count_tolerance_percentage: 10 data_sample_percentage: 10 data_tolerance_percentage: 0.5 ``` My second question is to clarify the meaning of the parameters as I find it a little unclear to me. If I have 1000 files, 100 files will be spot checked and from that 100 if 5 files are found to be different the check will fail? Is that the correct way to read this? In addition if there is more than a 10% change in the total number of files the check will fail given `count_tolerance_percentage`? ### Steps to reproduce _No response_ ### Actual behavior _No response_ ### Expected behavior _No response_ ### Other notes / implementation ideas _No response_ ### borgmatic version 1.8.11 ### borgmatic installation method container ### Borg version 1.2.8 ### Python version 3.12.3 ### Database version (if applicable) mysql Ver 15.1 Distrib 10.11.6-MariaDB, for Linux (x86_64) using readline 5.1 ### Operating system and version Alpine 3.19

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

1 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#868

No description provided.