View quota usage/available #967

New issue

Open

opened 2025-01-14 12:17:31 +00:00 by tarwebrood · 1 comment

tarwebrood commented 2025-01-14 12:17:31 +00:00

Copy link

What I'm trying to do and why

I want to collect statistics on my backup tasks, including quota usage & remaining quota space.

I already have "Healthchecks.io" hook configured in Borgmatic, to alert me whenever an error occurred during a backup task, or when no backup has been successfully completed for a predetermined amount of time (using the healthchecks.io service settings).

Now I also want to extract numbers from my backups, to create graphs (in Zabbix or Grafana, using the Loki hook) and to have my monitoring solution warn me before I run out of space, or whenever the backup increases in size unexpectedly.

I have seen a thread on the borgbackup issue tracker regarding viewing quota ( https://github.com/borgbackup/borg/issues/2870 ) but it is unclear to me if this information is available in Borgmatic, and if so, how to output it.

Can anyone tell me how I can get quota usage/available numbers to Loki or as JSON?

Additional link that might be useful:
Borgbackup issue #7757 Quota with a central repository server - https://github.com/borgbackup/borg/issues/7757

Steps to reproduce

No response

Actual behavior

No response

Expected behavior

No response

Other notes / implementation ideas

a huge amount of thanks for everybody contributing to this project!

borgmatic version

1.9.5

borgmatic installation method

pip install

Borg version

borg 1.4.0

Python version

Python 3.11.2

Database version (if applicable)

No response

Operating system and version

Debian GNU/Linux 12 (bookworm)

### What I'm trying to do and why I want to collect statistics on my backup tasks, including quota usage & remaining quota space. I already have "Healthchecks.io" hook configured in Borgmatic, to alert me whenever an error occurred during a backup task, or when no backup has been successfully completed for a predetermined amount of time (using the healthchecks.io service settings). Now I also want to extract numbers from my backups, to create graphs (in Zabbix or Grafana, using the Loki hook) and to have my monitoring solution warn me before I run out of space, or whenever the backup increases in size unexpectedly. I have seen a thread on the borgbackup issue tracker regarding viewing quota ( https://github.com/borgbackup/borg/issues/2870 ) but it is unclear to me if this information is available in Borgmatic, and if so, how to output it. Can anyone tell me how I can get quota usage/available numbers to Loki or as JSON? Additional link that might be useful: Borgbackup issue #7757 Quota with a central repository server - https://github.com/borgbackup/borg/issues/7757 ### Steps to reproduce _No response_ ### Actual behavior _No response_ ### Expected behavior _No response_ ### Other notes / implementation ideas a huge amount of thanks for everybody contributing to this project! ### borgmatic version 1.9.5 ### borgmatic installation method pip install ### Borg version borg 1.4.0 ### Python version Python 3.11.2 ### Database version (if applicable) _No response_ ### Operating system and version Debian GNU/Linux 12 (bookworm)

witten commented 2025-01-15 00:03:07 +00:00

Owner

Copy link

Thanks for taking the time to file this. If borgmatic were to support this, I think the first step would be get to the quota usage and limit out of Borg in a programmatic way. I see the quota usage logged upon create, but it's just a plaintext log and doesn't appear to show up in the create --json output. And I don't see it all in borg info output, which doesn't make a whole lot of sense to me given the first ticket you linked. And it's also kind of weird because I can get the plaintext quota info out of many other commands.. prune, compact, etc.

Once we have the quota usage in borgmatic, the next step would be getting it somewhere useful. For Loki, do you have a sense of a good place to put that data? For instance, I see that the existing borgmatic Loki hook has a per-logging-stream "label" mechanism. Is that an appropriate place to record the quota usage data?

If you're looking to just do this yourself, the place I've found for the quota usage data is in the output of borgmatic -v 1 create (-v 2 also works). Example:

...
Storage quota: 702.52 kB out of 1.00 GB used.
...

Related: #617

Thanks for taking the time to file this. If borgmatic were to support this, I think the first step would be get to the quota usage and limit out of Borg in a programmatic way. I see the quota usage logged upon `create`, but it's just a plaintext log and doesn't appear to show up in the `create --json` output. And I don't see it all in `borg info` output, which doesn't make a whole lot of sense to me given the first ticket you linked. And it's also kind of weird because I can get the plaintext quota info out of many other commands.. `prune`, `compact`, etc. Once we have the quota usage in borgmatic, the next step would be getting it somewhere useful. For Loki, do you have a sense of a good place to put that data? For instance, I see that the existing borgmatic Loki hook has a per-logging-stream "label" mechanism. Is that an appropriate place to record the quota usage data? If you're looking to just do this yourself, the place I've found for the quota usage data is in the output of `borgmatic -v 1 create` (`-v 2` also works). Example: ``` ... Storage quota: 702.52 kB out of 1.00 GB used. ... ``` Related: #617

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

parallel-borgmatic

command-hook-steps

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

blocked

  

breaking

  

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

borgmatic-collective/borgmatic#967

No description provided.