borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
12
63
Fork 16
Code Issues 50 Pull Requests 1 Actions Packages Releases 133 Activity

Feature Request: Store backup results to a JSON file #617

New Issue
Open
opened 2022-12-01 20:06:05 +00:00 by crosenbe · 7 comments
crosenbe commented 2022-12-01 20:06:05 +00:00
Copy Link

What I'm trying to do and why

Monitoring backups is essential and the only way to provide information to a monitoring system is to use hooks. Traps are not reliable, storing to a file is perhaps the better idea. But the status is not enough, to monitor the backups more informations will be useful.

  • start time of the last backup
  • finish time of the last backup
  • status flag of the last backup
  • duration of the last backup
  • time of last successful backup
  • used borgmatic configuration file
  • valid maximum age of the last backup

Example JSON file

[
    {
        "finished": "24.11.2022 - 01:15:13",
        "started": "24.11.2022 - 01:13:20",
        "finished_unix": 1669248913,
        "status": 0,
        "started_unix": 1669248800,
        "last_success": "24.11.2022 - 01:15:13",
        "config": "/etc/borgmatic/nas/wifictl.yaml",
        "duration": 1,
        "last_success_unix": 1669248913,
        "max_age": 691200
    },
]

This can be done by using hooks and a provided python script, i.e.

hooks:
    before_backup:
        - write-status -c start -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8

    after_backup:
        - write-status -c success -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8

    on_error:
        - write-status -c failure -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8

write-status

import json
import getopt
from datetime import datetime as dt
import sys

def getTime():
    stamp = dt.now()
    return [ stamp.strftime('%d.%m.%Y - %H:%M:%S'), int(stamp.timestamp()) ]

def readJson(file):
    try:
        with open(file, 'r') as json_file:
            backup_results = json.load(json_file)
            return backup_results
    except OSError as e:
        return []

def writeJson(file, backup_results):
    json_string = json.dumps(backup_results, indent=4)
    with open(file, 'w') as json_file:
        json_file.write(json_string)

def run(command, config, file, max_age):
    try:
        max_age = int(max_age) * 86400
    except ValueError:
        usage()
    backup_results = readJson(file)
    time_stamp, time_stamp_unix = getTime()
    if backup_results is not None:
        config_list = [d['config'] for d in backup_results]
        if config not in config_list:
            backup_results.append( {'config': config } )
            config_list.append(config)

    list_entry = config_list.index(config)
    if command == 'start':
        backup_results[list_entry]['started'] = time_stamp
        backup_results[list_entry]['started_unix'] = time_stamp_unix
        backup_results[list_entry]['status'] = 1
    elif command == 'success':
        backup_results[list_entry]['duration'] = int(( time_stamp_unix - backup_results[list_entry]['started_unix'] ) / 60)
        backup_results[list_entry]['finished'] = time_stamp
        backup_results[list_entry]['finished_unix'] = time_stamp_unix
        backup_results[list_entry]['last_success'] = time_stamp
        backup_results[list_entry]['last_success_unix'] = time_stamp_unix
        backup_results[list_entry]['status'] = 0
    elif command == 'failure':
        backup_results[list_entry]['duration'] = int(( time_stamp_unix - backup_results[list_entry]['started_unix'] ) / 60)
        backup_results[list_entry]['finished'] = time_stamp
        backup_results[list_entry]['finished_unix'] = time_stamp_unix
        backup_results[list_entry]['status'] = 2
    else:
        usage()

    if 'last_success_unix' not in backup_results[list_entry]:
        backup_results[list_entry]['last_success_unix'] = 0
    if 'duration' not in backup_results[list_entry]:
        backup_results[list_entry]['duration'] = 0

    backup_results[list_entry]['max_age'] = max_age
    writeJson(file, backup_results)

def usage():
    print('write-status -c <command> -C <configuration> -f <file> -m <max-age>')
    print('     command:       start,success,failure)')
    print('     configuration: path to the borgmatic configuration file (yaml)')
    print('     file:          path to log json file results shoule be written')
    print('     max-age:       maximum age of last successful backup in days')
    exit(0)

def start(argv):
    command = ''
    config = ''
    file = ''
    max_age = ''

    try:
        opts, args = getopt.getopt(argv,"hc:C:f:m:",["help","command=","config=","file=","max-age="])
    except getopt.GetoptError:
        usage()
    for opt,arg in opts:
        if opt in ("-h", "--help"):
            usage()
        elif opt in ("-c", "--command"):
            if arg in ('start', 'success', 'failure'):
                command = arg
            else:
                usage()
        elif opt in ("-C", "--config"):
            config = arg
        elif opt in ("-f", "--file"):
            file = arg
        elif opt in ("-m", "--max-age"):
            max_age = arg

    if '' in (command, config, file, max_age):
        usage()
    else:
        run(command, config, file, max_age)

if __name__ == "__main__":
    start(sys.argv[1:])

Other notes / implementation ideas

It would be more elegant to have this feature integrated and extended providing more information about the backup. The integration could be done by providing relevant information as variable, i.e. defining the max_age inside the borgmatic configuration and to be used like {configuration_filename}. Additional information like transferred data to the backup repository would be also nice to see in the status. But this information is only forwarded from borg through borgmatic. Maybe it's possible to send all forwarded logs through a couple of regex and fetch data and store it also to variables.

Log Example
borgmatic: INFO RemoteRepository: 366.38 MB bytes sent, 32.64 kB bytes received, 215 messages sent

Maybe to fetch the data like that and store this to variables as well.

\s(?P<sent>[^\s]+)\s(?P<sentunit>\w\w)\sbytes\ssent,\s(?P<received>[^\s]+)\s(?P<receivedunit>\w\w)\sbytes\sreceived
#### What I'm trying to do and why Monitoring backups is essential and the only way to provide information to a monitoring system is to use hooks. Traps are not reliable, storing to a file is perhaps the better idea. But the status is not enough, to monitor the backups more informations will be useful. * start time of the last backup * finish time of the last backup * status flag of the last backup * duration of the last backup * time of last successful backup * used borgmatic configuration file * valid maximum age of the last backup *Example JSON file* ``` [ { "finished": "24.11.2022 - 01:15:13", "started": "24.11.2022 - 01:13:20", "finished_unix": 1669248913, "status": 0, "started_unix": 1669248800, "last_success": "24.11.2022 - 01:15:13", "config": "/etc/borgmatic/nas/wifictl.yaml", "duration": 1, "last_success_unix": 1669248913, "max_age": 691200 }, ] ``` This can be done by using hooks and a provided python script, i.e. ``` hooks: before_backup: - write-status -c start -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8 after_backup: - write-status -c success -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8 on_error: - write-status -c failure -C {configuration_filename} -f /var/log/backups/borgmatic.json -m 8 ``` *write-status* ``` import json import getopt from datetime import datetime as dt import sys def getTime(): stamp = dt.now() return [ stamp.strftime('%d.%m.%Y - %H:%M:%S'), int(stamp.timestamp()) ] def readJson(file): try: with open(file, 'r') as json_file: backup_results = json.load(json_file) return backup_results except OSError as e: return [] def writeJson(file, backup_results): json_string = json.dumps(backup_results, indent=4) with open(file, 'w') as json_file: json_file.write(json_string) def run(command, config, file, max_age): try: max_age = int(max_age) * 86400 except ValueError: usage() backup_results = readJson(file) time_stamp, time_stamp_unix = getTime() if backup_results is not None: config_list = [d['config'] for d in backup_results] if config not in config_list: backup_results.append( {'config': config } ) config_list.append(config) list_entry = config_list.index(config) if command == 'start': backup_results[list_entry]['started'] = time_stamp backup_results[list_entry]['started_unix'] = time_stamp_unix backup_results[list_entry]['status'] = 1 elif command == 'success': backup_results[list_entry]['duration'] = int(( time_stamp_unix - backup_results[list_entry]['started_unix'] ) / 60) backup_results[list_entry]['finished'] = time_stamp backup_results[list_entry]['finished_unix'] = time_stamp_unix backup_results[list_entry]['last_success'] = time_stamp backup_results[list_entry]['last_success_unix'] = time_stamp_unix backup_results[list_entry]['status'] = 0 elif command == 'failure': backup_results[list_entry]['duration'] = int(( time_stamp_unix - backup_results[list_entry]['started_unix'] ) / 60) backup_results[list_entry]['finished'] = time_stamp backup_results[list_entry]['finished_unix'] = time_stamp_unix backup_results[list_entry]['status'] = 2 else: usage() if 'last_success_unix' not in backup_results[list_entry]: backup_results[list_entry]['last_success_unix'] = 0 if 'duration' not in backup_results[list_entry]: backup_results[list_entry]['duration'] = 0 backup_results[list_entry]['max_age'] = max_age writeJson(file, backup_results) def usage(): print('write-status -c <command> -C <configuration> -f <file> -m <max-age>') print(' command: start,success,failure)') print(' configuration: path to the borgmatic configuration file (yaml)') print(' file: path to log json file results shoule be written') print(' max-age: maximum age of last successful backup in days') exit(0) def start(argv): command = '' config = '' file = '' max_age = '' try: opts, args = getopt.getopt(argv,"hc:C:f:m:",["help","command=","config=","file=","max-age="]) except getopt.GetoptError: usage() for opt,arg in opts: if opt in ("-h", "--help"): usage() elif opt in ("-c", "--command"): if arg in ('start', 'success', 'failure'): command = arg else: usage() elif opt in ("-C", "--config"): config = arg elif opt in ("-f", "--file"): file = arg elif opt in ("-m", "--max-age"): max_age = arg if '' in (command, config, file, max_age): usage() else: run(command, config, file, max_age) if __name__ == "__main__": start(sys.argv[1:]) ``` #### Other notes / implementation ideas It would be more elegant to have this feature integrated and extended providing more information about the backup. The integration could be done by providing relevant information as variable, i.e. defining the max_age inside the borgmatic configuration and to be used like {configuration_filename}. Additional information like transferred data to the backup repository would be also nice to see in the status. But this information is only forwarded from borg through borgmatic. Maybe it's possible to send all forwarded logs through a couple of regex and fetch data and store it also to variables. *Log Example* `borgmatic: INFO RemoteRepository: 366.38 MB bytes sent, 32.64 kB bytes received, 215 messages sent` Maybe to fetch the data like that and store this to variables as well. ``` \s(?P<sent>[^\s]+)\s(?P<sentunit>\w\w)\sbytes\ssent,\s(?P<received>[^\s]+)\s(?P<receivedunit>\w\w)\sbytes\sreceived ```
witten commented 2022-12-02 06:26:04 +00:00
Owner
Copy Link

That's an interesting idea about parsing the Borg output for basic backup statistics. The currently supported way to monitor the status and duration of borgmatic backups is to use one of the supported third-party monitoring services, some of which you can host yourself if you prefer. I assume none of those will work for your use case or you're already settled on a different tool? And/or perhaps you need the additional stats like bytes sent/received, etc?

Could you talk about how you intend to consume this JSON file? And what actions (if any) do you intend to take when this downstream tool consumes the JSON? Thanks!

That's an interesting idea about parsing the Borg output for basic backup statistics. The currently supported way to monitor the status and duration of borgmatic backups is to use one of the supported [third-party monitoring services](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#third-party-monitoring-services), some of which you can host yourself if you prefer. I assume none of those will work for your use case or you're already settled on a different tool? And/or perhaps you need the additional stats like bytes sent/received, etc? Could you talk about how you intend to consume this JSON file? And what actions (if any) do you intend to take when this downstream tool consumes the JSON? Thanks!
crosenbe commented 2022-12-02 08:22:59 +00:00
Author
Copy Link

I consume the file with Zabbix, just monitoring this file and discover it. The big advantage is i don't have to do to much configuration in zabbix in case of iterating this file and creating the monitored items in an automated way. As soon i create a new backup job a new JSON entry will be created and detected by Zabbix. And i don't need additional tools and be able to integrate borgmatic into the central monitoring solution.

Zabbix template

{
    "zabbix_export": {
        "version": "6.0",
        "date": "2022-12-02T08:17:39Z",
        "groups": [
            {
                "uuid": "7df96b18c230490a9a0a9e2307226338",
                "name": "Templates"
            },
        ],
        "templates": [
            {
                "uuid": "60b5c1230248485991079e203c80451e",
                "template": "BorgMatic",
                "name": "BorgMatic",
                "description": "Monitor backup result file.",
                "groups": [
                    {
                        "name": "Templates"
                    },
                ],
                "items": [
                    {
                        "uuid": "8983cc49c662405298b31dbe6164aff0",
                        "name": "Backup Status File",
                        "key": "vfs.file.contents[{$MON_FILE}]",
                        "history": "365d",
                        "trends": "0",
                        "value_type": "TEXT"
                    }
                ],
                "discovery_rules": [
                    {
                        "uuid": "4ea0345efbcb4d4aacc774b104e6031f",
                        "name": "Borgmatic Discovery",
                        "type": "DEPENDENT",
                        "key": "borgmatic.discover",
                        "delay": "0",
                        "item_prototypes": [
                            {
                                "uuid": "3689ee87e9a14e4dba663d2378bcb750",
                                "name": "Borgmatic Last Backup Duration [{#CONFIG}]",
                                "type": "DEPENDENT",
                                "key": "borgmatic.last.duration[{#CONFIG}]",
                                "delay": "0",
                                "units": "m",
                                "preprocessing": [
                                    {
                                        "type": "JSONPATH",
                                        "parameters": [
                                            "$[?(@.config == \"{#CONFIG}\")].duration.first()"
                                        ]
                                    }
                                ],
                                "master_item": {
                                    "key": "vfs.file.contents[{$MON_FILE}]"
                                },
                                "tags": [
                                    {
                                        "tag": "Application",
                                        "value": "Borgmatic"
                                    }
                                ]
                            },
                            {
                                "uuid": "9219741cce5640168dd45614fcce8ca0",
                                "name": "Borgmatic Last Success Backup [{#CONFIG}]",
                                "type": "DEPENDENT",
                                "key": "borgmatic.last_success[{#CONFIG}]",
                                "delay": "0",
                                "units": "unixtime",
                                "preprocessing": [
                                    {
                                        "type": "JSONPATH",
                                        "parameters": [
                                            "$[?(@.config == \"{#CONFIG}\")].last_success_unix.first()"
                                        ]
                                    }
                                ],
                                "master_item": {
                                    "key": "vfs.file.contents[{$MON_FILE}]"
                                },
                                "tags": [
                                    {
                                        "tag": "Application",
                                        "value": "Borgmatic"
                                    }
                                ]
                            },
                            {
                                "uuid": "f5056227e1414e0ea3eff1038c882271",
                                "name": "Borgmatic Valid Max Age [{#CONFIG}]",
                                "type": "DEPENDENT",
                                "key": "borgmatic.max_age[{#CONFIG}]",
                                "delay": "0",
                                "units": "s",
                                "preprocessing": [
                                    {
                                        "type": "JSONPATH",
                                        "parameters": [
                                            "$[?(@.config == \"{#CONFIG}\")].max_age.first()"
                                        ]
                                    }
                                ],
                                "master_item": {
                                    "key": "vfs.file.contents[{$MON_FILE}]"
                                },
                                "tags": [
                                    {
                                        "tag": "Application",
                                        "value": "Borgmatic"
                                    }
                                ]
                            },
                            {
                                "uuid": "247fe516d3374fe48d6b859884217e97",
                                "name": "Borgmatic Status [{#CONFIG}]",
                                "type": "DEPENDENT",
                                "key": "borgmatic.status[{#CONFIG}]",
                                "delay": "0",
                                "valuemap": {
                                    "name": "Backup Status"
                                },
                                "preprocessing": [
                                    {
                                        "type": "JSONPATH",
                                        "parameters": [
                                            "$[?(@.config == \"{#CONFIG}\")].status.first()"
                                        ]
                                    }
                                ],
                                "master_item": {
                                    "key": "vfs.file.contents[{$MON_FILE}]"
                                },
                                "tags": [
                                    {
                                        "tag": "Application",
                                        "value": "Borgmatic"
                                    }
                                ]
                            }
                        ],
                        "trigger_prototypes": [
                            {
                                "uuid": "28e78881fa0e4754bd4d3ac32dc4be20",
                                "expression": "now() - last(/BorgMatic/borgmatic.max_age[{#CONFIG}]) > last(/BorgMatic/borgmatic.last_success[{#CONFIG}])",
                                "recovery_mode": "RECOVERY_EXPRESSION",
                                "recovery_expression": "now() - last(/BorgMatic/borgmatic.max_age[{#CONFIG}]) < last(/BorgMatic/borgmatic.last_success[{#CONFIG}])",
                                "name": "Old Backup [{#CONFIG}]",
                                "priority": "HIGH",
                                "description": "Last Successful Backup is to old",
                                "manual_close": "YES"
                            }
                        ],
                        "master_item": {
                            "key": "vfs.file.contents[{$MON_FILE}]"
                        },
                        "lld_macro_paths": [
                            {
                                "lld_macro": "{#CONFIG}",
                                "path": "$..config.first()"
                            }
                        ]
                    }
                ],
                "macros": [
                    {
                        "macro": "{$MON_FILE}",
                        "value": "/var/log/backups/borgmatic.json"
                    },
                ],
                "valuemaps": [
                    {
                        "uuid": "ac3d95e26ada42c68e7f6c1cc46fb6c8",
                        "name": "Backup Status",
                        "mappings": [
                            {
                                "value": "0",
                                "newvalue": "Success"
                            },
                            {
                                "value": "1",
                                "newvalue": "Running"
                            },
                            {
                                "value": "2",
                                "newvalue": "Failure"
                            }
                        ]
                    }
                ]
            }
        ]
    }
}
I consume the file with Zabbix, just monitoring this file and discover it. The big advantage is i don't have to do to much configuration in zabbix in case of iterating this file and creating the monitored items in an automated way. As soon i create a new backup job a new JSON entry will be created and detected by Zabbix. And i don't need additional tools and be able to integrate borgmatic into the central monitoring solution. *Zabbix template* ``` { "zabbix_export": { "version": "6.0", "date": "2022-12-02T08:17:39Z", "groups": [ { "uuid": "7df96b18c230490a9a0a9e2307226338", "name": "Templates" }, ], "templates": [ { "uuid": "60b5c1230248485991079e203c80451e", "template": "BorgMatic", "name": "BorgMatic", "description": "Monitor backup result file.", "groups": [ { "name": "Templates" }, ], "items": [ { "uuid": "8983cc49c662405298b31dbe6164aff0", "name": "Backup Status File", "key": "vfs.file.contents[{$MON_FILE}]", "history": "365d", "trends": "0", "value_type": "TEXT" } ], "discovery_rules": [ { "uuid": "4ea0345efbcb4d4aacc774b104e6031f", "name": "Borgmatic Discovery", "type": "DEPENDENT", "key": "borgmatic.discover", "delay": "0", "item_prototypes": [ { "uuid": "3689ee87e9a14e4dba663d2378bcb750", "name": "Borgmatic Last Backup Duration [{#CONFIG}]", "type": "DEPENDENT", "key": "borgmatic.last.duration[{#CONFIG}]", "delay": "0", "units": "m", "preprocessing": [ { "type": "JSONPATH", "parameters": [ "$[?(@.config == \"{#CONFIG}\")].duration.first()" ] } ], "master_item": { "key": "vfs.file.contents[{$MON_FILE}]" }, "tags": [ { "tag": "Application", "value": "Borgmatic" } ] }, { "uuid": "9219741cce5640168dd45614fcce8ca0", "name": "Borgmatic Last Success Backup [{#CONFIG}]", "type": "DEPENDENT", "key": "borgmatic.last_success[{#CONFIG}]", "delay": "0", "units": "unixtime", "preprocessing": [ { "type": "JSONPATH", "parameters": [ "$[?(@.config == \"{#CONFIG}\")].last_success_unix.first()" ] } ], "master_item": { "key": "vfs.file.contents[{$MON_FILE}]" }, "tags": [ { "tag": "Application", "value": "Borgmatic" } ] }, { "uuid": "f5056227e1414e0ea3eff1038c882271", "name": "Borgmatic Valid Max Age [{#CONFIG}]", "type": "DEPENDENT", "key": "borgmatic.max_age[{#CONFIG}]", "delay": "0", "units": "s", "preprocessing": [ { "type": "JSONPATH", "parameters": [ "$[?(@.config == \"{#CONFIG}\")].max_age.first()" ] } ], "master_item": { "key": "vfs.file.contents[{$MON_FILE}]" }, "tags": [ { "tag": "Application", "value": "Borgmatic" } ] }, { "uuid": "247fe516d3374fe48d6b859884217e97", "name": "Borgmatic Status [{#CONFIG}]", "type": "DEPENDENT", "key": "borgmatic.status[{#CONFIG}]", "delay": "0", "valuemap": { "name": "Backup Status" }, "preprocessing": [ { "type": "JSONPATH", "parameters": [ "$[?(@.config == \"{#CONFIG}\")].status.first()" ] } ], "master_item": { "key": "vfs.file.contents[{$MON_FILE}]" }, "tags": [ { "tag": "Application", "value": "Borgmatic" } ] } ], "trigger_prototypes": [ { "uuid": "28e78881fa0e4754bd4d3ac32dc4be20", "expression": "now() - last(/BorgMatic/borgmatic.max_age[{#CONFIG}]) > last(/BorgMatic/borgmatic.last_success[{#CONFIG}])", "recovery_mode": "RECOVERY_EXPRESSION", "recovery_expression": "now() - last(/BorgMatic/borgmatic.max_age[{#CONFIG}]) < last(/BorgMatic/borgmatic.last_success[{#CONFIG}])", "name": "Old Backup [{#CONFIG}]", "priority": "HIGH", "description": "Last Successful Backup is to old", "manual_close": "YES" } ], "master_item": { "key": "vfs.file.contents[{$MON_FILE}]" }, "lld_macro_paths": [ { "lld_macro": "{#CONFIG}", "path": "$..config.first()" } ] } ], "macros": [ { "macro": "{$MON_FILE}", "value": "/var/log/backups/borgmatic.json" }, ], "valuemaps": [ { "uuid": "ac3d95e26ada42c68e7f6c1cc46fb6c8", "name": "Backup Status", "mappings": [ { "value": "0", "newvalue": "Success" }, { "value": "1", "newvalue": "Running" }, { "value": "2", "newvalue": "Failure" } ] } ] } ] } } ```
witten commented 2023-03-27 03:48:56 +00:00
Owner
Copy Link

Thanks for providing the additional detail, and I apologize for the delay in getting back to this. Here would be my recommendation for anyone looking to integrate a feature like this with borgmatic:

  • Create a new borgmatic monitoring hook called "json.py" or similar, modeled off one of the existing hooks like Cronitor, Healthchecks, ntfy, etc.
  • Instead of connecting to an external monitoring service when triggered, write monitoring information to a JSON file at a configured location, similar to the example code above.
  • borgmatic doesn't currently track any statistics on borgmatic create results other than success/failure, so some data gathering may need to be added in order to support some of the requested JSON fields. It's possible that an initial version could start simple with only a few fields, and then additional fields could be added over time as needed.

Related ticket: #385.

Thanks for providing the additional detail, and I apologize for the delay in getting back to this. Here would be my recommendation for anyone looking to integrate a feature like this with borgmatic: * Create a new borgmatic monitoring hook called "json.py" or similar, modeled off one of the existing hooks like Cronitor, Healthchecks, ntfy, etc. * Instead of connecting to an external monitoring service when triggered, write monitoring information to a JSON file at a configured location, similar to the example code above. * borgmatic doesn't currently track any statistics on `borgmatic create` results other than success/failure, so some data gathering may need to be added in order to support some of the requested JSON fields. It's possible that an initial version could start simple with only a few fields, and then additional fields could be added over time as needed. Related ticket: #385.
diivi commented 2023-03-27 05:47:28 +00:00
Collaborator
Copy Link

I can take this, it might make it easier for me to test monitoring hooks too.
Should I also add additional data gathered from the output of the create command to other monitoring hooks?

I can take this, it might make it easier for me to test monitoring hooks too. Should I also add additional data gathered from the output of the `create` command to other monitoring hooks?
witten commented 2023-03-27 05:56:12 +00:00
Owner
Copy Link

If it makes sense, sure. Not all of the monitoring hooks have anywhere to slot in that data in a structured way though. I'd also be fine just starting with this proposed hook and worrying about the other monitoring hooks later.

If it makes sense, sure. Not all of the monitoring hooks have anywhere to slot in that data in a structured way though. I'd also be fine just starting with this proposed hook and worrying about the other monitoring hooks later.
👍 1
witten added the
new feature area
 label 2023-06-28 18:39:46 +00:00
witten commented 2024-11-18 20:04:15 +00:00
Owner
Copy Link

Since this ticket was created, a Zabbix monitoring hook has been added to borgmatic. However it's mostly just sending start/finish/fail information, which implicitly satisfies some of the asks here but not all of them. So my question is if:

  1. There's still a need here for the other data points in this ticket, and:
  2. If so, would it work if the existing Zabbix monitoring hook (which connects to a Zabbix server via API) was expanded to send the additional data points as well—instead of writing them to a separate JSON file.
Since this ticket was created, a [Zabbix monitoring hook](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#zabbix-hook) has been added to borgmatic. However it's mostly just sending start/finish/fail information, which implicitly satisfies some of the asks here but not all of them. So my question is if: 1. There's still a need here for the other data points in this ticket, and: 2. If so, would it work if the existing Zabbix monitoring hook (which connects to a Zabbix server via API) was expanded to send the additional data points as well—instead of writing them to a separate JSON file.
witten added the
waiting for response
 label 2024-11-18 20:04:24 +00:00
crosenbe commented 2024-11-18 20:43:18 +00:00
Author
Copy Link

Thanks a lot to integrate a Zabbix hook. The only information which might be useful is to provide metadata of the backup, i.e. how much data are transferred. The JSON file is just a workaround. The most flexible method could be sending different values to specific item ID's. In the best case also manual defined values. The reason behind is the monitoring if a backup should run but it didn't run or stopped before the status were pushed to Zabbix. If we expect to run a backup once a week an alert has to be triggered if the last backup was done 8 days before. This was the reason i used the max-age parameter individually with every backup job.

Thanks a lot to integrate a Zabbix hook. The only information which might be useful is to provide metadata of the backup, i.e. how much data are transferred. The JSON file is just a workaround. The most flexible method could be sending different values to specific item ID's. In the best case also manual defined values. The reason behind is the monitoring if a backup should run but it didn't run or stopped before the status were pushed to Zabbix. If we expect to run a backup once a week an alert has to be triggered if the last backup was done 8 days before. This was the reason i used the max-age parameter individually with every backup job.
👍 1
witten removed the
waiting for response
 label 2024-11-18 20:54:02 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
new feature area
Milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#617
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?