#192 Separate before/after backup run and individual archive hooks

已關閉
elho8 月之前建立 · 5 條評論

The current before_backup/after_backup hooks are only sufficient in a single configuration setup.

When doing backups to multiple archives using /etc/borgmatic.d, there is demand for hooks to run once before all of these individual backups and once after (e.g. creating/removing LVM/filesystem snapshots of the entire system), plus some to run before and after each archive (e.g. dumping a database that goes to one archive only).

The current `before_backup`/`after_backup` hooks are only sufficient in a single configuration setup. When doing backups to multiple archives using `/etc/borgmatic.d`, there is demand for hooks to run once before all of these individual backups and once after (e.g. creating/removing LVM/filesystem snapshots of the entire system), plus some to run before and after each archive (e.g. dumping a database that goes to one archive only).
witten 評論 8 月之前
所有者

Thank you for filing this.. This is something I've wondered about with more complex configurations. I assume when you say “archive” here you mean a Borg “repository”? If so, can you say more about the case when you'd want to dump a database that goes to one repository only? And I'm assuming this is all within one config file that has multiple repositories listed?

Current hooks run per-configuration file, at the time that configuration file is executed. So for instance, if you have a directory full of configuration files in /etc/borgmatic.d, hooks in each configuration file will run before/after all the repositories in that config file do backups.

So if I'm understanding your ask, it sounds like you'd like some hooks to be defined and run per-repository rather than just per-configuration file. (E.g. for a configuration file that has multiple repositories.) Any thoughts on how do define them syntactically? One idea I have is to call out any repositories that the hook applies to when defining them. Example:

hooks:
    before_repository_backup:
        repositories:
          - user@backupserver:example.borg
          - user@otherserver:example.borg
        commands:
          - dump-a-database /to/file.sql
    ...

Would it be at all useful to be able to declare this for multiple repositories as per the example above? Or would it only ever be one repository at a time?

Or am I completely misunderstanding your ask?

Thank you for filing this.. This is something I've wondered about with more complex configurations. I assume when you say "archive" here you mean a Borg "repository"? If so, can you say more about the case when you'd want to dump a database that goes to one repository only? And I'm assuming this is all within one config file that has multiple repositories listed? Current hooks run per-configuration file, at the time that configuration file is executed. So for instance, if you have a directory full of configuration files in `/etc/borgmatic.d`, hooks in each configuration file will run before/after *all* the repositories in that config file do backups. So if I'm understanding your ask, it sounds like you'd like some hooks to be defined and run *per-repository* rather than just per-configuration file. (E.g. for a configuration file that has multiple repositories.) Any thoughts on how do define them syntactically? One idea I have is to call out any repositories that the hook applies to when defining them. Example: ``` hooks: before_repository_backup: repositories: - user@backupserver:example.borg - user@otherserver:example.borg commands: - dump-a-database /to/file.sql ... ``` Would it be at all useful to be able to declare this for multiple repositories as per the example above? Or would it only ever be one repository at a time? Or am I completely misunderstanding your ask?
elho 評論 8 月之前
發佈者

No, by “archive” I do mean borg archive, its the exact opposite situation. :wink: I (currently, due to still testing and refining the setup) only have one repository, but different source directories going into different archives there to allow different retention policies for them, each setup in a separate file in /etc/borgmatic.d, YAML including defaults and repository config from /etc/borgmatic. (In other words, its like the “per-application backups” scenario in the documentation.)

And thus the current hooks would be run whenever one of the config files (for one of the archives) is processed, but I do need tasks to be done before the first and after the last one only. The workaround is quite easy, I do have a wrapper script around borgmatic that invokes the scripts to run before and after borgmatic that works for me, but having this integrated would be nicer, of course.

No, by "archive" I do mean borg archive, its the exact opposite situation. :wink: I (currently, due to still testing and refining the setup) only have one repository, but different source directories going into different archives there to allow different retention policies for them, each setup in a separate file in `/etc/borgmatic.d`, YAML including defaults and repository config from `/etc/borgmatic`. (In other words, its like the "per-application backups" scenario in the documentation.) And thus the current hooks would be run whenever one of the config files (for one of the archives) is processed, but I do need tasks to be done before the first and after the last one only. The workaround is quite easy, I do have a wrapper script around borgmatic that invokes the scripts to run before and after borgmatic that works for me, but having this integrated would be nicer, of course.
witten 評論 8 月之前
所有者

Gotcha! Thanks for clarifying. Okay, so I could see something like this potentially working:

hooks:
    before_all_backups:
        - set-stuff-up-globally
    after_all_backups:
        - clean-stuff-up-globally
    ...

And then you could define those in one or more of your existing borgmatic configuration files. They'd get all gathered up from the various configuration files and run together before/after all backups.

Does that sound like it'd work for you?

Gotcha! Thanks for clarifying. Okay, so I could see something like this potentially working: ```yaml hooks: before_all_backups: - set-stuff-up-globally after_all_backups: - clean-stuff-up-globally ... ``` And then you could define those in one or more of your existing borgmatic configuration files. They'd get all gathered up from the various configuration files and run together before/after all backups. Does that sound like it'd work for you?
elho 評論 8 月之前
發佈者

Yeah, sounds good.

Yeah, sounds good.
witten added the
design finalized
label 8 月之前
witten 評論 4 月之前
所有者

Implemented and released as part of borgmatic 1.3.21. As soon as the docs deploy, you can read about the new hooks here: https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/

They're called before_everything and after_everything. Let me know how they work out for you!

Implemented and released as part of borgmatic 1.3.21. As soon as the docs deploy, you can read about the new hooks here: https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/ They're called `before_everything` and `after_everything`. Let me know how they work out for you!
witten4 月之前 關閉
登入 才能加入這對話。
未選擇里程碑
No Assignees
2 參與者
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
取消
儲存
尚未有任何內容