From 5e3c2da79c82c10ffeac7610c2cb4aebcce4193e Mon Sep 17 00:00:00 2001 From: Dan Helfman Date: Wed, 23 Oct 2019 15:35:37 -0700 Subject: [PATCH] Database dump hooks documentation (#225). --- .eleventy.js | 4 - NEWS | 2 +- README.md | 1 + ...reparation-and-cleanup-steps-to-backups.md | 33 ++++---- docs/how-to/backup-your-databases.md | 81 +++++++++++++++++++ docs/how-to/deal-with-very-large-backups.md | 2 +- docs/how-to/develop-on-borgmatic.md | 2 +- docs/how-to/inspect-your-backups.md | 8 +- docs/how-to/make-per-application-backups.md | 2 +- docs/how-to/monitor-your-backups.md | 12 +-- docs/how-to/restore-a-backup.md | 6 +- docs/how-to/set-up-backups.md | 14 ++-- docs/how-to/upgrade.md | 2 +- docs/reference/command-line.md | 4 +- docs/reference/configuration.md | 4 +- setup.py | 2 +- 16 files changed, 130 insertions(+), 49 deletions(-) create mode 100644 docs/how-to/backup-your-databases.md diff --git a/.eleventy.js b/.eleventy.js index 7ab5109a9..1b5f48b3b 100644 --- a/.eleventy.js +++ b/.eleventy.js @@ -13,11 +13,7 @@ module.exports = function(eleventyConfig) { html: true, breaks: false, linkify: true, - // Replace links to .md files with links to directories. This allows unparsed Markdown links - // to work on GitHub, while rendered links elsewhere also work. replaceLink: function (link, env) { - link = link.replace(/\.md$/, '/'); - if (process.env.NODE_ENV == "production") { return link; } diff --git a/NEWS b/NEWS index 2dd7107b2..7798dde1a 100644 --- a/NEWS +++ b/NEWS @@ -1,4 +1,4 @@ -1.3.27.dev0 +1.4.0 * #225: Database dump hooks for PostgreSQL, so you can easily dump your databases before backups run. * #230: Rename "borgmatic list --pattern-from" flag to "--patterns-from" to match Borg. diff --git a/README.md b/README.md index e4d80ee44..87faaef39 100644 --- a/README.md +++ b/README.md @@ -73,6 +73,7 @@ href="https://asciinema.org/a/203761" target="_blank">screencast. * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) * [Restore a backup](https://torsion.org/borgmatic/docs/how-to/restore-a-backup/) + * [Backup your databases](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/) * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/) * [Upgrade borgmatic](https://torsion.org/borgmatic/docs/how-to/upgrade/) * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/) diff --git a/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md b/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md index b7d04d32c..1f3b0c3e1 100644 --- a/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md +++ b/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md @@ -1,23 +1,26 @@ --- -title: Add preparation and cleanup steps to backups +title: How to add preparation and cleanup steps to backups --- ## Preparation and cleanup hooks If you find yourself performing prepraration tasks before your backup runs, or -cleanup work afterwards, borgmatic hooks may be of interest. Hooks are -shell commands that borgmatic executes for you at various points, and they're -configured in the `hooks` section of your configuration file. +cleanup work afterwards, borgmatic hooks may be of interest. Hooks are shell +commands that borgmatic executes for you at various points, and they're +configured in the `hooks` section of your configuration file. But if you're +looking to backup a database, it's probably easier to use the [database backup +feature](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/) +instead. -For instance, you can specify `before_backup` hooks to dump a database to file -before backing it up, and specify `after_backup` hooks to delete the temporary -file afterwards. Here's an example: +You can specify `before_backup` hooks to perform preparation steps before +running backups, and specify `after_backup` hooks to perform cleanup steps +afterwards. Here's an example: ```yaml hooks: before_backup: - - dump-a-database /to/file.sql + - mount /some/filesystem after_backup: - - rm /to/file.sql + - umount /some/filesystem ``` The `before_backup` and `after_backup` hooks each run once per configuration @@ -49,14 +52,14 @@ a backup or a backup hook, but not if an error occurs during a borgmatic also runs `on_error` hooks if an error occurs, either when creating a backup or running a backup hook. See the [monitoring and alerting -documentation](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups.md) +documentation](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) for more information. ## Hook output Any output produced by your hooks shows up both at the console and in syslog (when run in a non-interactive console). For more information, read about inspecting +href="https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/">inspecting your backups. ## Security @@ -70,7 +73,7 @@ invoked by hooks. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [Make per-application backups](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups.md) - * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups.md) - * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [Backup your databases](https://torsion.org/borgmatic/docs/how-to/backup-your-databases/) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) + * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) diff --git a/docs/how-to/backup-your-databases.md b/docs/how-to/backup-your-databases.md new file mode 100644 index 000000000..ac6551d52 --- /dev/null +++ b/docs/how-to/backup-your-databases.md @@ -0,0 +1,81 @@ +--- +title: How to backup your databases +--- +## Database dump hooks + +If you want to backup a database, it's best practice with most database +systems to backup an exported database dump, rather than backing up your +database's internal file storage. That's because the internal storage can +change while you're reading from it. In contrast, a database dump creates a +consistent snapshot that is more suited for backups. + +Fortunately, borgmatic includes built-in support for creating database dumps +prior to running backups. For example, here is everything you need to dump and +backup a couple of local PostgreSQL databases: + +```yaml +hooks: + postgresql_databases: + - name: users + - name: orders +``` + +Prior to each backup, borgmatic dumps each configured database to a file +(located in `~/.borgmatic/`) and includes it in the backup. After the backup +completes, borgmatic removes the database dump files to recover disk space. + +Here's a more involved example that connects to a remote database: + +```yaml +hooks: + postgresql_databases: + - name: users + hostname: database.example.org + port: 5433 + username: dbuser + password: trustsome1 + format: tar + options: "--role=someone" +``` + +If you want to dump all databases on a host, use `all` for the database name: + +```yaml +hooks: + postgresql_databases: + - name: all +``` + +Note that you may need to use a `username` of the `postgres` superuser for +this to work. + +## Supported databases + +As of now, borgmatic only supports PostgreSQL databases directly. But see +below about general-purpose preparation and cleanup hooks as a work-around +with other database systems. Also, please [file a +ticket](https://torsion.org/borgmatic/#issues) for additional database systems +that you'd like supported. + +## Database restoration + +borgmatic does not yet perform integrated database restoration when you +[restore a backup](http://localhost:8080/docs/how-to/restore-a-backup/), but +that feature is coming in a future release. In the meantime, you can restore +a database manually after restoring a dump file in the `~/.borgmatic` path. + +## Preparation and cleanup hooks + +If this database integration is too limited for needs, borgmatic also supports +general-purpose [preparation and cleanup +hooks](https://torsion.org/borgmatic/docs/how-to/set-up-backups/). These +hooks allows you to trigger arbitrary commands or scripts before and after +backups. So if necessary, you can use these hooks to create database dumps +with any database system. + +## Related documentation + + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) + * [Restore a backup](http://localhost:8080/docs/how-to/restore-a-backup/) diff --git a/docs/how-to/deal-with-very-large-backups.md b/docs/how-to/deal-with-very-large-backups.md index 219015c8e..f13d3c82b 100644 --- a/docs/how-to/deal-with-very-large-backups.md +++ b/docs/how-to/deal-with-very-large-backups.md @@ -106,4 +106,4 @@ backups. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) diff --git a/docs/how-to/develop-on-borgmatic.md b/docs/how-to/develop-on-borgmatic.md index 9a6254acd..e618cff8a 100644 --- a/docs/how-to/develop-on-borgmatic.md +++ b/docs/how-to/develop-on-borgmatic.md @@ -109,4 +109,4 @@ also linked from the commits list on each pull request. ## Related documentation - * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups.md) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) diff --git a/docs/how-to/inspect-your-backups.md b/docs/how-to/inspect-your-backups.md index aac26dc56..50ce170ce 100644 --- a/docs/how-to/inspect-your-backups.md +++ b/docs/how-to/inspect-your-backups.md @@ -86,7 +86,7 @@ already has this rate limit disabled. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups.md) - * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md) - * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) + * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/) + * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/) diff --git a/docs/how-to/make-per-application-backups.md b/docs/how-to/make-per-application-backups.md index 0333c3392..f962b8138 100644 --- a/docs/how-to/make-per-application-backups.md +++ b/docs/how-to/make-per-application-backups.md @@ -112,4 +112,4 @@ directly, please see the section above about standard includes. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) diff --git a/docs/how-to/monitor-your-backups.md b/docs/how-to/monitor-your-backups.md index 282e79727..db84032f2 100644 --- a/docs/how-to/monitor-your-backups.md +++ b/docs/how-to/monitor-your-backups.md @@ -90,7 +90,7 @@ here: Note that borgmatic does not run `on_error` hooks if an error occurs within a `before_everything` or `after_everything` hook. For more about hooks, see the [borgmatic hooks -documentation](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md), +documentation](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/), especially the security information. @@ -151,8 +151,8 @@ fancier with your archive listing. See `borg list --help` for more flags. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups.md) - * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups.md) - * [Restore a backup](https://torsion.org/borgmatic/docs/how-to/restore-a-backup.md) - * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) + * [Add preparation and cleanup steps to backups](https://torsion.org/borgmatic/docs/how-to/add-preparation-and-cleanup-steps-to-backups/) + * [Restore a backup](https://torsion.org/borgmatic/docs/how-to/restore-a-backup/) + * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/) diff --git a/docs/how-to/restore-a-backup.md b/docs/how-to/restore-a-backup.md index 0f771abe4..89756fc3a 100644 --- a/docs/how-to/restore-a-backup.md +++ b/docs/how-to/restore-a-backup.md @@ -63,6 +63,6 @@ Like a whole-archive restore, this also restores into the current directory. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups.md) - * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) + * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) diff --git a/docs/how-to/set-up-backups.md b/docs/how-to/set-up-backups.md index 526cfb5f1..f436041c2 100644 --- a/docs/how-to/set-up-backups.md +++ b/docs/how-to/set-up-backups.md @@ -77,7 +77,7 @@ else borgmatic won't recognize the option. Also be sure to use spaces rather than tabs for indentation; YAML does not allow tabs. You can also get the same sample configuration file from the [configuration -reference](https://torsion.org/borgmatic/docs/reference/configuration.md), the authoritative set of +reference](https://torsion.org/borgmatic/docs/reference/configuration/), the authoritative set of all configuration options. This is handy if borgmatic has added new options since you originally created your configuration file. @@ -244,9 +244,9 @@ it. ## Related documentation - * [Make per-application backups](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups.md) - * [Deal with very large backups](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups.md) - * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups.md) - * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups.md) - * [borgmatic configuration reference](https://torsion.org/borgmatic/docs/reference/configuration.md) - * [borgmatic command-line reference](https://torsion.org/borgmatic/docs/reference/command-line.md) + * [Make per-application backups](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/) + * [Deal with very large backups](https://torsion.org/borgmatic/docs/how-to/deal-with-very-large-backups/) + * [Inspect your backups](https://torsion.org/borgmatic/docs/how-to/inspect-your-backups/) + * [Monitor your backups](https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/) + * [borgmatic configuration reference](https://torsion.org/borgmatic/docs/reference/configuration/) + * [borgmatic command-line reference](https://torsion.org/borgmatic/docs/reference/command-line/) diff --git a/docs/how-to/upgrade.md b/docs/how-to/upgrade.md index da1103a55..c66ebd47a 100644 --- a/docs/how-to/upgrade.md +++ b/docs/how-to/upgrade.md @@ -76,4 +76,4 @@ files. ## Related documentation - * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic.md) + * [Develop on borgmatic](https://torsion.org/borgmatic/docs/how-to/develop-on-borgmatic/) diff --git a/docs/reference/command-line.md b/docs/reference/command-line.md index b66e5fad5..eca09d322 100644 --- a/docs/reference/command-line.md +++ b/docs/reference/command-line.md @@ -13,5 +13,5 @@ each action sub-command: ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [borgmatic configuration reference](https://torsion.org/borgmatic/docs/reference/configuration.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [borgmatic configuration reference](https://torsion.org/borgmatic/docs/reference/configuration/) diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index e741a7c51..550c0f6cc 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -15,5 +15,5 @@ file](https://torsion.org/borgmatic/docs/reference/config.yaml) for use locally. ## Related documentation - * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups.md) - * [borgmatic command-line reference](https://torsion.org/borgmatic/docs/reference/command-line.md) + * [Set up backups with borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/) + * [borgmatic command-line reference](https://torsion.org/borgmatic/docs/reference/command-line/) diff --git a/setup.py b/setup.py index e43926367..6f4cef61d 100644 --- a/setup.py +++ b/setup.py @@ -1,6 +1,6 @@ from setuptools import find_packages, setup -VERSION = '1.3.27.dev0' +VERSION = '1.4.0' setup(