Tom-Hubrecht
/
borgmatic
forked from borgmatic-collective/borgmatic
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,537 Commits 2 Branches 144 Tags 7.7 MiB
Commit Graph

553 Commits

Author SHA1 Message Date
Dan Helfman e76bfa555f Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523). 2022-05-28 14:42:19 -07:00
Dan Helfman 8ddb7268eb Reuse "borg info" function. 2022-05-27 13:51:11 -07:00
Dan Helfman 77b84f8a48 Add Bash completion script so you can tab-complete the borgmatic command-line. 2022-05-26 10:27:53 -07:00
Dan Helfman be59a3e574 Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options (#536). 2022-05-25 10:26:26 -07:00
polyzen 8708ca07f4 Remove trailing whitespace 2022-05-25 00:43:40 +00:00
Dan Helfman 54933ebef5 Change connection failures for monitoring hooks to be warnings instead of errors (#439). 2022-05-24 15:50:04 -07:00
Dan Helfman 157e59ac88 Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460). 2022-05-24 14:44:33 -07:00
Dan Helfman 666f0dd751 Add missing Healthchecks "states" option example in configuration schema (#525). 2022-05-24 14:17:19 -07:00
Dan Helfman 8b179e4647 Reverse logic of Healtchecks "skip_states" option to just "states" (#525). 2022-05-24 14:09:42 -07:00
Dan Helfman 865eff7d98 Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525). 2022-05-24 13:59:28 -07:00
Dan Helfman b9741f4d0b Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294). 2022-05-24 12:23:38 -07:00
Dan Helfman 02781662f8 Change monitoring hooks to specify the ping URL / integration key as a named option. 2022-05-23 20:02:10 -07:00
Dan Helfman 32a1043468 Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402). 2022-05-23 16:11:24 -07:00
Dan Helfman 3e4aeec649 Warn when an unsupported variable is used in a hook command (#420). 2022-05-23 15:27:54 -07:00
Dan Helfman b98b827594 Remove stale comment. 2022-05-23 10:59:56 -07:00
Dan Helfman 255cc6ec23 When deep merging common configuration, merge colliding list values by appending them (#531). 2022-05-20 15:28:28 -07:00
Dan Helfman 51fc37d57a Improve the error message when a configuration override contains an invalid value (#528). 2022-05-20 13:38:53 -07:00
Dan Helfman 788281cfb9 When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including (#532). 2022-05-19 17:15:05 -07:00
Dan Helfman 525266ede6 Deep merging when including common configuration (#381). 2022-04-25 21:18:37 -07:00
Dan Helfman 0e6b425ac5 Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515). 2022-04-23 14:03:15 -07:00
Dan Helfman ed7fe5c6d0 Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473). 2022-04-21 22:08:25 -07:00
Dan Helfman cbce6707f4 Clarify one_file_system behavior in schema comment (#520). 2022-04-12 11:05:22 -07:00
Dan Helfman e40e726687 Change Healthchecks logs truncation size from 10k bytes to 100k bytes, corresponding to that same change on Healthchecks.io. 2022-04-06 22:00:18 -07:00
Dan Helfman 0c027a3050 Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516). 2022-04-03 13:12:48 -07:00
Dan Helfman 9f44bbad65 Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517). 2022-04-02 22:28:41 -07:00
Dan Helfman db7499db82 Document "repositories" context to for "before_*" and "after_*" command action hooks (#469). 2022-03-14 13:34:14 -07:00
Dan Helfman 6b500c2a8b Add repositories context for command hooks. 
Reviewed-on: borgmatic-collective/borgmatic#469
 2022-03-14 20:13:15 +00:00
Dan Helfman 976516d0e1 When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444). 2022-03-08 10:19:36 -08:00
Dan Helfman 574eb91921 Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507). 2022-03-07 21:46:12 -08:00
Dan Helfman 28fef3264b Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486). 2022-03-07 15:32:07 -08:00
Dan Helfman 9161dbcb7d Removing unnecessary leading underscores from functions. 2022-03-07 11:58:29 -08:00
Dan Helfman 0eb2634f9b Working directory option to support source directories with relative paths (#431). 
Reviewed-on: borgmatic-collective/borgmatic#477
 2022-03-03 19:28:17 +00:00
Dan Helfman 9317cbaaf0 Code formatting. 2022-02-10 10:23:34 -08:00
Dan Helfman 1b5f04b79f When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:16:09 -08:00
Dan Helfman 948c86f62c When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:09:18 -08:00
Dan Helfman 7e7209322a When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 09:51:13 -08:00
Dan Helfman 00a57fd947 Code formatting. 2022-02-09 21:20:28 -08:00
Dan Helfman 6bf6ac310b When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-09 21:11:00 -08:00
Dan Helfman 4b5af2770d When the "atime" option is used, tailor the flags passed to Borg depending on version (#394). 2022-02-09 16:54:35 -08:00
Dan Helfman b525e70e1c Run "compact" action by default when no actions are specified (#394). 2022-02-09 14:33:12 -08:00
Dan Helfman 4498671233 Remove references to removed long-deprecated options (#394). 2022-02-09 11:08:02 -08:00
Dan Helfman 9997aa9a92 Fix capitalization on compact help. 2022-02-08 15:58:09 -08:00
Dan Helfman 48f44d2f3d Add tests for compact action (#394). 2022-02-08 12:05:02 -08:00
Dan Helfman 9582324c88 Compact repository segments with new "borgmatic compact" action (#394). 2022-02-07 23:29:44 -08:00
Dan Helfman bec73245e9 Fix traceback when a YAML validation error occurs (#480, #482). 2022-01-19 20:39:03 -08:00
fabianschilling b39f08694d Merge branch 'master' into pr-working-directory 2022-01-05 09:30:27 +00:00
Dan Helfman 2ee75546f5 Add MongoDB database hook documentation. 2022-01-04 16:26:38 -08:00
Andrea Ghensi 7c6ce9399c fix integration tests and mongodb auth 2021-12-29 22:18:50 +01:00
Andrea Ghensi 6b7653484b Add mongodb dump hook 2021-12-26 01:00:58 +01:00
Fabian Schilling 2a80e48a92 Pass working directory to execute functions 2021-12-10 18:23:44 +01:00
Fabian Schilling 5821c6782e Add defaults to not set in schema 2021-12-10 18:23:08 +01:00
Fabian Schilling f15498f6d9 Add working_directory to borgmatic schema 2021-12-10 17:58:27 +01:00
Dan Helfman a1673d1fa1 Fix unicode error when restoring particular MySQL databases (#476). 2021-12-08 16:40:25 -08:00
Dan Helfman 2e99a1898c Fix f-string with missing expression. 2021-11-29 14:05:36 -08:00
Dan Helfman 0e8e9ced64 When command-line configuration override produces a parse error, error cleanly (#471). 2021-11-29 12:49:21 -08:00
nebulon42 c3e76585fc
move mysqldump options to the beginning of the command due to MySQL bug 30994. 2021-11-26 17:16:03 +01:00
Chen Yufei 0014b149f8 remove configuration_filename as it's already set. 2021-11-26 11:38:58 +08:00
Chen Yufei 091c07bbe2 Add context for various hooks. 2021-11-26 11:35:10 +08:00
Dan Helfman c646edf2c7 Bump version for release. 2021-11-22 13:19:15 -08:00
nebulon42 3729ba5ca3
add list_options setting, fixes #306 2021-11-20 15:43:58 +01:00
Dan Helfman 38ebfd2969 Rename retry_timeout to retry_wait and standardize log formatting (#28). 2021-11-15 11:51:17 -08:00
cadamswaite 976a877a25 Formatting 2021-11-14 22:37:42 +00:00
cadamswaite b4117916b8 Add timeout and tests 2021-11-14 22:15:22 +00:00
cadamswaite 6b182c9d2d Merge branch 'master' into master 2021-11-14 18:24:17 +00:00
Kim B. Heino 717cfd2d37 validate: add support for both jsonschema v3 and old v2 
RHEL8 and RHEL7 have old jsonschema v2. Try v3 (Draft7) first but
fallback to v2 (Draft4) if needed.
 2021-10-23 15:04:07 +03:00
Dan Helfman 449896f661 Fix error when configured source directories are not present on the filesystem at the time of backup (#387). 2021-10-11 10:40:10 -07:00
Dan Helfman fbbb096cec Note in documentation that borgmatic requires Python 3.6+. 2021-10-04 11:15:51 -07:00
Dan Helfman 77980511c6 Add another glob pattern example to exclude patterns. 2021-09-16 09:51:40 -07:00
Dan Helfman acb2ca79d9 Fix traceback that can occur when dumping a database (#440). 2021-08-06 08:58:11 -07:00
Dan Helfman 5890a1cb48 Fix "message too long" error when logging to rsyslog (#389). 2021-07-30 09:48:13 -07:00
Dan Helfman b3f5a9d18f Fix error when configuration file contains "umask" option (#437). 2021-07-27 10:04:22 -07:00
Dan Helfman 1f3907a6a5 Fix for failing PostgreSQL directory format test (#430). 2021-07-26 09:42:14 -07:00
Dan Helfman 1709f57ff0 Fix hang when restoring a PostgreSQL "tar" format database dump (#430). 2021-07-25 22:30:15 -07:00
cadamswaite 89baf757cf Sort imports 2021-07-14 23:17:35 +01:00
cadamswaite 4f36fe2b9f Run Black on changed file 2021-07-14 22:53:01 +01:00
cadamswaite 510449ce65 Change default retries to 0 2021-07-14 22:49:03 +01:00
cadamswaite 4cc4b8d484 Add queue based retry logic 2021-07-14 22:46:02 +01:00
Dan Helfman 54d57e1349 Add test for #407: Fix syslog logging on FreeBSD. 2021-06-23 10:21:45 -07:00
Dan Helfman 27d37b606b Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema). 2021-06-22 13:27:59 -07:00
Dan Helfman cf8882f2bc Run arbitrary Borg commands with new "borgmatic borg" action (#425). 2021-06-17 20:41:44 -07:00
Dan Helfman 9fd28d2eed Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)! 2021-06-08 11:43:55 -07:00
Dan Helfman 9016f4be43 Clarify that spaces in path names should not be backslashed in path names (#406). 2021-04-18 17:28:11 -07:00
Dan Helfman eb4b4cc92b Fix line length in schema. 2020-11-25 19:21:06 -08:00
Dan Helfman 41bf520585 Document that passphrase is used for Borg keyfile encryption, not just repokey encryption (#373). 2020-11-25 18:36:23 -08:00
Dan Helfman 9b819f32f8 Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config (#367). 2020-11-18 15:01:10 -08:00
Dan Helfman b619bde037 Fix broken end-to-end test. 2020-11-18 14:20:07 -08:00
François Poulain ebac02f118 Mention placeholders for repositories in documentation; fix #369 2020-11-18 20:22:41 +01:00
Dan Helfman 1c9ae81987 Fix signal forwarding from borgmatic to Borg resulting in recursion traceback (#368). 2020-11-18 10:29:03 -08:00
Dan Helfman 8aa7830f0d Fix broken "--override" action in Python 3.7 and below. 2020-11-15 13:39:15 -08:00
Dan Helfman 1ea04aedf0 Merge override values when specifying the "--override" flag multiple times (#361). 2020-10-23 16:06:00 -07:00
Diego Blanco 2d10e758e0 Fixed minor typo in help for "info" command 2020-10-12 00:04:15 +02:00
Dan Helfman 0e978299cf Fix traceback when a database hook value is null in a configuration file (#355). 2020-10-07 15:41:45 -07:00
root 7cad5a8608 Add `temporary_directory` as a configuration option defaulting to $TMPDIR 2020-08-26 15:29:02 +00:00
Matthias 631c3068a9 systemd security settings 2020-08-22 15:41:25 +02:00
networkjanitor 3c723e8d99 Merge branch 'master' into master 2020-08-12 21:11:45 +00:00
networkjanitor c5776447b9 fixed description strings for before_extract and after_extract 2020-08-12 23:07:57 +02:00
Dan Helfman 5356f487a5 Move before/after_everything hooks so they're easier to find. 2020-08-12 13:46:31 -07:00
Dan Helfman 72bd96c656 Fix traceback when a configuration directory is non-readable due to directory permissions (#350). 2020-08-12 11:32:00 -07:00
Jakub Duchateau 95c415f416
Change "before" to "after" in desc 2020-08-07 16:41:35 +02:00
Dan Helfman 06dc336481 Fix schema exclude patterns example path. 2020-08-06 09:59:48 -07:00
networkjanitor 893fca2816 Add before_extract and after_extract hooks 2020-08-01 16:08:32 +02:00
Dan Helfman b3fd1be5f6 Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300). 2020-07-22 21:53:55 -07:00
Dan Helfman a23083f737 Clarify schema documentation about skipping pruning. 2020-07-22 10:35:16 -07:00
Dan Helfman 0a659a397f Attempt to fix no-repro test failure of log_outputs() (#339). 2020-07-07 22:05:53 -07:00
Dan Helfman 3aaa89fb08 Formatting. 2020-06-25 20:25:29 -07:00
Dan Helfman 35d542a676 Fix for traceback when running Cronitor, Cronhub, and PagerDuty monitor hooks (#336). 2020-06-25 20:23:25 -07:00
Dan Helfman 37cc229749 Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328). 2020-06-23 11:01:03 -07:00
Dan Helfman 17c2d109e5 Add tests for pass-through of BORG_* environment variables. 2020-06-21 14:41:22 -07:00
Dan Helfman c8d5de2179 Fix broken pass-through of BORG_* environment variables to Borg (#327). 
Reviewed-on: witten/borgmatic#327
 2020-06-21 21:29:59 +00:00
Dan Helfman f5ebca4907 Add SSL support to PostgreSQL database configuration (#331). 
Reviewed-on: witten/borgmatic#331
 2020-06-20 21:24:14 +00:00
Edward Shornock 01db676d68 Change the example for the ssl_mode parameter 2020-06-20 23:32:24 +03:00
Dan Helfman 27cbe9dfc0 Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333). 2020-06-19 20:16:38 -07:00
Edward Shornock 8fb830099f Re-add the ilbpq-ssl documentation URL to the schema 
It's been moved from describing `ssl_mode` to the general
postgresql_database description key.
 2020-06-19 13:22:39 +03:00
Edward Shornock 463a133a63 Ensure schema lines are less than 80 characters in length 2020-06-19 13:22:39 +03:00
Edward Shornock a16fed8887 Rename PostgreSQL SSL config variables 
e.g. s/sslmode/ssl_mode/g to conform with borgmatic naming conventions.
 2020-06-19 13:20:14 +03:00
Edward Shornock 33113890f5 Reduce duplication with a common function 2020-06-19 12:32:36 +03:00
Edward Shornock abd47fc14e Add SSL support to PostgreSQL hooks 2020-06-19 02:19:17 +03:00
Dan Helfman 7fb4061759 Improve configuration reference documentation readability via more aggressive word-wrapping in configuration schema descriptions. 2020-06-17 23:15:12 -07:00
Ralph Heinkel a12a1121b6 Use values from BORG_* env variables if they are not specified in config.yaml 2020-06-15 19:50:11 +02:00
Dan Helfman aa14449857 Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive (#324). 2020-06-06 14:57:14 -07:00
Dan Helfman ed7b1cd3d7 Add some no-cover pragmas on functions that don't need tests. 2020-06-06 14:33:06 -07:00
Dan Helfman a155eefa23 Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files (#323). 2020-06-06 14:30:04 -07:00
Dan Helfman 398665be9e Allow before_backup and similiar hooks to exit with a soft failure without altering the monitoring status (#292). 2020-06-02 14:33:41 -07:00
Dan Helfman d7277893fb Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316). 2020-06-02 12:40:32 -07:00
Dan Helfman 00033bf0a8 Tweak comment indentation in generated configuration file for clarity. 2020-06-02 11:37:13 -07:00
Dan Helfman 097a09578a Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315). 2020-05-26 09:20:14 -07:00
Dan Helfman 65472c8de2 Fix error message when there are no MySQL databases to dump for "all" databases (#319). 2020-05-26 08:59:04 -07:00
Dan Helfman 96df52ec50 Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first (#316). 2020-05-20 13:33:53 -07:00
Dan Helfman 89cb5eb76d Fix regression in support for PostgreSQL's "directory" dump format (#314). 2020-05-18 11:31:29 -07:00
Dan Helfman 2ddf38f99c Fix error handling when executing commands to handle more edge cases. 2020-05-14 23:04:01 -07:00
Dan Helfman d88f321cef Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310). 2020-05-14 22:38:38 -07:00
Dan Helfman 15ea70a71b Flesh out missing test coverage for execute.py. 2020-05-12 15:49:49 -07:00
Dan Helfman 3bcef72050 Add some missing test mocks that were masking lack of unit coverage. 2020-05-11 11:17:24 -07:00
Dan Helfman 695c764a01 Merge log output functions into one. 2020-05-11 10:55:50 -07:00
Dan Helfman f7c93ea2e8 Wait for process to finish before trying to check exit status. 2020-05-09 23:09:48 -07:00
Dan Helfman 1ea047dd94 Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores. 2020-05-09 21:53:16 -07:00
Dan Helfman 4b523f9e2c Make database restore output only show at verbosity 2. 2020-05-08 19:38:33 -07:00
Dan Helfman 6a61070d85 Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe. 2020-05-08 19:11:26 -07:00
Dan Helfman 1ba996ad93 Additional test coverage. 2020-05-07 12:14:27 -07:00
Dan Helfman a23fdf946d Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258). 2020-05-07 11:44:04 -07:00
Dan Helfman 12cf6913ef Remove unused function parameter. 2020-05-03 14:07:34 -07:00
Dan Helfman dce1928dc4
Fix PostgreSQL restore error on "all" database dump. 2020-04-24 15:50:33 -07:00
Nathan Beals 3c8dc4929f Added `test_restore_all_database_dump` unit test. 
Updated the other unit tests, as I had to re-arrange argument order
Added an 'all' test for the postgres end-to-end test.

Ran black formatter on it all.
 2020-04-24 18:32:53 -04:00
Dan Helfman e511014a28 Fix MySQL restore error on "all" database dump by excluding system tables (#301). 2020-04-22 12:17:22 -07:00
Nathan Beals f6407bafcb Remove the `--create` flag, was causing an error 2020-04-10 11:24:13 -04:00
Nathan Beals d5e9f67cec Finished. Now uses 'psql' to run the plain-text scripts that pg_dumpall creates 2020-04-10 10:55:53 -04:00
Nathan Beals b14f371c05 First attempt at fixing this pg_dumpall/restoring issue 2020-04-10 09:20:00 -04:00
Dan Helfman d6562c4b1e Fix the "--stats" and "--files" flags so that they yield output at verbosity 0 (#290). 2020-01-31 10:23:36 -08:00
Dan Helfman 55141bda67 Specify "--archive latest" to all actions that accept an archive (#289). 2020-01-29 16:59:02 -08:00
Dan Helfman bc02c123e6 Monitor backups with PagerDuty hook integration (#245). 2020-01-27 15:32:09 -08:00