Commit Graph

353 Commits

Author SHA1 Message Date
Dan Helfman bcad0de1a4
Add verify_tls option for Healthchecks to optionally disable TLS verification. 2022-07-23 23:11:41 -07:00
Uli 5c6407047f feat: add verify_tls flag for Healthchecks 2022-07-24 07:37:00 +02:00
Dan Helfman 6ddae20fa1 Fix handling of "repository" and "data" consistency checks to prevent invalid Borg flags (#565). 2022-07-23 21:02:21 -07:00
Dan Helfman 16066942e3 Fix traceback with "create" action and "--json" flag when a database hook is configured (#563). 2022-07-19 10:25:10 -07:00
Dan Helfman e85d551eac Fix all database hooks to error when the requested database to restore isn't present in the Borg archive (#560). 2022-07-06 23:21:24 -07:00
Dan Helfman 2b23a63a08 Add end-to-end test for overrides. 2022-07-06 18:20:51 -07:00
Dan Helfman 457ed80744 Fix environment variable plumbing so options in one configuration file aren't used for others (#555). 2022-06-30 13:42:17 -07:00
Dan Helfman 10723efc68 Fix all monitoring hooks to warn if the server returns an HTTP 4xx error (#554). 2022-06-29 21:19:40 -07:00
Dan Helfman bd4d109009 Fix logging to include the full traceback when Borg experiences an internal error (#553). 2022-06-28 13:38:24 -07:00
Dan Helfman d372a86fe6 Code formatting. 2022-06-23 10:41:04 -07:00
Sébastien MB b63c854509 Fix escaped environment variable in configuration
- when an env variable is escaped in the configuration file, we expect
  not to resolve it and remove the escape char `\`
2022-06-17 09:50:56 +02:00
Dan Helfman aecb6fcd74 Code style, rename command-line flag, and move new code into its own file (#546) 2022-06-16 11:35:24 -07:00
Sébastien MB 97b5cd089d Allow environment variable resolution in configuration file
- all string fields containing an environment variable like ${FOO} will
  be resolved
- supported format ${FOO}, ${FOO:-bar} and ${FOO-bar} to allow default
  values if variable is not present in environment
- add --no-env argument for CLI to disable the feature which is enabled
  by default

Resolves: #546
2022-06-16 18:52:54 +02:00
Gavin Chappell a8b8d507b6
add a hook for sending push notifications via ntfy.sh 2022-06-09 21:10:38 +01:00
Dan Helfman 3561c93d74 Fix Healthchecks tests that leak global state, breaking downstream tests (discovered in #543). 2022-06-09 11:05:44 -07:00
Dan Helfman 9aefb5179f Fix None find paths (#541). 2022-06-03 15:20:05 -07:00
Dan Helfman d14f22e121 Add "borgmatic list --find" flag for searching for files across multiple archives (#541). 2022-06-03 15:12:14 -07:00
Dan Helfman 80ec3e7d97 Deprecate "borgmatic list --successful" flag, as listing only non-checkpoint (successful) archives is now the default in newer versions of Borg. 2022-06-02 20:35:39 -07:00
Dan Helfman 2bc91ac3d2 Add "generate-borgmatic-config --overwrite" flag to replace an existing destination file (#539). 2022-05-29 16:03:55 -07:00
Dan Helfman 5b615d51a4 Add support for "borgmatic borg debug" command (#538). 2022-05-29 15:43:03 -07:00
Dan Helfman 8fa90053cf Add "borgmatic check --force" flag to ignore configured check frequencies (#523). 2022-05-28 19:29:33 -07:00
Dan Helfman ad0e2e0d7c Tweak default check frequency to 1 month (#523). 2022-05-28 15:49:50 -07:00
Dan Helfman e76bfa555f Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523). 2022-05-28 14:42:19 -07:00
Dan Helfman 8ddb7268eb Reuse "borg info" function. 2022-05-27 13:51:11 -07:00
Dan Helfman cb5fe02ebd Fix broken Bash completion end-to-end test. 2022-05-26 11:18:46 -07:00
Dan Helfman 77b84f8a48 Add Bash completion script so you can tab-complete the borgmatic command-line. 2022-05-26 10:27:53 -07:00
Dan Helfman be59a3e574 Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options (#536). 2022-05-25 10:26:26 -07:00
Dan Helfman 54933ebef5 Change connection failures for monitoring hooks to be warnings instead of errors (#439). 2022-05-24 15:50:04 -07:00
Dan Helfman 157e59ac88 Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460). 2022-05-24 14:44:33 -07:00
Dan Helfman 8b179e4647 Reverse logic of Healtchecks "skip_states" option to just "states" (#525). 2022-05-24 14:09:42 -07:00
Dan Helfman 865eff7d98 Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525). 2022-05-24 13:59:28 -07:00
Dan Helfman b9741f4d0b Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294). 2022-05-24 12:23:38 -07:00
Dan Helfman 02781662f8 Change monitoring hooks to specify the ping URL / integration key as a named option. 2022-05-23 20:02:10 -07:00
Dan Helfman 32a1043468 Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402). 2022-05-23 16:11:24 -07:00
Dan Helfman 3e4aeec649 Warn when an unsupported variable is used in a hook command (#420). 2022-05-23 15:27:54 -07:00
Dan Helfman 255cc6ec23 When deep merging common configuration, merge colliding list values by appending them (#531). 2022-05-20 15:28:28 -07:00
Dan Helfman 788281cfb9 When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including (#532). 2022-05-19 17:15:05 -07:00
Dan Helfman 525266ede6 Deep merging when including common configuration (#381). 2022-04-25 21:18:37 -07:00
Dan Helfman 0e6b425ac5 Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515). 2022-04-23 14:03:15 -07:00
Dan Helfman ed7fe5c6d0 Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473). 2022-04-21 22:08:25 -07:00
Dan Helfman 0c027a3050 Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516). 2022-04-03 13:12:48 -07:00
Dan Helfman 9f44bbad65 Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517). 2022-04-02 22:28:41 -07:00
Dan Helfman 976516d0e1 When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444). 2022-03-08 10:19:36 -08:00
Dan Helfman 574eb91921 Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507). 2022-03-07 21:46:12 -08:00
Dan Helfman 28fef3264b Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486). 2022-03-07 15:32:07 -08:00
Dan Helfman 9161dbcb7d Removing unnecessary leading underscores from functions. 2022-03-07 11:58:29 -08:00
Dan Helfman 4b3027e4fc Add test for new working_directory option (#431). 2022-03-03 11:48:18 -08:00
Dan Helfman 0eb2634f9b Working directory option to support source directories with relative paths (#431).
Reviewed-on: borgmatic-collective/borgmatic#477
2022-03-03 19:28:17 +00:00
Dan Helfman 9317cbaaf0 Code formatting. 2022-02-10 10:23:34 -08:00
Dan Helfman 1b5f04b79f When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:16:09 -08:00
Dan Helfman 948c86f62c When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 10:09:18 -08:00
Dan Helfman 7e7209322a When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-10 09:51:13 -08:00
Dan Helfman 00a57fd947 Code formatting. 2022-02-09 21:20:28 -08:00
Dan Helfman 6bf6ac310b When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394). 2022-02-09 21:11:00 -08:00
Dan Helfman 4b5af2770d When the "atime" option is used, tailor the flags passed to Borg depending on version (#394). 2022-02-09 16:54:35 -08:00
Dan Helfman b525e70e1c Run "compact" action by default when no actions are specified (#394). 2022-02-09 14:33:12 -08:00
Dan Helfman e3f4bf0293 Build fix for ruamel.yaml.clib error. 2022-02-08 12:52:45 -08:00
Dan Helfman 48f44d2f3d Add tests for compact action (#394). 2022-02-08 12:05:02 -08:00
Dan Helfman bec73245e9 Fix traceback when a YAML validation error occurs (#480, #482). 2022-01-19 20:39:03 -08:00
fabianschilling b39f08694d Merge branch 'master' into pr-working-directory 2022-01-05 09:30:27 +00:00
Dan Helfman 2ee75546f5 Add MongoDB database hook documentation. 2022-01-04 16:26:38 -08:00
Andrea Ghensi 7c6ce9399c fix integration tests and mongodb auth 2021-12-29 22:18:50 +01:00
Andrea Ghensi 6b7653484b Add mongodb dump hook 2021-12-26 01:00:58 +01:00
Fabian Schilling 85e0334826 Add missing working_directory arg to pass tests 2021-12-10 18:24:41 +01:00
Dan Helfman a1673d1fa1 Fix unicode error when restoring particular MySQL databases (#476). 2021-12-08 16:40:25 -08:00
Dan Helfman 7a086d8430 Fix import ordering. 2021-11-29 14:00:14 -08:00
Dan Helfman 0e8e9ced64 When command-line configuration override produces a parse error, error cleanly (#471). 2021-11-29 12:49:21 -08:00
nebulon42 c3e76585fc
move mysqldump options to the beginning of the command due to MySQL bug 30994. 2021-11-26 17:16:03 +01:00
nebulon42 3729ba5ca3
add list_options setting, fixes #306 2021-11-20 15:43:58 +01:00
Dan Helfman 38ebfd2969 Rename retry_timeout to retry_wait and standardize log formatting (#28). 2021-11-15 11:51:17 -08:00
cadamswaite 976a877a25 Formatting 2021-11-14 22:37:42 +00:00
cadamswaite b4117916b8 Add timeout and tests 2021-11-14 22:15:22 +00:00
cadamswaite 19cad89978 Add some tests for retry logic 2021-11-14 21:35:23 +00:00
Dan Helfman 449896f661 Fix error when configured source directories are not present on the filesystem at the time of backup (#387). 2021-10-11 10:40:10 -07:00
Dan Helfman acb2ca79d9 Fix traceback that can occur when dumping a database (#440). 2021-08-06 08:58:11 -07:00
Dan Helfman 80b33fbf8a Code style reformatting. 2021-07-27 09:39:48 -07:00
Marek Szuba e8b8d86592 tests/integration/test_execute: use plain Python rather than xxd
Removes this test's dependencies on vim and /dev/urandom.

Signed-off-by: Marek Szuba <marek.szuba@cern.ch>
2021-07-27 13:50:16 +01:00
Dan Helfman 2a8692c64f Fix integration test to hopefully work on Alpine (#430). 2021-07-25 22:50:00 -07:00
Dan Helfman 1709f57ff0 Fix hang when restoring a PostgreSQL "tar" format database dump (#430). 2021-07-25 22:30:15 -07:00
Dan Helfman 54d57e1349 Add test for #407: Fix syslog logging on FreeBSD. 2021-06-23 10:21:45 -07:00
Dan Helfman 27d37b606b Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema). 2021-06-22 13:27:59 -07:00
Dan Helfman cf8882f2bc Run arbitrary Borg commands with new "borgmatic borg" action (#425). 2021-06-17 20:41:44 -07:00
Dan Helfman 9fd28d2eed Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)! 2021-06-08 11:43:55 -07:00
Dan Helfman d6e376d32d Fix end-to-end test broken by change in source directory examples. 2021-04-18 17:54:54 -07:00
Dan Helfman d543109ef4 "Fix" build failure with Alpine Edge by switching from Edge to Alpine 3.13. 2021-04-09 15:58:23 -07:00
Dan Helfman 4897a78fd3 Fix database tests broken by PostgreSQL upgrade in Alpine Edge. 2020-12-24 22:23:09 -08:00
Dan Helfman ad7198ba66 Tweak to test failing on some machines. 2020-11-26 16:22:42 -08:00
Dan Helfman c0ae01f5d5 Code formatting. 2020-11-25 17:46:57 -08:00
Dan Helfman ccd1627175 Fix timing-related test error in Alpine Edge. 2020-11-25 15:48:33 -08:00
Dan Helfman 1f4f28b4dc Drop support for Python 3.5. Only support black code formatter on Python 3.8+. 2020-11-22 17:27:21 -08:00
Dan Helfman ea6cd53067 Update versions of test dependencies (test_requirements.txt and test containers). 2020-11-22 14:48:07 -08:00
Dan Helfman 9b819f32f8 Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config (#367). 2020-11-18 15:01:10 -08:00
Dan Helfman b619bde037 Fix broken end-to-end test. 2020-11-18 14:20:07 -08:00
François Poulain ebac02f118 Mention placeholders for repositories in documentation; fix #369 2020-11-18 20:22:41 +01:00
Dan Helfman 1ea04aedf0 Merge override values when specifying the "--override" flag multiple times (#361). 2020-10-23 16:06:00 -07:00
Dan Helfman 0e978299cf Fix traceback when a database hook value is null in a configuration file (#355). 2020-10-07 15:41:45 -07:00
networkjanitor 3c723e8d99 Merge branch 'master' into master 2020-08-12 21:11:45 +00:00
Dan Helfman 72bd96c656 Fix traceback when a configuration directory is non-readable due to directory permissions (#350). 2020-08-12 11:32:00 -07:00
networkjanitor 893fca2816 Add before_extract and after_extract hooks 2020-08-01 16:08:32 +02:00
Dan Helfman b3fd1be5f6 Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300). 2020-07-22 21:53:55 -07:00