554 Commits

Author	SHA1	Message	Date
Dan Helfman	6629f40cab	In bash completion script, warn when script is out of date using script contents instead of version. (Fewer spurious warnings that way.)	2022-05-28 15:27:11 -07:00
Dan Helfman	e76bfa555f	Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523).	2022-05-28 14:42:19 -07:00
Dan Helfman	8ddb7268eb	Reuse "borg info" function.	2022-05-27 13:51:11 -07:00
Dan Helfman	77b84f8a48	Add Bash completion script so you can tab-complete the borgmatic command-line.	2022-05-26 10:27:53 -07:00
Dan Helfman	be59a3e574	Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options (#536).	2022-05-25 10:26:26 -07:00
polyzen	8708ca07f4	Remove trailing whitespace	2022-05-25 00:43:40 +00:00
Dan Helfman	54933ebef5	Change connection failures for monitoring hooks to be warnings instead of errors (#439).	2022-05-24 15:50:04 -07:00
Dan Helfman	157e59ac88	Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460).	2022-05-24 14:44:33 -07:00
Dan Helfman	666f0dd751	Add missing Healthchecks "states" option example in configuration schema (#525).	2022-05-24 14:17:19 -07:00
Dan Helfman	8b179e4647	Reverse logic of Healtchecks "skip_states" option to just "states" (#525).	2022-05-24 14:09:42 -07:00
Dan Helfman	865eff7d98	Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525).	2022-05-24 13:59:28 -07:00
Dan Helfman	b9741f4d0b	Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294).	2022-05-24 12:23:38 -07:00
Dan Helfman	02781662f8	Change monitoring hooks to specify the ping URL / integration key as a named option.	2022-05-23 20:02:10 -07:00
Dan Helfman	32a1043468	Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402).	2022-05-23 16:11:24 -07:00
Dan Helfman	3e4aeec649	Warn when an unsupported variable is used in a hook command (#420).	2022-05-23 15:27:54 -07:00
Dan Helfman	b98b827594	Remove stale comment.	2022-05-23 10:59:56 -07:00
Dan Helfman	255cc6ec23	When deep merging common configuration, merge colliding list values by appending them (#531).	2022-05-20 15:28:28 -07:00
Dan Helfman	51fc37d57a	Improve the error message when a configuration override contains an invalid value (#528).	2022-05-20 13:38:53 -07:00
Dan Helfman	788281cfb9	When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including (#532).	2022-05-19 17:15:05 -07:00
Dan Helfman	525266ede6	Deep merging when including common configuration (#381).	2022-04-25 21:18:37 -07:00
Dan Helfman	0e6b425ac5	Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515).	2022-04-23 14:03:15 -07:00
Dan Helfman	ed7fe5c6d0	Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473).	2022-04-21 22:08:25 -07:00
Dan Helfman	cbce6707f4	Clarify one_file_system behavior in schema comment (#520).	2022-04-12 11:05:22 -07:00
Dan Helfman	e40e726687	Change Healthchecks logs truncation size from 10k bytes to 100k bytes, corresponding to that same change on Healthchecks.io.	2022-04-06 22:00:18 -07:00
Dan Helfman	0c027a3050	Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516).	2022-04-03 13:12:48 -07:00
Dan Helfman	9f44bbad65	Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517).	2022-04-02 22:28:41 -07:00
Dan Helfman	db7499db82	Document "repositories" context to for "before_*" and "after_*" command action hooks (#469).	2022-03-14 13:34:14 -07:00
Dan Helfman	6b500c2a8b	Add repositories context for command hooks. ... Reviewed-on: #469	2022-03-14 20:13:15 +00:00
Dan Helfman	976516d0e1	When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444).	2022-03-08 10:19:36 -08:00
Dan Helfman	574eb91921	Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507).	2022-03-07 21:46:12 -08:00
Dan Helfman	28fef3264b	Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486).	2022-03-07 15:32:07 -08:00
Dan Helfman	9161dbcb7d	Removing unnecessary leading underscores from functions.	2022-03-07 11:58:29 -08:00
Dan Helfman	0eb2634f9b	Working directory option to support source directories with relative paths (#431). ... Reviewed-on: #477	2022-03-03 19:28:17 +00:00
Dan Helfman	9317cbaaf0	Code formatting.	2022-02-10 10:23:34 -08:00
Dan Helfman	1b5f04b79f	When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 10:16:09 -08:00
Dan Helfman	948c86f62c	When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 10:09:18 -08:00
Dan Helfman	7e7209322a	When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 09:51:13 -08:00
Dan Helfman	00a57fd947	Code formatting.	2022-02-09 21:20:28 -08:00
Dan Helfman	6bf6ac310b	When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-09 21:11:00 -08:00
Dan Helfman	4b5af2770d	When the "atime" option is used, tailor the flags passed to Borg depending on version (#394).	2022-02-09 16:54:35 -08:00
Dan Helfman	b525e70e1c	Run "compact" action by default when no actions are specified (#394).	2022-02-09 14:33:12 -08:00
Dan Helfman	4498671233	Remove references to removed long-deprecated options (#394).	2022-02-09 11:08:02 -08:00
Dan Helfman	9997aa9a92	Fix capitalization on compact help.	2022-02-08 15:58:09 -08:00
Dan Helfman	48f44d2f3d	Add tests for compact action (#394).	2022-02-08 12:05:02 -08:00
Dan Helfman	9582324c88	Compact repository segments with new "borgmatic compact" action (#394).	2022-02-07 23:29:44 -08:00
Dan Helfman	bec73245e9	Fix traceback when a YAML validation error occurs (#480, #482).	2022-01-19 20:39:03 -08:00
fabianschilling	b39f08694d	Merge branch 'master' into pr-working-directory	2022-01-05 09:30:27 +00:00
Dan Helfman	2ee75546f5	Add MongoDB database hook documentation.	2022-01-04 16:26:38 -08:00
Andrea Ghensi	7c6ce9399c	fix integration tests and mongodb auth	2021-12-29 22:18:50 +01:00
Andrea Ghensi	6b7653484b	Add mongodb dump hook	2021-12-26 01:00:58 +01:00
Fabian Schilling	2a80e48a92	Pass working directory to execute functions	2021-12-10 18:23:44 +01:00
Fabian Schilling	5821c6782e	Add defaults to not set in schema	2021-12-10 18:23:08 +01:00
Fabian Schilling	f15498f6d9	Add working_directory to borgmatic schema	2021-12-10 17:58:27 +01:00
Dan Helfman	a1673d1fa1	Fix unicode error when restoring particular MySQL databases (#476).	2021-12-08 16:40:25 -08:00
Dan Helfman	2e99a1898c	Fix f-string with missing expression.	2021-11-29 14:05:36 -08:00
Dan Helfman	0e8e9ced64	When command-line configuration override produces a parse error, error cleanly (#471).	2021-11-29 12:49:21 -08:00
nebulon42	c3e76585fc	move mysqldump options to the beginning of the command due to MySQL bug 30994.	2021-11-26 17:16:03 +01:00
Chen Yufei	0014b149f8	remove configuration_filename as it's already set.	2021-11-26 11:38:58 +08:00
Chen Yufei	091c07bbe2	Add context for various hooks.	2021-11-26 11:35:10 +08:00
Dan Helfman	c646edf2c7	Bump version for release.	2021-11-22 13:19:15 -08:00
nebulon42	3729ba5ca3	add list_options setting, fixes #306	2021-11-20 15:43:58 +01:00
Dan Helfman	38ebfd2969	Rename retry_timeout to retry_wait and standardize log formatting (#28).	2021-11-15 11:51:17 -08:00
cadamswaite	976a877a25	Formatting	2021-11-14 22:37:42 +00:00
cadamswaite	b4117916b8	Add timeout and tests	2021-11-14 22:15:22 +00:00
cadamswaite	6b182c9d2d	Merge branch 'master' into master	2021-11-14 18:24:17 +00:00
Kim B. Heino	717cfd2d37	validate: add support for both jsonschema v3 and old v2 ... RHEL8 and RHEL7 have old jsonschema v2. Try v3 (Draft7) first but fallback to v2 (Draft4) if needed.	2021-10-23 15:04:07 +03:00
Dan Helfman	449896f661	Fix error when configured source directories are not present on the filesystem at the time of backup (#387).	2021-10-11 10:40:10 -07:00
Dan Helfman	fbbb096cec	Note in documentation that borgmatic requires Python 3.6+.	2021-10-04 11:15:51 -07:00
Dan Helfman	77980511c6	Add another glob pattern example to exclude patterns.	2021-09-16 09:51:40 -07:00
Dan Helfman	acb2ca79d9	Fix traceback that can occur when dumping a database (#440).	2021-08-06 08:58:11 -07:00
Dan Helfman	5890a1cb48	Fix "message too long" error when logging to rsyslog (#389).	2021-07-30 09:48:13 -07:00
Dan Helfman	b3f5a9d18f	Fix error when configuration file contains "umask" option (#437).	2021-07-27 10:04:22 -07:00
Dan Helfman	1f3907a6a5	Fix for failing PostgreSQL directory format test (#430).	2021-07-26 09:42:14 -07:00
Dan Helfman	1709f57ff0	Fix hang when restoring a PostgreSQL "tar" format database dump (#430).	2021-07-25 22:30:15 -07:00
cadamswaite	89baf757cf	Sort imports	2021-07-14 23:17:35 +01:00
cadamswaite	4f36fe2b9f	Run Black on changed file	2021-07-14 22:53:01 +01:00
cadamswaite	510449ce65	Change default retries to 0	2021-07-14 22:49:03 +01:00
cadamswaite	4cc4b8d484	Add queue based retry logic	2021-07-14 22:46:02 +01:00
Dan Helfman	54d57e1349	Add test for #407: Fix syslog logging on FreeBSD.	2021-06-23 10:21:45 -07:00
Dan Helfman	27d37b606b	Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema).	2021-06-22 13:27:59 -07:00
Dan Helfman	cf8882f2bc	Run arbitrary Borg commands with new "borgmatic borg" action (#425).	2021-06-17 20:41:44 -07:00
Dan Helfman	9fd28d2eed	Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)!	2021-06-08 11:43:55 -07:00
Dan Helfman	9016f4be43	Clarify that spaces in path names should not be backslashed in path names (#406).	2021-04-18 17:28:11 -07:00
Dan Helfman	eb4b4cc92b	Fix line length in schema.	2020-11-25 19:21:06 -08:00
Dan Helfman	41bf520585	Document that passphrase is used for Borg keyfile encryption, not just repokey encryption (#373).	2020-11-25 18:36:23 -08:00
Dan Helfman	9b819f32f8	Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config (#367).	2020-11-18 15:01:10 -08:00
Dan Helfman	b619bde037	Fix broken end-to-end test.	2020-11-18 14:20:07 -08:00
François Poulain	ebac02f118	Mention placeholders for repositories in documentation; fix #369	2020-11-18 20:22:41 +01:00
Dan Helfman	1c9ae81987	Fix signal forwarding from borgmatic to Borg resulting in recursion traceback (#368).	2020-11-18 10:29:03 -08:00
Dan Helfman	8aa7830f0d	Fix broken "--override" action in Python 3.7 and below.	2020-11-15 13:39:15 -08:00
Dan Helfman	1ea04aedf0	Merge override values when specifying the "--override" flag multiple times (#361).	2020-10-23 16:06:00 -07:00
Diego Blanco	2d10e758e0	Fixed minor typo in help for "info" command	2020-10-12 00:04:15 +02:00
Dan Helfman	0e978299cf	Fix traceback when a database hook value is null in a configuration file (#355).	2020-10-07 15:41:45 -07:00
root	7cad5a8608	Add temporary_directory as a configuration option defaulting to $TMPDIR	2020-08-26 15:29:02 +00:00
palto42	631c3068a9	systemd security settings	2020-08-22 15:41:25 +02:00
networkjanitor	3c723e8d99	Merge branch 'master' into master	2020-08-12 21:11:45 +00:00
networkjanitor	c5776447b9	fixed description strings for before_extract and after_extract	2020-08-12 23:07:57 +02:00
Dan Helfman	5356f487a5	Move before/after_everything hooks so they're easier to find.	2020-08-12 13:46:31 -07:00
Dan Helfman	72bd96c656	Fix traceback when a configuration directory is non-readable due to directory permissions (#350).	2020-08-12 11:32:00 -07:00
Jakub Duchateau	95c415f416	Change "before" to "after" in desc	2020-08-07 16:41:35 +02:00
Dan Helfman	06dc336481	Fix schema exclude patterns example path.	2020-08-06 09:59:48 -07:00
networkjanitor	893fca2816	Add before_extract and after_extract hooks	2020-08-01 16:08:32 +02:00
Dan Helfman	b3fd1be5f6	Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300).	2020-07-22 21:53:55 -07:00
Dan Helfman	a23083f737	Clarify schema documentation about skipping pruning.	2020-07-22 10:35:16 -07:00
Dan Helfman	0a659a397f	Attempt to fix no-repro test failure of log_outputs() (#339).	2020-07-07 22:05:53 -07:00
Dan Helfman	3aaa89fb08	Formatting.	2020-06-25 20:25:29 -07:00
Dan Helfman	35d542a676	Fix for traceback when running Cronitor, Cronhub, and PagerDuty monitor hooks (#336).	2020-06-25 20:23:25 -07:00
Dan Helfman	37cc229749	Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328).	2020-06-23 11:01:03 -07:00
Dan Helfman	17c2d109e5	Add tests for pass-through of BORG_* environment variables.	2020-06-21 14:41:22 -07:00
Dan Helfman	c8d5de2179	Fix broken pass-through of BORG_* environment variables to Borg (#327). ... Reviewed-on: witten/borgmatic#327	2020-06-21 21:29:59 +00:00
Dan Helfman	f5ebca4907	Add SSL support to PostgreSQL database configuration (#331). ... Reviewed-on: witten/borgmatic#331	2020-06-20 21:24:14 +00:00
Edward Shornock	01db676d68	Change the example for the ssl_mode parameter	2020-06-20 23:32:24 +03:00
Dan Helfman	27cbe9dfc0	Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333).	2020-06-19 20:16:38 -07:00
Edward Shornock	8fb830099f	Re-add the ilbpq-ssl documentation URL to the schema ... It's been moved from describing `ssl_mode` to the general postgresql_database description key.	2020-06-19 13:22:39 +03:00
Edward Shornock	463a133a63	Ensure schema lines are less than 80 characters in length	2020-06-19 13:22:39 +03:00
Edward Shornock	a16fed8887	Rename PostgreSQL SSL config variables ... e.g. s/sslmode/ssl_mode/g to conform with borgmatic naming conventions.	2020-06-19 13:20:14 +03:00
Edward Shornock	33113890f5	Reduce duplication with a common function	2020-06-19 12:32:36 +03:00
Edward Shornock	abd47fc14e	Add SSL support to PostgreSQL hooks	2020-06-19 02:19:17 +03:00
Dan Helfman	7fb4061759	Improve configuration reference documentation readability via more aggressive word-wrapping in configuration schema descriptions.	2020-06-17 23:15:12 -07:00
Ralph Heinkel	a12a1121b6	Use values from BORG_* env variables if they are not specified in config.yaml	2020-06-15 19:50:11 +02:00
Dan Helfman	aa14449857	Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive (#324).	2020-06-06 14:57:14 -07:00
Dan Helfman	ed7b1cd3d7	Add some no-cover pragmas on functions that don't need tests.	2020-06-06 14:33:06 -07:00
Dan Helfman	a155eefa23	Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files (#323).	2020-06-06 14:30:04 -07:00
Dan Helfman	398665be9e	Allow before_backup and similiar hooks to exit with a soft failure without altering the monitoring status (#292).	2020-06-02 14:33:41 -07:00
Dan Helfman	d7277893fb	Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316).	2020-06-02 12:40:32 -07:00
Dan Helfman	00033bf0a8	Tweak comment indentation in generated configuration file for clarity.	2020-06-02 11:37:13 -07:00
Dan Helfman	097a09578a	Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315).	2020-05-26 09:20:14 -07:00
Dan Helfman	65472c8de2	Fix error message when there are no MySQL databases to dump for "all" databases (#319).	2020-05-26 08:59:04 -07:00
Dan Helfman	96df52ec50	Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first (#316).	2020-05-20 13:33:53 -07:00
Dan Helfman	89cb5eb76d	Fix regression in support for PostgreSQL's "directory" dump format (#314).	2020-05-18 11:31:29 -07:00
Dan Helfman	2ddf38f99c	Fix error handling when executing commands to handle more edge cases.	2020-05-14 23:04:01 -07:00
Dan Helfman	d88f321cef	Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310).	2020-05-14 22:38:38 -07:00
Dan Helfman	15ea70a71b	Flesh out missing test coverage for execute.py.	2020-05-12 15:49:49 -07:00
Dan Helfman	3bcef72050	Add some missing test mocks that were masking lack of unit coverage.	2020-05-11 11:17:24 -07:00
Dan Helfman	695c764a01	Merge log output functions into one.	2020-05-11 10:55:50 -07:00
Dan Helfman	f7c93ea2e8	Wait for process to finish before trying to check exit status.	2020-05-09 23:09:48 -07:00
Dan Helfman	1ea047dd94	Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores.	2020-05-09 21:53:16 -07:00
Dan Helfman	4b523f9e2c	Make database restore output only show at verbosity 2.	2020-05-08 19:38:33 -07:00
Dan Helfman	6a61070d85	Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe.	2020-05-08 19:11:26 -07:00
Dan Helfman	1ba996ad93	Additional test coverage.	2020-05-07 12:14:27 -07:00
Dan Helfman	a23fdf946d	Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258).	2020-05-07 11:44:04 -07:00
Dan Helfman	12cf6913ef	Remove unused function parameter.	2020-05-03 14:07:34 -07:00
Dan Helfman	dce1928dc4	Fix PostgreSQL restore error on "all" database dump.	2020-04-24 15:50:33 -07:00
Nathan Beals	3c8dc4929f	Added test_restore_all_database_dump unit test. ... Updated the other unit tests, as I had to re-arrange argument order Added an 'all' test for the postgres end-to-end test. Ran black formatter on it all.	2020-04-24 18:32:53 -04:00
Dan Helfman	e511014a28	Fix MySQL restore error on "all" database dump by excluding system tables (#301).	2020-04-22 12:17:22 -07:00
Nathan Beals	f6407bafcb	Remove the --create flag, was causing an error	2020-04-10 11:24:13 -04:00
Nathan Beals	d5e9f67cec	Finished. Now uses 'psql' to run the plain-text scripts that pg_dumpall creates	2020-04-10 10:55:53 -04:00
Nathan Beals	b14f371c05	First attempt at fixing this pg_dumpall/restoring issue	2020-04-10 09:20:00 -04:00
Dan Helfman	d6562c4b1e	Fix the "--stats" and "--files" flags so that they yield output at verbosity 0 (#290).	2020-01-31 10:23:36 -08:00
Dan Helfman	55141bda67	Specify "--archive latest" to all actions that accept an archive (#289).	2020-01-29 16:59:02 -08:00