647 Commits

Author	SHA1	Message	Date
Dan Helfman	31d04d9ee3	Optionally error if a source directory does not exist. ... feat: add optional check for existence of source directories	2023-03-18 13:59:20 -07:00
Divyansh Singh	f803836416	reformat	2023-03-18 17:27:33 +05:30
Divyansh Singh	997f60b3e6	add tests	2023-03-18 17:24:21 +05:30
Soumik Dutta	fb9677230b	add test to ensure integers are converted to string ... before setting them up to be environment variable values Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-18 02:57:56 +05:30
Dan Helfman	7605838bfe	Add "--repository" flag to all actions where it makes sense (#564).	2023-03-16 13:27:08 -07:00
Nain	3e22414613	Update tests ... Make them more explicit. Also formatting.	2023-03-16 14:01:29 -04:00
Nain	5f87ea3ec5	Add "--repository" flag to the "create" action	2023-03-16 13:15:49 -04:00
Nain	a8aeace5b5	Add "--repository" flag to the "compact" action	2023-03-16 11:13:45 -04:00
Nain	480addd7ce	Add "--repository" flag to the "check" action	2023-03-16 10:41:13 -04:00
Nain	ce0ce4cd1c	Merge mostly repetetive tests	2023-03-16 08:23:21 -04:00
Nain	7de9260b0d	Remove test now that --repository isn't expected to error ... As discussed #652#issuecomment-5579	2023-03-15 14:59:12 -04:00
Nain	cdbe6cdf3a	Add "--repository" flag to the "prune" action ... part of ticket #564	2023-03-15 14:43:17 -04:00
Dan Helfman	8cec7c74d8	Add "--strip-components all" on the "extract" action to remove leading path components (#647).	2023-03-09 10:09:16 -08:00
Dan Helfman	b343363bb8	Change the default action order to: "create", "prune", "compact", "check" (#304).	2023-03-08 14:05:06 -08:00
Dan Helfman	9db31bd1e9	Run any command-line actions in the order specified instead of using a fixed ordering (#304).	2023-03-08 13:19:41 -08:00
Dan Helfman	d88bcc8be9	Add Healthchecks "log" state feature to NEWS.	2023-03-07 15:45:23 -08:00
Dan Helfman	332f7c4bb6	Add support for healthchecks "log" feature (#628). ... Reviewed-on: #645	2023-03-07 22:21:30 +00:00
Dan Helfman	5d19d86e4a	Add flake8-quotes to complain about incorrect quoting so I don't have to!	2023-03-07 14:08:35 -08:00
Soumik Dutta	044ae7869a	fix tests ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-08 03:30:12 +05:30
Soumik Dutta	98e429594e	added tests to make sure unsupported log states are detected ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 20:31:00 +05:30
Soumik Dutta	f442aeae9c	fix logs_monitor_start_error() ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 05:21:56 +05:30
Soumik Dutta	e211863cba	update test_borgmatic.py ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 05:12:24 +05:30
Soumik Dutta	45256ae33f	add test for healthchecks ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 03:38:08 +05:30
Dan Helfman	9ec220c600	Add SQLite database dump/restore hook (#295). ... feat: add dump-restore support for sqlite databases	2023-03-04 09:47:21 -08:00
Divyansh Singh	c71eb60cd2	mock os.remove instead of actually removing a file	2023-03-04 13:08:30 +05:30
Divyansh Singh	675e54ba9f	use os.remove and improve tests	2023-03-04 12:43:07 +05:30
Divyansh Singh	767a7d900b	e2e tests schema update	2023-03-04 01:29:01 +05:30
Divyansh Singh	903507bd03	code review	2023-03-04 01:27:07 +05:30
Dan Helfman	a071e02d20	With the "create" action and the "--list" ("--files") flag, only show excluded files at verbosity 2 (#620).	2023-03-02 15:33:42 -08:00
Divyansh Singh	3aa88085ed	formatting fix	2023-03-03 00:01:52 +05:30
Divyansh Singh	af1cc27988	feat: add dump-restore support for sqlite databases	2023-03-02 23:55:16 +05:30
Dan Helfman	dbf8301c19	Add "checkpoint_volume" configuration option to creates checkpoints every specified number of bytes.	2023-02-27 10:47:17 -08:00
Dan Helfman	2a306bef12	Fix tests.	2023-02-26 23:34:17 -08:00
Dan Helfman	d7a07f0428	Support status character changes in Borg 2.0.0b5 when filtering out special files that cause Borg to hang.	2023-02-26 22:36:13 -08:00
Dan Helfman	da321e180d	Fix the "create" action with the "--dry-run" flag querying for databases when a PostgreSQL/MySQL "all" database is configured.	2023-02-26 22:15:12 -08:00
Dan Helfman	c6582e1171	Internally support new Borg 2.0.0b5 "--filter" status characters / item flags for the "create" action.	2023-02-26 17:17:25 -08:00
Dan Helfman	9b83afe491	With the "create" action, only one of "--list" ("--files") and "--progress" flags can be used.	2023-02-26 17:05:56 -08:00
Tom Hubrecht	95575c3450	Add auth test for the ntfy hook	2023-02-25 20:04:39 +01:00
Dan Helfman	418ebc8843	Add MySQL database hook "add_drop_database" configuration option to control whether dumped MySQL databases get dropped right before restore (#642).	2023-02-20 15:32:47 -08:00
Dan Helfman	f5a448c7c2	Fix for potential data loss (data not getting backed up) when dumping large "directory" format PostgreSQL/MongoDB databases (#643).	2023-02-20 15:18:51 -08:00
Dan Helfman	c1eb210253	Fix code style flake issue.	2023-01-26 15:09:35 -08:00
Dan Helfman	30cca62d09	Add configuration options for database command customization (#630).	2023-01-26 14:59:17 -08:00
Dan Helfman	0e6b2c6773	Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).	2023-01-25 23:31:07 -08:00
palto42	fd7ad86daa	conditional warning for excluding special files	2023-01-03 21:53:51 +01:00
Dan Helfman	f67c544be6	Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).	2022-12-15 22:59:42 -08:00
Dan Helfman	c657764367	Fix logs that interfere with JSON output by making warnings go to stderr instead of stdout (#602).	2022-12-02 12:12:10 -08:00
Dan Helfman	ca7e18bb29	Override PostgreSQL dump/restore commands via configuration options (#311). ... Merge pull request #49 from jpaniagualaconich/specify-pg-dump-restore-commands	2022-11-18 08:33:14 -08:00
Dan Helfman	6975a5b155	Fix "data" consistency check to support "check_last" and consistency "prefix" options (#611).	2022-11-17 10:19:48 -08:00
Javier Paniagua	faf682ca35	specify pg dump/restore commands (#311)	2022-11-06 11:12:53 +01:00
Dan Helfman	89500df429	Fix traceback when a configuration section is present but lacking any options (#604).	2022-10-23 13:56:03 -07:00
Dan Helfman	9c42e7e817	Fix regression in which "check" action errored on certain systems (#597, #598).	2022-10-14 16:19:26 -07:00
Dan Helfman	953277a066	Fix special file detection when broken symlinks are encountered (#596).	2022-10-14 09:41:08 -07:00
Dan Helfman	d7f1c10c8c	To prevent Borg hangs, unconditionally delete stale named pipes before dumping databases (#360).	2022-10-12 10:26:09 -07:00
Dan Helfman	a31ce337e9	Skip auto-exclusion of special files when user explicitly sets read_special to true (#587).	2022-10-06 11:07:43 -07:00
Dan Helfman	c31702d092	Fix for potential data loss with "patterns_from". Also, display excluded files (#590).	2022-10-04 22:57:18 -07:00
Dan Helfman	ba8fbe7a44	Add "break-lock" action for removing any repository and cache locks leftover from Borg aborting (#357).	2022-10-04 13:42:18 -07:00
Dan Helfman	2774c2e4c0	Add support for Borg 2's "--match-archives" flag (replaces "--glob-archives") (#591).	2022-10-03 22:50:37 -07:00
Dan Helfman	ae036aebd7	When the "read_special" option is true or database hooks are enabled, auto-exclude special files for a "create" action to prevent Borg from hanging (#587).	2022-10-03 12:58:13 -07:00
Dan Helfman	ac7c7d4036	Warn when ignoring a configured "read_special" value of false, as true is needed when database hooks are enabled (#587).	2022-09-20 13:52:13 -07:00
Dan Helfman	679bb839d7	Fix hang when database hooks are enabled and "patterns" contains a parent directory of "~/.borgmatic" (#582).	2022-09-08 10:16:42 -07:00
Dan Helfman	9e64d847ef	Fix regression in which "borgmatic info --archive ..." showed repository info instead of archive info with Borg 1 (#577).	2022-08-30 20:42:42 -07:00
Dan Helfman	87c97b7568	Fixed spurious, intermittent test failures related to command execution and logging.	2022-08-28 09:06:06 -07:00
Dan Helfman	c46f2b8508	Fix conflict between "patterns" and "source_directories" (#574), make "source_directories" optional (#542).	2022-08-25 11:55:34 -07:00
Dan Helfman	ef7e95e22a	Fix end-to-end tests.	2022-08-21 23:29:13 -07:00
Dan Helfman	317dc7fbce	Add "before_actions" and "after_actions" command hooks that run before/after all the actions for each repository, update docs to cover per-repository configurations (#463).	2022-08-21 21:48:37 -07:00
Dan Helfman	462326406e	Drop only-style actions like "--create", rename "prune --files" to "prune --list", and add "--list" alias to "create" and "export-tar" (#571).	2022-08-21 14:25:16 -07:00
Dan Helfman	540f9f6b72	Add missing test for "transfer" action (#557).	2022-08-19 09:40:29 -07:00
Dan Helfman	f9d7faf884	Fix mount action to work without archive again (#557).	2022-08-18 23:33:05 -07:00
Dan Helfman	7dee6194a2	Add new "transfer" action for Borg 2 (#557).	2022-08-18 23:06:51 -07:00
Dan Helfman	68f9c1b950	Add generate-borgmatic-config end-to-end test.	2022-08-18 14:28:46 -07:00
Dan Helfman	43d711463c	Add additional command-line flags to rcreate action (#557).	2022-08-18 14:28:12 -07:00
Dan Helfman	b40e9b7da2	Ignore archive filter parameters passed to list action when --archive is given (#557).	2022-08-18 09:59:48 -07:00
Dan Helfman	89d201c8ff	Fleshing out NEWS for the Borg 2 changes.	2022-08-17 21:54:00 -07:00
Dan Helfman	f47c98c4a5	Rename several configuration options to match Borg 2 (#557).	2022-08-17 21:14:58 -07:00
Dan Helfman	3b6ed06686	Add --other-repo flag to rcreate action (#557).	2022-08-17 17:33:09 -07:00
Dan Helfman	57009e22b5	Use flag-related utility functions in info action (#557).	2022-08-17 17:11:02 -07:00
Dan Helfman	3ab7a3b64a	Replace use of --prefix with --glob-archives in info action (#557).	2022-08-17 15:36:19 -07:00
Dan Helfman	596dd49cf5	Use --glob-archives instead of --prefix on rlist command (#557).	2022-08-17 14:26:35 -07:00
Dan Helfman	28d847b8b1	Warn and tranform on non-ssh://-style repositories (#557).	2022-08-17 10:13:11 -07:00
Dan Helfman	30abd0e3de	Update borg action for Borg 2 support (#557).	2022-08-16 09:30:00 -07:00
Dan Helfman	f36e38ec20	Update mount action for Borg 2 support (#557).	2022-08-15 19:32:37 -07:00
Dan Helfman	d807ce095e	Update export-tar action for Borg 2 support (#557).	2022-08-15 17:34:12 -07:00
Dan Helfman	7626fe1189	Disallow borg list --json with --archive or --find (#557).	2022-08-15 15:40:28 -07:00
Dan Helfman	cc04bf57df	Update list action for Borg 2 support, add rinfo action, and update extract consistency check for Borg 2.	2022-08-15 15:04:40 -07:00
Dan Helfman	cce6d56661	Update extract action for Borg 2 support (#557).	2022-08-13 23:07:29 -07:00
Dan Helfman	a05d0f378e	Factor out repository/archive flags formatting code from create action (#557).	2022-08-13 22:50:14 -07:00
Dan Helfman	94321aec7a	Update compact action for Borg 2 support (#557).	2022-08-13 22:07:15 -07:00
Dan Helfman	4a55749bd2	Update prune action for Borg 2 support (#557).	2022-08-13 17:26:51 -07:00
Dan Helfman	2898e63166	Update create action for Borg 2 support (#557).	2022-08-12 23:54:13 -07:00
Dan Helfman	c7176bd00a	Add rinfo action for Borg 2 support (#557).	2022-08-12 23:06:56 -07:00
Dan Helfman	647ecdac29	Borg 2 support in borgmatic check action (#557).	2022-08-12 15:46:33 -07:00
Dan Helfman	e7a8acfb96	Add missing rinfo action source files (#557).	2022-08-12 14:59:03 -07:00
Dan Helfman	622caa0c21	Support for Borg 2's rcreate and rinfo sub-commands (#557).	2022-08-12 14:53:20 -07:00
Dan Helfman	9aece3936a	Modify "mount" and "extract" actions to require the "--repository" flag when multiple repositories are configured (#566).	2022-07-25 11:30:02 -07:00
Dan Helfman	bcad0de1a4	Add verify_tls option for Healthchecks to optionally disable TLS verification.	2022-07-23 23:11:41 -07:00
Uli	5c6407047f	feat: add verify_tls flag for Healthchecks	2022-07-24 07:37:00 +02:00
Dan Helfman	6ddae20fa1	Fix handling of "repository" and "data" consistency checks to prevent invalid Borg flags (#565).	2022-07-23 21:02:21 -07:00
Dan Helfman	16066942e3	Fix traceback with "create" action and "--json" flag when a database hook is configured (#563).	2022-07-19 10:25:10 -07:00
Dan Helfman	e85d551eac	Fix all database hooks to error when the requested database to restore isn't present in the Borg archive (#560).	2022-07-06 23:21:24 -07:00
Dan Helfman	2b23a63a08	Add end-to-end test for overrides.	2022-07-06 18:20:51 -07:00
Dan Helfman	457ed80744	Fix environment variable plumbing so options in one configuration file aren't used for others (#555).	2022-06-30 13:42:17 -07:00
Dan Helfman	10723efc68	Fix all monitoring hooks to warn if the server returns an HTTP 4xx error (#554).	2022-06-29 21:19:40 -07:00
Dan Helfman	bd4d109009	Fix logging to include the full traceback when Borg experiences an internal error (#553).	2022-06-28 13:38:24 -07:00
Dan Helfman	d372a86fe6	Code formatting.	2022-06-23 10:41:04 -07:00
Sébastien MB	b63c854509	Fix escaped environment variable in configuration ... - when an env variable is escaped in the configuration file, we expect not to resolve it and remove the escape char `\`	2022-06-17 09:50:56 +02:00
Dan Helfman	aecb6fcd74	Code style, rename command-line flag, and move new code into its own file (#546)	2022-06-16 11:35:24 -07:00
Sébastien MB	97b5cd089d	Allow environment variable resolution in configuration file ... - all string fields containing an environment variable like ${FOO} will be resolved - supported format ${FOO}, ${FOO:-bar} and ${FOO-bar} to allow default values if variable is not present in environment - add --no-env argument for CLI to disable the feature which is enabled by default Resolves: #546	2022-06-16 18:52:54 +02:00
Gavin Chappell	a8b8d507b6	add a hook for sending push notifications via ntfy.sh	2022-06-09 21:10:38 +01:00
Dan Helfman	3561c93d74	Fix Healthchecks tests that leak global state, breaking downstream tests (discovered in #543).	2022-06-09 11:05:44 -07:00
Dan Helfman	9aefb5179f	Fix None find paths (#541).	2022-06-03 15:20:05 -07:00
Dan Helfman	d14f22e121	Add "borgmatic list --find" flag for searching for files across multiple archives (#541).	2022-06-03 15:12:14 -07:00
Dan Helfman	80ec3e7d97	Deprecate "borgmatic list --successful" flag, as listing only non-checkpoint (successful) archives is now the default in newer versions of Borg.	2022-06-02 20:35:39 -07:00
Dan Helfman	2bc91ac3d2	Add "generate-borgmatic-config --overwrite" flag to replace an existing destination file (#539).	2022-05-29 16:03:55 -07:00
Dan Helfman	5b615d51a4	Add support for "borgmatic borg debug" command (#538).	2022-05-29 15:43:03 -07:00
Dan Helfman	8fa90053cf	Add "borgmatic check --force" flag to ignore configured check frequencies (#523).	2022-05-28 19:29:33 -07:00
Dan Helfman	ad0e2e0d7c	Tweak default check frequency to 1 month (#523).	2022-05-28 15:49:50 -07:00
Dan Helfman	e76bfa555f	Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523).	2022-05-28 14:42:19 -07:00
Dan Helfman	8ddb7268eb	Reuse "borg info" function.	2022-05-27 13:51:11 -07:00
Dan Helfman	cb5fe02ebd	Fix broken Bash completion end-to-end test.	2022-05-26 11:18:46 -07:00
Dan Helfman	77b84f8a48	Add Bash completion script so you can tab-complete the borgmatic command-line.	2022-05-26 10:27:53 -07:00
Dan Helfman	be59a3e574	Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options (#536).	2022-05-25 10:26:26 -07:00
Dan Helfman	54933ebef5	Change connection failures for monitoring hooks to be warnings instead of errors (#439).	2022-05-24 15:50:04 -07:00
Dan Helfman	157e59ac88	Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460).	2022-05-24 14:44:33 -07:00
Dan Helfman	8b179e4647	Reverse logic of Healtchecks "skip_states" option to just "states" (#525).	2022-05-24 14:09:42 -07:00
Dan Helfman	865eff7d98	Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525).	2022-05-24 13:59:28 -07:00
Dan Helfman	b9741f4d0b	Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294).	2022-05-24 12:23:38 -07:00
Dan Helfman	02781662f8	Change monitoring hooks to specify the ping URL / integration key as a named option.	2022-05-23 20:02:10 -07:00
Dan Helfman	32a1043468	Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402).	2022-05-23 16:11:24 -07:00
Dan Helfman	3e4aeec649	Warn when an unsupported variable is used in a hook command (#420).	2022-05-23 15:27:54 -07:00
Dan Helfman	255cc6ec23	When deep merging common configuration, merge colliding list values by appending them (#531).	2022-05-20 15:28:28 -07:00
Dan Helfman	788281cfb9	When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including (#532).	2022-05-19 17:15:05 -07:00
Dan Helfman	525266ede6	Deep merging when including common configuration (#381).	2022-04-25 21:18:37 -07:00
Dan Helfman	0e6b425ac5	Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515).	2022-04-23 14:03:15 -07:00
Dan Helfman	ed7fe5c6d0	Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473).	2022-04-21 22:08:25 -07:00
Dan Helfman	0c027a3050	Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516).	2022-04-03 13:12:48 -07:00
Dan Helfman	9f44bbad65	Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517).	2022-04-02 22:28:41 -07:00
Dan Helfman	976516d0e1	When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444).	2022-03-08 10:19:36 -08:00
Dan Helfman	574eb91921	Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507).	2022-03-07 21:46:12 -08:00
Dan Helfman	28fef3264b	Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486).	2022-03-07 15:32:07 -08:00
Dan Helfman	9161dbcb7d	Removing unnecessary leading underscores from functions.	2022-03-07 11:58:29 -08:00
Dan Helfman	4b3027e4fc	Add test for new working_directory option (#431).	2022-03-03 11:48:18 -08:00
Dan Helfman	0eb2634f9b	Working directory option to support source directories with relative paths (#431). ... Reviewed-on: #477	2022-03-03 19:28:17 +00:00
Dan Helfman	9317cbaaf0	Code formatting.	2022-02-10 10:23:34 -08:00
Dan Helfman	1b5f04b79f	When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 10:16:09 -08:00
Dan Helfman	948c86f62c	When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 10:09:18 -08:00
Dan Helfman	7e7209322a	When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-10 09:51:13 -08:00
Dan Helfman	00a57fd947	Code formatting.	2022-02-09 21:20:28 -08:00
Dan Helfman	6bf6ac310b	When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394).	2022-02-09 21:11:00 -08:00
Dan Helfman	4b5af2770d	When the "atime" option is used, tailor the flags passed to Borg depending on version (#394).	2022-02-09 16:54:35 -08:00
Dan Helfman	b525e70e1c	Run "compact" action by default when no actions are specified (#394).	2022-02-09 14:33:12 -08:00
Dan Helfman	e3f4bf0293	Build fix for ruamel.yaml.clib error.	2022-02-08 12:52:45 -08:00
Dan Helfman	48f44d2f3d	Add tests for compact action (#394).	2022-02-08 12:05:02 -08:00
Dan Helfman	bec73245e9	Fix traceback when a YAML validation error occurs (#480, #482).	2022-01-19 20:39:03 -08:00
fabianschilling	b39f08694d	Merge branch 'master' into pr-working-directory	2022-01-05 09:30:27 +00:00
Dan Helfman	2ee75546f5	Add MongoDB database hook documentation.	2022-01-04 16:26:38 -08:00
Andrea Ghensi	7c6ce9399c	fix integration tests and mongodb auth	2021-12-29 22:18:50 +01:00
Andrea Ghensi	6b7653484b	Add mongodb dump hook	2021-12-26 01:00:58 +01:00
Fabian Schilling	85e0334826	Add missing working_directory arg to pass tests	2021-12-10 18:24:41 +01:00
Dan Helfman	a1673d1fa1	Fix unicode error when restoring particular MySQL databases (#476).	2021-12-08 16:40:25 -08:00
Dan Helfman	7a086d8430	Fix import ordering.	2021-11-29 14:00:14 -08:00
Dan Helfman	0e8e9ced64	When command-line configuration override produces a parse error, error cleanly (#471).	2021-11-29 12:49:21 -08:00
nebulon42	c3e76585fc	move mysqldump options to the beginning of the command due to MySQL bug 30994.	2021-11-26 17:16:03 +01:00
nebulon42	3729ba5ca3	add list_options setting, fixes #306	2021-11-20 15:43:58 +01:00
Dan Helfman	38ebfd2969	Rename retry_timeout to retry_wait and standardize log formatting (#28).	2021-11-15 11:51:17 -08:00
cadamswaite	976a877a25	Formatting	2021-11-14 22:37:42 +00:00
cadamswaite	b4117916b8	Add timeout and tests	2021-11-14 22:15:22 +00:00
cadamswaite	19cad89978	Add some tests for retry logic	2021-11-14 21:35:23 +00:00
Dan Helfman	449896f661	Fix error when configured source directories are not present on the filesystem at the time of backup (#387).	2021-10-11 10:40:10 -07:00
Dan Helfman	acb2ca79d9	Fix traceback that can occur when dumping a database (#440).	2021-08-06 08:58:11 -07:00
Dan Helfman	80b33fbf8a	Code style reformatting.	2021-07-27 09:39:48 -07:00
Marek Szuba	e8b8d86592	tests/integration/test_execute: use plain Python rather than xxd ... Removes this test's dependencies on vim and /dev/urandom. Signed-off-by: Marek Szuba <marek.szuba@cern.ch>	2021-07-27 13:50:16 +01:00
Dan Helfman	2a8692c64f	Fix integration test to hopefully work on Alpine (#430).	2021-07-25 22:50:00 -07:00
Dan Helfman	1709f57ff0	Fix hang when restoring a PostgreSQL "tar" format database dump (#430).	2021-07-25 22:30:15 -07:00
Dan Helfman	54d57e1349	Add test for #407: Fix syslog logging on FreeBSD.	2021-06-23 10:21:45 -07:00
Dan Helfman	27d37b606b	Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema).	2021-06-22 13:27:59 -07:00
Dan Helfman	cf8882f2bc	Run arbitrary Borg commands with new "borgmatic borg" action (#425).	2021-06-17 20:41:44 -07:00
Dan Helfman	9fd28d2eed	Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)!	2021-06-08 11:43:55 -07:00
Dan Helfman	d6e376d32d	Fix end-to-end test broken by change in source directory examples.	2021-04-18 17:54:54 -07:00
Dan Helfman	d543109ef4	"Fix" build failure with Alpine Edge by switching from Edge to Alpine 3.13.	2021-04-09 15:58:23 -07:00
Dan Helfman	4897a78fd3	Fix database tests broken by PostgreSQL upgrade in Alpine Edge.	2020-12-24 22:23:09 -08:00
Dan Helfman	ad7198ba66	Tweak to test failing on some machines.	2020-11-26 16:22:42 -08:00
Dan Helfman	c0ae01f5d5	Code formatting.	2020-11-25 17:46:57 -08:00
Dan Helfman	ccd1627175	Fix timing-related test error in Alpine Edge.	2020-11-25 15:48:33 -08:00
Dan Helfman	1f4f28b4dc	Drop support for Python 3.5. Only support black code formatter on Python 3.8+.	2020-11-22 17:27:21 -08:00
Dan Helfman	ea6cd53067	Update versions of test dependencies (test_requirements.txt and test containers).	2020-11-22 14:48:07 -08:00
Dan Helfman	9b819f32f8	Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config (#367).	2020-11-18 15:01:10 -08:00
Dan Helfman	b619bde037	Fix broken end-to-end test.	2020-11-18 14:20:07 -08:00
François Poulain	ebac02f118	Mention placeholders for repositories in documentation; fix #369	2020-11-18 20:22:41 +01:00
Dan Helfman	1ea04aedf0	Merge override values when specifying the "--override" flag multiple times (#361).	2020-10-23 16:06:00 -07:00
Dan Helfman	0e978299cf	Fix traceback when a database hook value is null in a configuration file (#355).	2020-10-07 15:41:45 -07:00
networkjanitor	3c723e8d99	Merge branch 'master' into master	2020-08-12 21:11:45 +00:00
Dan Helfman	72bd96c656	Fix traceback when a configuration directory is non-readable due to directory permissions (#350).	2020-08-12 11:32:00 -07:00
networkjanitor	893fca2816	Add before_extract and after_extract hooks	2020-08-01 16:08:32 +02:00
Dan Helfman	b3fd1be5f6	Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300).	2020-07-22 21:53:55 -07:00
Dan Helfman	6f82c9979b	Add #339 to NEWS and add test.	2020-07-07 22:31:17 -07:00
Dan Helfman	37cc229749	Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328).	2020-06-23 11:01:03 -07:00
Dan Helfman	17c2d109e5	Add tests for pass-through of BORG_* environment variables.	2020-06-21 14:41:22 -07:00
Dan Helfman	32e15dc905	Add a few more mocks to PostgreSQL SSL tests.	2020-06-20 14:39:16 -07:00
Dan Helfman	f5ebca4907	Add SSL support to PostgreSQL database configuration (#331). ... Reviewed-on: witten/borgmatic#331	2020-06-20 21:24:14 +00:00
Edward Shornock	d2d92b1f1a	Add tests for the PostgreSQL SSL options	2020-06-20 23:32:24 +03:00
Dan Helfman	27cbe9dfc0	Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333).	2020-06-19 20:16:38 -07:00
Edward Shornock	abd47fc14e	Add SSL support to PostgreSQL hooks	2020-06-19 02:19:17 +03:00
Dan Helfman	7fb4061759	Improve configuration reference documentation readability via more aggressive word-wrapping in configuration schema descriptions.	2020-06-17 23:15:12 -07:00
Dan Helfman	aa14449857	Add "borgmatic extract --strip-components" flag to remove leading path components when extracting an archive (#324).	2020-06-06 14:57:14 -07:00
Dan Helfman	a155eefa23	Fix for certain configuration options like ssh_command impacting Borg invocations for separate configuration files (#323).	2020-06-06 14:30:04 -07:00
Dan Helfman	d7277893fb	Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316).	2020-06-02 12:40:32 -07:00
Dan Helfman	097a09578a	Fix enabled database hooks to implicitly set one_file_system configuration option to true to prevent Borg hang. (#315).	2020-05-26 09:20:14 -07:00
Dan Helfman	65472c8de2	Fix error message when there are no MySQL databases to dump for "all" databases (#319).	2020-05-26 08:59:04 -07:00
Dan Helfman	96df52ec50	Fix hang when streaming a database dump to Borg with implicit duplicate source directories by deduplicating them first (#316).	2020-05-20 13:33:53 -07:00
Dan Helfman	89cb5eb76d	Fix regression in support for PostgreSQL's "directory" dump format (#314).	2020-05-18 11:31:29 -07:00
Dan Helfman	048a9ebb52	Add an additional end-to-end database test.	2020-05-15 10:12:49 -07:00
Dan Helfman	3e5a19d95a	Add missing test coverage.	2020-05-14 23:21:43 -07:00
Dan Helfman	d88f321cef	Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310).	2020-05-14 22:38:38 -07:00
Dan Helfman	15ea70a71b	Flesh out missing test coverage for execute.py.	2020-05-12 15:49:49 -07:00
Dan Helfman	8b91c01a4c	Add some missing test coverage.	2020-05-11 11:34:14 -07:00
Dan Helfman	3bcef72050	Add some missing test mocks that were masking lack of unit coverage.	2020-05-11 11:17:24 -07:00
Dan Helfman	695c764a01	Merge log output functions into one.	2020-05-11 10:55:50 -07:00
Dan Helfman	1ea047dd94	Remove "borgmatic restore --progress" flag, as it now conflicts with streaming database restores.	2020-05-09 21:53:16 -07:00
Dan Helfman	4b523f9e2c	Make database restore output only show at verbosity 2.	2020-05-08 19:38:33 -07:00
Dan Helfman	6a61070d85	Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe.	2020-05-08 19:11:26 -07:00
Dan Helfman	f36082938e	Additional test coverage.	2020-05-08 09:48:04 -07:00
Dan Helfman	1ba996ad93	Additional test coverage.	2020-05-07 12:14:27 -07:00
Dan Helfman	a23fdf946d	Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258).	2020-05-07 11:44:04 -07:00
Dan Helfman	12cf6913ef	Remove unused function parameter.	2020-05-03 14:07:34 -07:00
Dan Helfman	e4eff0e3dc	Bump version for release.	2020-04-24 15:56:56 -07:00
Dan Helfman	dce1928dc4	Fix PostgreSQL restore error on "all" database dump.	2020-04-24 15:50:33 -07:00
Nathan Beals	3c8dc4929f	Added `test_restore_all_database_dump` unit test. ... Updated the other unit tests, as I had to re-arrange argument order Added an 'all' test for the postgres end-to-end test. Ran black formatter on it all.	2020-04-24 18:32:53 -04:00
Dan Helfman	e511014a28	Fix MySQL restore error on "all" database dump by excluding system tables (#301).	2020-04-22 12:17:22 -07:00
Dan Helfman	bae5f88824	Upgrade test database versions.	2020-04-21 10:01:26 -07:00
Dan Helfman	55141bda67	Specify "--archive latest" to all actions that accept an archive (#289).	2020-01-29 16:59:02 -08:00
Dan Helfman	bc02c123e6	Monitor backups with PagerDuty hook integration (#245).	2020-01-27 15:32:09 -08:00
Dan Helfman	e76d5ad988	Fix tests.	2020-01-27 12:56:12 -08:00
Dan Helfman	2405e97c38	Backup to a removable drive or intermittent server via "soft failure" feature (#284).	2020-01-24 20:52:48 -08:00
Dan Helfman	fdbb2ee905	View consistency check progress via "--progress" flag for "check" action (#287).	2020-01-24 11:27:16 -08:00
Dan Helfman	94b9ef56be	Change "exclude_if_present" option to support multiple filenames, rather than just a single filename (#280).	2020-01-23 13:41:37 -08:00
Dan Helfman	53e6ff9524	No longer list files or show stats by default at verbosity 2.	2020-01-22 15:23:49 -08:00
Dan Helfman	f66fd1caaa	Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag (#277).	2020-01-22 15:10:47 -08:00
Dan Helfman	d93fdbc5ad	Support "--files" and "--stats" flags at verbosity level 0.	2020-01-22 13:28:24 -08:00
palto42	75b5e7254e	changes as per comments in PR #283	2020-01-22 19:03:26 +01:00
palto42	5f0c084bee	Merge 'upstream/master' into list-files	2020-01-22 18:12:26 +01:00
Dan Helfman	7824a034ca	Add test for database dump directory removal.	2020-01-21 10:34:46 -08:00
palto42	e27ba0d08a	less detail at v1 + option "--files" for details	2020-01-11 16:38:07 +01:00
Dan Helfman	d0c533555e	In "borgmatic --help", don't expand $HOME in listing of default "--config" paths.	2020-01-02 10:37:31 -08:00
Dan Helfman	911668f0c8	Only trigger "on_error" hooks and monitoring failures for "prune", "create", and "check" actions, and not for other actions (#270).	2019-12-20 13:58:02 -08:00
Dan Helfman	d64bcd5e83	When pruning with verbosity level 1, list pruned and kept archives.	2019-12-17 20:12:41 -08:00
Dan Helfman	f787dfe809	Override particular configuration options from the command-line via "--override" flag (#268).	2019-12-17 11:46:27 -08:00
Dan Helfman	e009bfeaa2	Update Healthchecks/Cronitor/Cronhub monitoring integrations to fire for "check" and "prune" actions, not just "create" (#249).	2019-12-12 22:54:45 -08:00
Dan Helfman	c6cb21a748	Switch to read-only container filesystem to avoid *.pyc getting created with busted permissions.	2019-12-11 21:24:37 -08:00
Dan Helfman	464ff2fe96	Run end-to-end tests on developer machines with Docker Compose for approximate parity with continuous integration tests.	2019-12-11 16:43:01 -08:00
Dan Helfman	0cc711173a	Merge branch 'master' into end-to-end-database-tests	2019-12-11 12:27:14 -08:00
Dan Helfman	14e5cfc8f8	Support piping "borgmatic list" output to grep. Retain colored output when piping/redirecting (#271).	2019-12-11 12:12:25 -08:00
Dan Helfman	68281339b7	Black.	2019-12-10 16:57:12 -08:00
Dan Helfman	01e2cf08d1	Fix Drone CI services syntax.	2019-12-10 16:43:43 -08:00
Dan Helfman	9f821862b7	End-to-end tests for database dump and restore.	2019-12-10 16:41:01 -08:00
Dan Helfman	8660af745e	Optionally change the internal database dump path via "borgmatic_source_directory" option in location configuration section (#259).	2019-12-10 16:04:34 -08:00
Dan Helfman	826e4352d1	Filter listed paths via "borgmatic list --path" flag (#269).	2019-12-08 14:07:02 -08:00
Dan Helfman	b94999bba4	Fix "borgmatic umount" so it only runs Borg once instead of once per repository / configuration file.	2019-12-07 21:36:51 -08:00
Dan Helfman	65cc4c9429	Fix "--repository" flag to accept relative paths.	2019-12-06 16:29:41 -08:00
Dan Helfman	df2be9620b	Mount whole repositories via "borgmatic mount" without any "--archive" flag (#253).	2019-12-06 15:58:54 -08:00
Dan Helfman	2ab9daaa0f	Attempt to repair any inconsistencies found during a consistency check via "borgmatic check --repair" flag (#266).	2019-12-04 16:07:00 -08:00
Dan Helfman	0c6c61a272	Pass extra options directly to particular Borg commands, handy for Borg options that borgmatic does not yet support natively (#235).	2019-12-04 15:48:10 -08:00
Dan Helfman	00f62ca023	Fix for "before_backup" hook not triggering an error when the command contains "borg" and has an exit code of 1 (#256).	2019-11-30 16:55:05 -08:00
Dan Helfman	9b2ca15de6	Fix for garbled Borg file listing when using "borgmatic create --progress" with verbosity level 1 or 2 (#257).	2019-11-30 15:31:36 -08:00
Dan Helfman	c4aa34bf5c	Fix for missing Healthchecks monitoring payload or HTTP 500 due to incorrect unicode encoding (#260).	2019-11-30 14:51:32 -08:00
Dan Helfman	d978a2d190	Fix for database dump removal incorrectly skipping some database dumps.	2019-11-25 15:27:59 -08:00
Dan Helfman	375036e409	Support for mounting an archive as a FUSE filesystem via "borgmatic mount" action, and unmounting via "borgmatic umount" (#123).	2019-11-25 14:56:20 -08:00
Dan Helfman	f4a231420f	Show summary log errors regardless of verbosity. Log the "summary:" header with level based on the contained logs.	2019-11-25 10:31:09 -08:00
Dan Helfman	55ebfdda39	Show full error logs at "--verbosity 0" so you can see command output without upping the verbosity level.	2019-11-19 10:09:25 -08:00
Dan Helfman	2cc743cf47	With "borgmatic check", consider Borg warnings as errors (#247).	2019-11-17 19:06:28 -08:00
Dan Helfman	d99e6d1994	Remove obnoxious log message when Healthchecks hook sends failure for check action.	2019-11-17 18:55:40 -08:00
Dan Helfman	50f62d73b7	When using the Healthchecks monitoring hook, include borgmatic logs in the payloads for completion and failure pings (#241).	2019-11-17 16:54:27 -08:00
Dan Helfman	1f166a47e9	Fix failing test due to dictionary order (or the lack thereof..) in Python 3.5.	2019-11-13 16:42:58 -08:00
Dan Helfman	6cdc92bd0c	Fix for "borgmatic restore" showing success and incorrectly extracting archive files, even when no databases are configured to restore (#246).	2019-11-13 10:41:57 -08:00
Dan Helfman	0a9f4e8708	Reopen the file given by "--log-file" flag if an external program rotates the log file while borgmatic is running.	2019-11-12 16:13:25 -08:00
Dan Helfman	781fac3266	Refactor ping monitors (Healthchecks, Cronitor, Cronhub) to share a common invocation and function signature.	2019-11-12 15:31:07 -08:00
Dan Helfman	04e5b42606	Fix repository does not exist error with "borgmatic extract" when repository is remote (#243).	2019-11-12 11:47:24 -08:00
Dan Helfman	ebeb5efe05	More accurately detecting restore of unconfigured database (#228).	2019-11-12 11:10:47 -08:00
Dan Helfman	a3e939f34b	Factor out filtering of database configuration to function with tests (#228).	2019-11-12 10:39:36 -08:00
Dan Helfman	2a771161e7	Finish test coverage for MySQL restore (#228).	2019-11-12 09:50:26 -08:00
Dan Helfman	ded042d8cc	First crack at MySQL/MariaDB database restore (#228).	2019-11-11 21:59:30 -08:00
Dan Helfman	9d29ecf304	Remove MySQL/MariaDB database dumps after backing them up (#228).	2019-11-08 11:53:27 -08:00
Dan Helfman	427b57e2a9	Database dump hooks for MySQL/MariaDB (#228).	2019-11-08 11:17:52 -08:00
Dan Helfman	17fda7281a	Monitor backups with Cronhub hook integration. Fix Healthchecks/Cronitor hooks to respect dry run.	2019-11-07 10:08:44 -08:00
Dan Helfman	ac777965d0	Fix regression of generate-borgmatic-config working without --source flag (#239).	2019-11-06 09:52:21 -08:00
Dan Helfman	2115eeb6a2	Upgrade your borgmatic configuration to get new options and comments via "generate-borgmatic-config --source" (#239).	2019-11-06 09:31:00 -08:00
palto42	18d3542fbc	new verbosity level "-1" for errors	2019-11-03 09:55:19 +01:00
Dan Helfman	a9104ed090	Handle log file error more consistently with other error. Add --log-file-verbosity flag. Add docs.	2019-11-02 11:23:18 -07:00
Dan Helfman	06f134cc71	Log to file instead of syslog via command-line "--log-file" flag (#233).	2019-11-02 17:43:39 +00:00
palto42	26a1a3d1e0	test cases for logfile option	2019-11-02 14:34:16 +01:00
Dan Helfman	f48f52079d	Support for Borg --keep-exclude-tags and --exclude-nodump options (#234).	2019-11-01 15:50:59 -07:00
Dan Helfman	8fd46b8c70	Monitor backups with Cronitor hook integration.	2019-11-01 11:33:15 -07:00
Dan Helfman	e3dd545345	Extract files to a particular directory via "borgmatic extract --destination" flag. Also rename "--restore-path" to "--path" to reduce confusion.	2019-11-01 10:00:26 -07:00
Dan Helfman	939f4d4e3d	Add additional tests for database restores.	2019-10-31 14:11:19 -07:00
Dan Helfman	3006db0cae	Restore backed up PostgreSQL databases via "borgmatic restore" sub-command (#229).	2019-10-31 12:57:36 -07:00
Dan Helfman	37dc94bc79	Add test for removal of database dumps.	2019-10-23 13:36:03 -07:00
Dan Helfman	fc274b43f0	Rename "borgmatic list --pattern-from" flag to "--patterns-from" to match Borg (#230).	2019-10-22 22:42:36 -07:00
Dan Helfman	9ab12e4312	Tests for database dumping (#225).	2019-10-22 21:39:30 -07:00
Dan Helfman	458e7776c5	Database dump hooks for PostgreSQL, so you can easily dump your databases before backups run (#225).	2019-10-22 16:28:42 -07:00
Dan Helfman	fa5fa1c11b	Move hooks into directory, so there can be one source file per type of hook (#225).	2019-10-21 15:52:14 -07:00
Dan Helfman	f8bc67be8d	Config generation support for sequences of maps, needed for database dump hooks (#225).	2019-10-21 15:17:47 -07:00