152 Commits

Author	SHA1	Message	Date
Divyansh Singh	f273e82d74	add tests	2023-04-15 02:57:51 +05:30
Jakub Jirutka	17f122bfe5	Use psql instead of pg_restore when format is "plain" ... pg_restore: error: input file appears to be a text format dump. Please use psql.	2023-04-14 17:38:19 +02:00
Jakub Jirutka	f0f43174c6	Swap if-else in restore_database_dump in postgresql hook for cleanliness	2023-04-14 17:38:19 +02:00
Jakub Jirutka	dfccc1b94a	Exit on error when restoring all PostgreSQL databases ... "--set ON_ERROR_STOP=on" is equivalent to "--exit-on-error" in pg_restore.	2023-04-14 17:38:18 +02:00
Jakub Jirutka	195024e505	Fix psql_command and pg_restore_command to accept command with arguments ... These commands are executed without `shell=True`, so the subprocess module treats e.g. "docker exec my_pg_container psql" as a single command (resulting in Errno 2 "No such file or directory") instead of a command with arguments.	2023-04-14 17:37:38 +02:00
Jakub Jirutka	19a00371f5	Run "psql" with "--no-psqlrc" ... Some settings in user's .psqlrc, e.g. "linestyle unicode", may break the CSV output. "--no-psqlrc" tells psql to not read startup file. This is not necessary for the analyze_command and restore_command (with all_databases), but it's generally recommended when running psql from a script.	2023-04-14 17:37:37 +02:00
Jakub Jirutka	874fba7672	Fix PostgreSQL hook not using "psql_command" for list when dumping "all"	2023-04-14 15:13:49 +02:00
Divyansh Singh	2fea429d78	collection restore for mongodb	2023-04-12 09:34:19 +05:30
Divyansh Singh	264cebd2b1	complete psql multi schema backup	2023-04-11 23:19:49 +05:30
Divyansh Singh	9bc2322f9a	feat: restore specific schemas	2023-04-06 02:10:36 +05:30
Dan Helfman	d0d3a39833	When a database command errors, display and log the error message instead of swallowing it (#396).	2023-03-27 10:36:39 -07:00
Dan Helfman	f42890430c	Add code style plugins to enforce use of Python f-strings and prevent single-letter variables.	2023-03-23 23:11:14 -07:00
Dan Helfman	d88bcc8be9	Add Healthchecks "log" state feature to NEWS.	2023-03-07 15:45:23 -08:00
Soumik Dutta	044ae7869a	fix tests ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-08 03:30:12 +05:30
Soumik Dutta	4fcfddbe08	return early if unsupported state is passed ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 19:58:57 +05:30
Soumik Dutta	45256ae33f	add test for healthchecks ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-06 03:38:08 +05:30
Soumik Dutta	1573d68fe2	update schema.yaml description ... also add monitor.State.LOG to cronitor. Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-05 21:57:13 +05:30
Soumik Dutta	69f6695253	Add support for healthchecks "log" feature #628 ... Signed-off-by: Soumik Dutta <shalearkane@gmail.com>	2023-03-05 19:27:32 +05:30
Dan Helfman	e1f02d9fa5	Add SQLite feature to NEWS and also integrations.	2023-03-04 09:59:16 -08:00
Divyansh Singh	cf0275a3ed	remove test path	2023-03-04 23:00:57 +05:30
Divyansh Singh	c71eb60cd2	mock os.remove instead of actually removing a file	2023-03-04 13:08:30 +05:30
Divyansh Singh	675e54ba9f	use os.remove and improve tests	2023-03-04 12:43:07 +05:30
Divyansh Singh	903507bd03	code review	2023-03-04 01:27:07 +05:30
Divyansh Singh	af1cc27988	feat: add dump-restore support for sqlite databases	2023-03-02 23:55:16 +05:30
Dan Helfman	da321e180d	Fix the "create" action with the "--dry-run" flag querying for databases when a PostgreSQL/MySQL "all" database is configured.	2023-02-26 22:15:12 -08:00
Dan Helfman	8a9d5d93f5	Add ntfy authentication to NEWS.	2023-02-25 14:23:42 -08:00
Tom Hubrecht	9b071ff92f	Make the auth logic more explicit and warnings if necessary	2023-02-25 20:04:39 +01:00
Tom Hubrecht	d80e716822	Add authentication to the ntfy hook	2023-02-24 17:35:53 +01:00
Dan Helfman	418ebc8843	Add MySQL database hook "add_drop_database" configuration option to control whether dumped MySQL databases get dropped right before restore (#642).	2023-02-20 15:32:47 -08:00
Dan Helfman	f5a448c7c2	Fix for potential data loss (data not getting backed up) when dumping large "directory" format PostgreSQL/MongoDB databases (#643).	2023-02-20 15:18:51 -08:00
Dan Helfman	30cca62d09	Add configuration options for database command customization (#630).	2023-01-26 14:59:17 -08:00
Dan Helfman	0e6b2c6773	Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).	2023-01-25 23:31:07 -08:00
Dan Helfman	f67c544be6	Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).	2022-12-15 22:59:42 -08:00
Dan Helfman	fd39f54df7	Code formatting.	2022-11-18 08:35:01 -08:00
Javier Paniagua	faf682ca35	specify pg dump/restore commands (#311)	2022-11-06 11:12:53 +01:00
Dan Helfman	9c42e7e817	Fix regression in which "check" action errored on certain systems (#597, #598).	2022-10-14 16:19:26 -07:00
Dan Helfman	d7f1c10c8c	To prevent Borg hangs, unconditionally delete stale named pipes before dumping databases (#360).	2022-10-12 10:26:09 -07:00
Uli	5c6407047f	feat: add verify_tls flag for Healthchecks	2022-07-24 07:37:00 +02:00
Dan Helfman	e85d551eac	Fix all database hooks to error when the requested database to restore isn't present in the Borg archive (#560).	2022-07-06 23:21:24 -07:00
Dan Helfman	10723efc68	Fix all monitoring hooks to warn if the server returns an HTTP 4xx error (#554).	2022-06-29 21:19:40 -07:00
Gavin Chappell	a8b8d507b6	add a hook for sending push notifications via ntfy.sh	2022-06-09 21:10:38 +01:00
Dan Helfman	e76bfa555f	Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523).	2022-05-28 14:42:19 -07:00
Dan Helfman	54933ebef5	Change connection failures for monitoring hooks to be warnings instead of errors (#439).	2022-05-24 15:50:04 -07:00
Dan Helfman	157e59ac88	Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460).	2022-05-24 14:44:33 -07:00
Dan Helfman	8b179e4647	Reverse logic of Healtchecks "skip_states" option to just "states" (#525).	2022-05-24 14:09:42 -07:00
Dan Helfman	865eff7d98	Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525).	2022-05-24 13:59:28 -07:00
Dan Helfman	b9741f4d0b	Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294).	2022-05-24 12:23:38 -07:00
Dan Helfman	02781662f8	Change monitoring hooks to specify the ping URL / integration key as a named option.	2022-05-23 20:02:10 -07:00
Dan Helfman	3e4aeec649	Warn when an unsupported variable is used in a hook command (#420).	2022-05-23 15:27:54 -07:00
Dan Helfman	b98b827594	Remove stale comment.	2022-05-23 10:59:56 -07:00
Dan Helfman	e40e726687	Change Healthchecks logs truncation size from 10k bytes to 100k bytes, corresponding to that same change on Healthchecks.io.	2022-04-06 22:00:18 -07:00
Dan Helfman	2ee75546f5	Add MongoDB database hook documentation.	2022-01-04 16:26:38 -08:00
Andrea Ghensi	7c6ce9399c	fix integration tests and mongodb auth	2021-12-29 22:18:50 +01:00
Andrea Ghensi	6b7653484b	Add mongodb dump hook	2021-12-26 01:00:58 +01:00
Dan Helfman	a1673d1fa1	Fix unicode error when restoring particular MySQL databases (#476).	2021-12-08 16:40:25 -08:00
nebulon42	c3e76585fc	move mysqldump options to the beginning of the command due to MySQL bug 30994.	2021-11-26 17:16:03 +01:00
nebulon42	3729ba5ca3	add list_options setting, fixes #306	2021-11-20 15:43:58 +01:00
Dan Helfman	0e978299cf	Fix traceback when a database hook value is null in a configuration file (#355).	2020-10-07 15:41:45 -07:00
Dan Helfman	3aaa89fb08	Formatting.	2020-06-25 20:25:29 -07:00
Dan Helfman	35d542a676	Fix for traceback when running Cronitor, Cronhub, and PagerDuty monitor hooks (#336).	2020-06-25 20:23:25 -07:00
Dan Helfman	37cc229749	Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328).	2020-06-23 11:01:03 -07:00
Edward Shornock	a16fed8887	Rename PostgreSQL SSL config variables ... e.g. s/sslmode/ssl_mode/g to conform with borgmatic naming conventions.	2020-06-19 13:20:14 +03:00
Edward Shornock	33113890f5	Reduce duplication with a common function	2020-06-19 12:32:36 +03:00
Edward Shornock	abd47fc14e	Add SSL support to PostgreSQL hooks	2020-06-19 02:19:17 +03:00
Dan Helfman	ed7b1cd3d7	Add some no-cover pragmas on functions that don't need tests.	2020-06-06 14:33:06 -07:00
Dan Helfman	398665be9e	Allow before_backup and similiar hooks to exit with a soft failure without altering the monitoring status (#292).	2020-06-02 14:33:41 -07:00
Dan Helfman	d7277893fb	Fix hang when a stale database dump named pipe from an aborted borgmatic run remains on disk (#316).	2020-06-02 12:40:32 -07:00
Dan Helfman	65472c8de2	Fix error message when there are no MySQL databases to dump for "all" databases (#319).	2020-05-26 08:59:04 -07:00
Dan Helfman	89cb5eb76d	Fix regression in support for PostgreSQL's "directory" dump format (#314).	2020-05-18 11:31:29 -07:00
Dan Helfman	d88f321cef	Fix legitimate database dump command errors (exit code 1) not being treated as errors by borgmatic (#310).	2020-05-14 22:38:38 -07:00
Dan Helfman	4b523f9e2c	Make database restore output only show at verbosity 2.	2020-05-08 19:38:33 -07:00
Dan Helfman	6a61070d85	Use shell redirection rather than the --file flag to sidestep synchronization issues when pg_dump/pg_dumpall tries to write to a named pipe.	2020-05-08 19:11:26 -07:00
Dan Helfman	1ba996ad93	Additional test coverage.	2020-05-07 12:14:27 -07:00
Dan Helfman	a23fdf946d	Stream database dumps and restores directly to/from Borg without using any additional filesystem space (#258).	2020-05-07 11:44:04 -07:00
Dan Helfman	dce1928dc4	Fix PostgreSQL restore error on "all" database dump.	2020-04-24 15:50:33 -07:00
Nathan Beals	3c8dc4929f	Added `test_restore_all_database_dump` unit test. ... Updated the other unit tests, as I had to re-arrange argument order Added an 'all' test for the postgres end-to-end test. Ran black formatter on it all.	2020-04-24 18:32:53 -04:00
Dan Helfman	e511014a28	Fix MySQL restore error on "all" database dump by excluding system tables (#301).	2020-04-22 12:17:22 -07:00
Nathan Beals	f6407bafcb	Remove the `--create` flag, was causing an error	2020-04-10 11:24:13 -04:00
Nathan Beals	d5e9f67cec	Finished. Now uses 'psql' to run the plain-text scripts that pg_dumpall creates	2020-04-10 10:55:53 -04:00
Nathan Beals	b14f371c05	First attempt at fixing this pg_dumpall/restoring issue	2020-04-10 09:20:00 -04:00
Dan Helfman	bc02c123e6	Monitor backups with PagerDuty hook integration (#245).	2020-01-27 15:32:09 -08:00
Dan Helfman	2405e97c38	Backup to a removable drive or intermittent server via "soft failure" feature (#284).	2020-01-24 20:52:48 -08:00
Dan Helfman	f66fd1caaa	Customize Healthchecks log level via borgmatic "--monitoring-verbosity" flag (#277).	2020-01-22 15:10:47 -08:00
Ronan Dunklau	8a91c79fb0	Support directory format dump cleanup. ... Previously, only deleting a dump in a single-file format was supported. This led to errors when performing a PostgreSQL directory format backup.	2020-01-19 15:15:47 +01:00
Dan Helfman	8660af745e	Optionally change the internal database dump path via "borgmatic_source_directory" option in location configuration section (#259).	2019-12-10 16:04:34 -08:00
Dan Helfman	c4aa34bf5c	Fix for missing Healthchecks monitoring payload or HTTP 500 due to incorrect unicode encoding (#260).	2019-11-30 14:51:32 -08:00
Dan Helfman	d978a2d190	Fix for database dump removal incorrectly skipping some database dumps.	2019-11-25 15:27:59 -08:00
Dan Helfman	d99e6d1994	Remove obnoxious log message when Healthchecks hook sends failure for check action.	2019-11-17 18:55:40 -08:00
Dan Helfman	50f62d73b7	When using the Healthchecks monitoring hook, include borgmatic logs in the payloads for completion and failure pings (#241).	2019-11-17 16:54:27 -08:00
Dan Helfman	6cdc92bd0c	Fix for "borgmatic restore" showing success and incorrectly extracting archive files, even when no databases are configured to restore (#246).	2019-11-13 10:41:57 -08:00
Dan Helfman	781fac3266	Refactor ping monitors (Healthchecks, Cronitor, Cronhub) to share a common invocation and function signature.	2019-11-12 15:31:07 -08:00
Dan Helfman	ebeb5efe05	More accurately detecting restore of unconfigured database (#228).	2019-11-12 11:10:47 -08:00
Dan Helfman	a3e939f34b	Factor out filtering of database configuration to function with tests (#228).	2019-11-12 10:39:36 -08:00
Dan Helfman	ded042d8cc	First crack at MySQL/MariaDB database restore (#228).	2019-11-11 21:59:30 -08:00
Dan Helfman	9d29ecf304	Remove MySQL/MariaDB database dumps after backing them up (#228).	2019-11-08 11:53:27 -08:00
Dan Helfman	427b57e2a9	Database dump hooks for MySQL/MariaDB (#228).	2019-11-08 11:17:52 -08:00
Dan Helfman	17fda7281a	Monitor backups with Cronhub hook integration. Fix Healthchecks/Cronitor hooks to respect dry run.	2019-11-07 10:08:44 -08:00
Dan Helfman	8fd46b8c70	Monitor backups with Cronitor hook integration.	2019-11-01 11:33:15 -07:00
Dan Helfman	939f4d4e3d	Add additional tests for database restores.	2019-10-31 14:11:19 -07:00
Dan Helfman	3006db0cae	Restore backed up PostgreSQL databases via "borgmatic restore" sub-command (#229).	2019-10-31 12:57:36 -07:00