borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
14
57
Fork 41
Code Issues 53 Pull Requests 2 Actions Packages Releases 122 Activity
2,235 Commits 2 Branches 164 Tags 9.9 MiB
Commit Graph

646 Commits

Author SHA1 Message Date
Jakub Jirutka dfccc1b94a Exit on error when restoring all PostgreSQL databases 
"--set ON_ERROR_STOP=on" is equivalent to "--exit-on-error" in
pg_restore.
 2023-04-14 17:38:18 +02:00
Jakub Jirutka 19a00371f5 Run "psql" with "--no-psqlrc" 
Some settings in user's .psqlrc, e.g. "linestyle unicode", may break the
CSV output. "--no-psqlrc" tells psql to not read startup file.

This is not necessary for the analyze_command and restore_command (with
all_databases), but it's generally recommended when running psql from a
script.
 2023-04-14 17:37:37 +02:00
Dan Helfman 50b0a9ce38 Remove newline at end of file.
continuous-integration/drone/push Build is failing Details
2023-04-13 19:13:50 -07:00
polyzen ebe5c5e839 Fix "TypeError: 'module' object is not callable" in test_commands.py 2023-04-14 01:01:31 +00:00
Dan Helfman 4a94c2c9bf Selectively omit list values when including configuration files (#672).
continuous-integration/drone/push Build is passing Details
2023-04-13 14:39:36 -07:00
Dan Helfman 1ea4433aa9 Selectively shallow merge certain mappings or sequences when including configuration files (#672).
continuous-integration/drone/push Build is failing Details
2023-04-11 21:49:10 -07:00
Dan Helfman 4c0e2cab78 View the results of configuration file merging via "validate-borgmatic-config --show" flag (#673).
continuous-integration/drone/push Build is passing Details
2023-04-11 10:49:09 -07:00
Dan Helfman 31a2ac914a Add optional support for running end-to-end tests and building documentation with rootless Podman instead of Docker.
continuous-integration/drone/push Build is passing Details
2023-04-10 14:26:54 -07:00
Dan Helfman b4b1fa939d
feat: add logfile name to hook context for interpolation 
Merge pull request #68 from diivi/feat/add-log-filename-to-hook-context
 2023-04-06 13:46:45 -07:00
Divyansh Singh 16d7131fb7 refactor tests 2023-04-07 01:00:38 +05:30
Divyansh Singh 091d60c226 refactor and improve tests 2023-04-06 12:36:10 +05:30
Divyansh Singh 0fbdf8d860 feat: add logfile name to hook context for interpolation 2023-04-06 09:31:24 +05:30
Dan Helfman 192bfe46a9 Fix error when running the "prune" action with both "archive_name_format" and "prefix" options set (#668).
continuous-integration/drone/push Build is passing Details
2023-04-05 14:58:05 -07:00
Dan Helfman a9a65ebe54 Fix integration tests to actually assert (#666).
continuous-integration/drone/push Build is passing Details
2023-04-04 22:11:36 -07:00
Dan Helfman 616eb6b6da Fix error with "info --match-archives" and fix "--match-archives" overriding logic (#666).
continuous-integration/drone/push Build is passing Details
2023-04-04 21:25:10 -07:00
Dan Helfman 7e6bee84b0 Add "--log-file-format" flag for customizing the log message format (#658).
continuous-integration/drone/push Build is passing Details
2023-04-02 23:06:36 -07:00
Dan Helfman 9712d00680 Add "match_archives" option (#588).
continuous-integration/drone/push Build is passing Details
2023-04-01 23:57:55 -07:00
Divyansh Singh 32ab17fa46 merge 2023-04-01 22:12:41 +05:30
Divyansh Singh 6054ced931 fix: run typos 2023-04-01 22:10:32 +05:30
kxxt fa8bc285c8 Fix randomly failing test. 2023-04-01 14:02:30 +08:00
Dan Helfman 3f78ac4085 Automatically use the "archive_name_format" option to filter which archives get used for borgmatic actions that operate on multiple archives (#479).
continuous-integration/drone/push Build is passing Details
2023-03-31 15:21:08 -07:00
Dan Helfman 5f595f7ac3 Fix regression in which the "transfer" action produced a traceback (#663).
continuous-integration/drone/push Build is passing Details
2023-03-30 23:21:20 -07:00
Dan Helfman fc2c181b74 Add missing Docker Compose depends.
continuous-integration/drone/push Build is passing Details
2023-03-28 15:31:37 -07:00
Dan Helfman f709125110 Error out if run-full-tests is run not inside a test container.
continuous-integration/drone/push Build encountered an error Details
2023-03-28 12:02:07 -07:00
Dan Helfman 3512191f3e Add check_repositories regression fix to NEWS (#662).
continuous-integration/drone/push Build is passing Details
2023-03-28 11:45:55 -07:00
Divyansh Singh 08e358e27f add and update tests 2023-03-28 22:51:35 +05:30
Dan Helfman 3e8e38011b
Labels for repositories (#635). 
Merge pull request #57 from diivi/feat/tag-repos
 2023-03-27 15:46:22 -07:00
Dan Helfman d0d3a39833 When a database command errors, display and log the error message instead of swallowing it (#396).
continuous-integration/drone/push Build is passing Details
2023-03-27 10:36:39 -07:00
Divyansh Singh b336b9bedf add tests for repo labels 2023-03-27 00:19:23 +05:30
Divyansh Singh ec9def4e71 rename repository arg to repository_path in all borg actions 2023-03-26 23:52:25 +05:30
Divyansh Singh a136fda92d check all tests 2023-03-26 23:35:47 +05:30
Divyansh Singh b511e679ae remove optional label for repos from tests 2023-03-26 16:59:29 +05:30
Dan Helfman 1c51a8e229
Allow defining custom variables in config file (#612). 
Merge pull request #60 from diivi/feat/constants-support
 2023-03-24 22:50:57 -07:00
Dan Helfman 739a58fe47 Rename scripts/run-full-dev-tests to scripts/run-end-to-end-dev-tests and make it run end-to-end tests only.
continuous-integration/drone/push Build is passing Details
2023-03-24 16:24:00 -07:00
Divyansh Singh af95134cd2 add test for complex constant 2023-03-25 02:03:36 +05:30
Divyansh Singh d6dfb8753a reformat 2023-03-25 01:50:47 +05:30
Divyansh Singh 1bc003560a Merge branch 'master' of https://github.com/diivi/borgmatic into feat/tag-repos 2023-03-25 01:39:26 +05:30
Divyansh Singh aeaf69f49e pass all tests 2023-03-25 01:34:03 +05:30
Divyansh Singh e83ad9e1e4 use repository["path"] instead of repository 2023-03-25 01:04:57 +05:30
Dan Helfman f42890430c Add code style plugins to enforce use of Python f-strings and prevent single-letter variables.
continuous-integration/drone/push Build is passing Details
2023-03-23 23:11:14 -07:00
Divyansh Singh 6f300b0079 feat: constants support 2023-03-24 02:39:37 +05:30
Dan Helfman 0a72c67c6c Add missing source directory error fix to NEWS (#655).
continuous-integration/drone/push Build is passing Details
2023-03-22 13:02:22 -07:00
Divyansh Singh 1e3a3bf1e7 review 2023-03-23 01:18:06 +05:30
Divyansh Singh bd235f0426 use exit_code_indicates_error and modify it to accept a command 2023-03-22 16:23:53 +05:30
Divyansh Singh 09183464cd fix: no error on database backups without source dirs 2023-03-22 09:41:39 +05:30
Dan Helfman f9ef52f9a5 Remove unused module and outdated test expectations (#576).
continuous-integration/drone/push Build is passing Details
2023-03-21 10:29:17 -07:00
Divyansh Singh 39ad8f64c4 add tests and remove magic number 2023-03-21 17:06:03 +05:30
Dan Helfman 55c153409e Add "source_directories_must_exist" option to NEWS (#501).
continuous-integration/drone/push Build is passing Details
2023-03-18 14:07:38 -07:00
Dan Helfman b115fb2fbe Merge branch 'master' of github.com:borgmatic-collective/borgmatic 2023-03-18 14:01:52 -07:00
Dan Helfman 31d04d9ee3
Optionally error if a source directory does not exist. 
feat: add optional check for existence of source directories
 2023-03-18 13:59:20 -07:00
Divyansh Singh f803836416 reformat 2023-03-18 17:27:33 +05:30
Divyansh Singh 997f60b3e6 add tests 2023-03-18 17:24:21 +05:30
Soumik Dutta fb9677230b add test to ensure integers are converted to string 
before setting them up to be environment variable values

Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-18 02:57:56 +05:30
Dan Helfman 7605838bfe Add "--repository" flag to all actions where it makes sense (#564).
continuous-integration/drone/push Build is passing Details
2023-03-16 13:27:08 -07:00
Nain 3e22414613 Update tests 
Make them more explicit. Also formatting.
 2023-03-16 14:01:29 -04:00
Nain 5f87ea3ec5 Add "--repository" flag to the "create" action 2023-03-16 13:15:49 -04:00
Nain a8aeace5b5 Add "--repository" flag to the "compact" action 2023-03-16 11:13:45 -04:00
Nain 480addd7ce Add "--repository" flag to the "check" action 2023-03-16 10:41:13 -04:00
Nain ce0ce4cd1c Merge mostly repetetive tests 2023-03-16 08:23:21 -04:00
Nain 7de9260b0d Remove test now that --repository isn't expected to error 
As discussed #652#issuecomment-5579
 2023-03-15 14:59:12 -04:00
Nain cdbe6cdf3a Add "--repository" flag to the "prune" action 
part of ticket #564
 2023-03-15 14:43:17 -04:00
Dan Helfman 8cec7c74d8 Add "--strip-components all" on the "extract" action to remove leading path components (#647).
continuous-integration/drone/push Build is passing Details
2023-03-09 10:09:16 -08:00
Dan Helfman b343363bb8 Change the default action order to: "create", "prune", "compact", "check" (#304).
continuous-integration/drone/push Build is passing Details
2023-03-08 14:05:06 -08:00
Dan Helfman 9db31bd1e9 Run any command-line actions in the order specified instead of using a fixed ordering (#304).
continuous-integration/drone/push Build is passing Details
2023-03-08 13:19:41 -08:00
Dan Helfman d88bcc8be9 Add Healthchecks "log" state feature to NEWS.
continuous-integration/drone/push Build is passing Details
2023-03-07 15:45:23 -08:00
Dan Helfman 332f7c4bb6 Add support for healthchecks "log" feature (#628).
continuous-integration/drone/push Build is failing Details 
Reviewed-on: #645
 2023-03-07 22:21:30 +00:00
Dan Helfman 5d19d86e4a Add flake8-quotes to complain about incorrect quoting so I don't have to!
continuous-integration/drone/push Build is passing Details
2023-03-07 14:08:35 -08:00
Soumik Dutta 044ae7869a fix tests 
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-08 03:30:12 +05:30
Soumik Dutta 98e429594e added tests to make sure unsupported log states are detected 
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-06 20:31:00 +05:30
Soumik Dutta f442aeae9c fix logs_monitor_start_error() 
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-06 05:21:56 +05:30
Soumik Dutta e211863cba update test_borgmatic.py 
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-06 05:12:24 +05:30
Soumik Dutta 45256ae33f add test for healthchecks 
Signed-off-by: Soumik Dutta <shalearkane@gmail.com>
 2023-03-06 03:38:08 +05:30
Dan Helfman 9ec220c600
Add SQLite database dump/restore hook (#295). 
feat: add dump-restore support for sqlite databases
 2023-03-04 09:47:21 -08:00
Divyansh Singh c71eb60cd2 mock os.remove instead of actually removing a file 2023-03-04 13:08:30 +05:30
Divyansh Singh 675e54ba9f use os.remove and improve tests 2023-03-04 12:43:07 +05:30
Divyansh Singh 767a7d900b e2e tests schema update 2023-03-04 01:29:01 +05:30
Divyansh Singh 903507bd03 code review 2023-03-04 01:27:07 +05:30
Dan Helfman a071e02d20 With the "create" action and the "--list" ("--files") flag, only show excluded files at verbosity 2 (#620).
continuous-integration/drone/push Build is failing Details
2023-03-02 15:33:42 -08:00
Divyansh Singh 3aa88085ed formatting fix 2023-03-03 00:01:52 +05:30
Divyansh Singh af1cc27988 feat: add dump-restore support for sqlite databases 2023-03-02 23:55:16 +05:30
Dan Helfman dbf8301c19 Add "checkpoint_volume" configuration option to creates checkpoints every specified number of bytes.
continuous-integration/drone/push Build is passing Details
2023-02-27 10:47:17 -08:00
Dan Helfman 2a306bef12 Fix tests.
continuous-integration/drone/push Build is passing Details
2023-02-26 23:34:17 -08:00
Dan Helfman d7a07f0428 Support status character changes in Borg 2.0.0b5 when filtering out special files that cause Borg to hang.
continuous-integration/drone/push Build is passing Details
2023-02-26 22:36:13 -08:00
Dan Helfman da321e180d Fix the "create" action with the "--dry-run" flag querying for databases when a PostgreSQL/MySQL "all" database is configured.
continuous-integration/drone/push Build is passing Details
2023-02-26 22:15:12 -08:00
Dan Helfman c6582e1171 Internally support new Borg 2.0.0b5 "--filter" status characters / item flags for the "create" action.
continuous-integration/drone/push Build is passing Details
2023-02-26 17:17:25 -08:00
Dan Helfman 9b83afe491 With the "create" action, only one of "--list" ("--files") and "--progress" flags can be used.
continuous-integration/drone/push Build is passing Details
2023-02-26 17:05:56 -08:00
Tom Hubrecht 95575c3450 Add auth test for the ntfy hook 2023-02-25 20:04:39 +01:00
Dan Helfman 418ebc8843 Add MySQL database hook "add_drop_database" configuration option to control whether dumped MySQL databases get dropped right before restore (#642).
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2023-02-20 15:32:47 -08:00
Dan Helfman f5a448c7c2 Fix for potential data loss (data not getting backed up) when dumping large "directory" format PostgreSQL/MongoDB databases (#643).
continuous-integration/drone/push Build is passing Details
2023-02-20 15:18:51 -08:00
Dan Helfman c1eb210253 Fix code style flake issue.
continuous-integration/drone/push Build is passing Details
2023-01-26 15:09:35 -08:00
Dan Helfman 30cca62d09 Add configuration options for database command customization (#630).
continuous-integration/drone/push Build is failing Details
2023-01-26 14:59:17 -08:00
Dan Helfman 0e6b2c6773 Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).
continuous-integration/drone/push Build is passing Details
2023-01-25 23:31:07 -08:00
palto42 fd7ad86daa
conditional warning for excluding special files 2023-01-03 21:53:51 +01:00
Dan Helfman f67c544be6 Optionally dump "all" PostgreSQL databases to separate files instead of one combined dump file (#438, #560).
continuous-integration/drone/push Build is passing Details
2022-12-15 22:59:42 -08:00
Dan Helfman c657764367 Fix logs that interfere with JSON output by making warnings go to stderr instead of stdout (#602).
continuous-integration/drone/push Build is passing Details
2022-12-02 12:12:10 -08:00
Dan Helfman ca7e18bb29
Override PostgreSQL dump/restore commands via configuration options (#311). 
Merge pull request #49 from jpaniagualaconich/specify-pg-dump-restore-commands
 2022-11-18 08:33:14 -08:00
Dan Helfman 6975a5b155 Fix "data" consistency check to support "check_last" and consistency "prefix" options (#611).
continuous-integration/drone/push Build is passing Details
2022-11-17 10:19:48 -08:00
Javier Paniagua faf682ca35 specify pg dump/restore commands (#311) 2022-11-06 11:12:53 +01:00
Dan Helfman 89500df429 Fix traceback when a configuration section is present but lacking any options (#604).
continuous-integration/drone/push Build is passing Details
2022-10-23 13:56:03 -07:00
Dan Helfman 9c42e7e817 Fix regression in which "check" action errored on certain systems (#597, #598).
continuous-integration/drone/push Build is failing Details
continuous-integration/drone/tag Build is passing Details
2022-10-14 16:19:26 -07:00
Dan Helfman 953277a066 Fix special file detection when broken symlinks are encountered (#596).
continuous-integration/drone/push Build is passing Details
2022-10-14 09:41:08 -07:00
Dan Helfman d7f1c10c8c To prevent Borg hangs, unconditionally delete stale named pipes before dumping databases (#360).
continuous-integration/drone/push Build is passing Details
2022-10-12 10:26:09 -07:00
Dan Helfman a31ce337e9 Skip auto-exclusion of special files when user explicitly sets read_special to true (#587).
continuous-integration/drone/push Build is passing Details
2022-10-06 11:07:43 -07:00
Dan Helfman c31702d092 Fix for potential data loss with "patterns_from". Also, display excluded files (#590).
continuous-integration/drone/push Build is passing Details
2022-10-04 22:57:18 -07:00
Dan Helfman ba8fbe7a44 Add "break-lock" action for removing any repository and cache locks leftover from Borg aborting (#357).
continuous-integration/drone/push Build is passing Details
2022-10-04 13:42:18 -07:00
Dan Helfman 2774c2e4c0 Add support for Borg 2's "--match-archives" flag (replaces "--glob-archives") (#591).
continuous-integration/drone/push Build is passing Details
2022-10-03 22:50:37 -07:00
Dan Helfman ae036aebd7 When the "read_special" option is true or database hooks are enabled, auto-exclude special files for a "create" action to prevent Borg from hanging (#587).
continuous-integration/drone/push Build is passing Details
2022-10-03 12:58:13 -07:00
Dan Helfman ac7c7d4036 Warn when ignoring a configured "read_special" value of false, as true is needed when database hooks are enabled (#587).
continuous-integration/drone/push Build is failing Details
2022-09-20 13:52:13 -07:00
Dan Helfman 679bb839d7 Fix hang when database hooks are enabled and "patterns" contains a parent directory of "~/.borgmatic" (#582).
continuous-integration/drone/push Build is passing Details
2022-09-08 10:16:42 -07:00
Dan Helfman 9e64d847ef Fix regression in which "borgmatic info --archive ..." showed repository info instead of archive info with Borg 1 (#577).
continuous-integration/drone/push Build is passing Details
2022-08-30 20:42:42 -07:00
Dan Helfman 87c97b7568 Fixed spurious, intermittent test failures related to command execution and logging.
continuous-integration/drone/push Build is passing Details
2022-08-28 09:06:06 -07:00
Dan Helfman c46f2b8508 Fix conflict between "patterns" and "source_directories" (#574), make "source_directories" optional (#542). 2022-08-25 11:55:34 -07:00
Dan Helfman ef7e95e22a Fix end-to-end tests.
continuous-integration/drone/push Build is passing Details
2022-08-21 23:29:13 -07:00
Dan Helfman 317dc7fbce Add "before_actions" and "after_actions" command hooks that run before/after all the actions for each repository, update docs to cover per-repository configurations (#463).
continuous-integration/drone/push Build is passing Details
2022-08-21 21:48:37 -07:00
Dan Helfman 462326406e Drop only-style actions like "--create", rename "prune --files" to "prune --list", and add "--list" alias to "create" and "export-tar" (#571).
continuous-integration/drone/push Build is passing Details
2022-08-21 14:25:16 -07:00
Dan Helfman 540f9f6b72 Add missing test for "transfer" action (#557).
continuous-integration/drone/push Build is passing Details
2022-08-19 09:40:29 -07:00
Dan Helfman f9d7faf884 Fix mount action to work without archive again (#557).
continuous-integration/drone/push Build is passing Details
2022-08-18 23:33:05 -07:00
Dan Helfman 7dee6194a2 Add new "transfer" action for Borg 2 (#557).
continuous-integration/drone/push Build is passing Details
2022-08-18 23:06:51 -07:00
Dan Helfman 68f9c1b950 Add generate-borgmatic-config end-to-end test.
continuous-integration/drone/push Build is passing Details
2022-08-18 14:28:46 -07:00
Dan Helfman 43d711463c Add additional command-line flags to rcreate action (#557). 2022-08-18 14:28:12 -07:00
Dan Helfman b40e9b7da2 Ignore archive filter parameters passed to list action when --archive is given (#557).
continuous-integration/drone Build is passing Details
2022-08-18 09:59:48 -07:00
Dan Helfman 89d201c8ff Fleshing out NEWS for the Borg 2 changes.
continuous-integration/drone/push Build is passing Details
2022-08-17 21:54:00 -07:00
Dan Helfman f47c98c4a5 Rename several configuration options to match Borg 2 (#557).
continuous-integration/drone/push Build is passing Details
2022-08-17 21:14:58 -07:00
Dan Helfman 3b6ed06686 Add --other-repo flag to rcreate action (#557).
continuous-integration/drone/push Build is passing Details
2022-08-17 17:33:09 -07:00
Dan Helfman 57009e22b5 Use flag-related utility functions in info action (#557).
continuous-integration/drone/push Build is running Details
2022-08-17 17:11:02 -07:00
Dan Helfman 3ab7a3b64a Replace use of --prefix with --glob-archives in info action (#557).
continuous-integration/drone/push Build is passing Details
2022-08-17 15:36:19 -07:00
Dan Helfman 596dd49cf5 Use --glob-archives instead of --prefix on rlist command (#557).
continuous-integration/drone/push Build is running Details
2022-08-17 14:26:35 -07:00
Dan Helfman 28d847b8b1 Warn and tranform on non-ssh://-style repositories (#557).
continuous-integration/drone/push Build is passing Details
2022-08-17 10:13:11 -07:00
Dan Helfman 30abd0e3de Update borg action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-16 09:30:00 -07:00
Dan Helfman f36e38ec20 Update mount action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-15 19:32:37 -07:00
Dan Helfman d807ce095e Update export-tar action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-15 17:34:12 -07:00
Dan Helfman 7626fe1189 Disallow borg list --json with --archive or --find (#557).
continuous-integration/drone/push Build is passing Details
2022-08-15 15:40:28 -07:00
Dan Helfman cc04bf57df Update list action for Borg 2 support, add rinfo action, and update extract consistency check for Borg 2.
continuous-integration/drone/push Build is passing Details
2022-08-15 15:04:40 -07:00
Dan Helfman cce6d56661 Update extract action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-13 23:07:29 -07:00
Dan Helfman a05d0f378e Factor out repository/archive flags formatting code from create action (#557).
continuous-integration/drone/push Build is passing Details
2022-08-13 22:50:14 -07:00
Dan Helfman 94321aec7a Update compact action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-13 22:07:15 -07:00
Dan Helfman 4a55749bd2 Update prune action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-13 17:26:51 -07:00
Dan Helfman 2898e63166 Update create action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-12 23:54:13 -07:00
Dan Helfman c7176bd00a Add rinfo action for Borg 2 support (#557).
continuous-integration/drone/push Build is passing Details
2022-08-12 23:06:56 -07:00
Dan Helfman 647ecdac29 Borg 2 support in borgmatic check action (#557).
continuous-integration/drone/push Build is passing Details
2022-08-12 15:46:33 -07:00
Dan Helfman e7a8acfb96 Add missing rinfo action source files (#557).
continuous-integration/drone/push Build is passing Details
2022-08-12 14:59:03 -07:00
Dan Helfman 622caa0c21 Support for Borg 2's rcreate and rinfo sub-commands (#557).
continuous-integration/drone/push Build is failing Details
2022-08-12 14:53:20 -07:00
Dan Helfman 9aece3936a Modify "mount" and "extract" actions to require the "--repository" flag when multiple repositories are configured (#566).
continuous-integration/drone/push Build is passing Details
2022-07-25 11:30:02 -07:00
Dan Helfman bcad0de1a4
Add verify_tls option for Healthchecks to optionally disable TLS verification. 2022-07-23 23:11:41 -07:00
Uli 5c6407047f feat: add verify_tls flag for Healthchecks 2022-07-24 07:37:00 +02:00
Dan Helfman 6ddae20fa1 Fix handling of "repository" and "data" consistency checks to prevent invalid Borg flags (#565).
continuous-integration/drone/push Build is passing Details
2022-07-23 21:02:21 -07:00
Dan Helfman 16066942e3 Fix traceback with "create" action and "--json" flag when a database hook is configured (#563).
continuous-integration/drone/push Build is passing Details
2022-07-19 10:25:10 -07:00
Dan Helfman e85d551eac Fix all database hooks to error when the requested database to restore isn't present in the Borg archive (#560).
continuous-integration/drone/push Build is passing Details
2022-07-06 23:21:24 -07:00
Dan Helfman 2b23a63a08 Add end-to-end test for overrides.
continuous-integration/drone/push Build is passing Details
2022-07-06 18:20:51 -07:00
Dan Helfman 457ed80744 Fix environment variable plumbing so options in one configuration file aren't used for others (#555).
continuous-integration/drone/push Build is passing Details
2022-06-30 13:42:17 -07:00
Dan Helfman 10723efc68 Fix all monitoring hooks to warn if the server returns an HTTP 4xx error (#554).
continuous-integration/drone/push Build is passing Details
2022-06-29 21:19:40 -07:00
Dan Helfman bd4d109009 Fix logging to include the full traceback when Borg experiences an internal error (#553).
continuous-integration/drone/push Build is passing Details
2022-06-28 13:38:24 -07:00
Dan Helfman d372a86fe6 Code formatting.
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build encountered an error Details
2022-06-23 10:41:04 -07:00
Sébastien MB b63c854509 Fix escaped environment variable in configuration 
- when an env variable is escaped in the configuration file, we expect
  not to resolve it and remove the escape char `\`
 2022-06-17 09:50:56 +02:00
Dan Helfman aecb6fcd74 Code style, rename command-line flag, and move new code into its own file (#546)
continuous-integration/drone/push Build is failing Details
2022-06-16 11:35:24 -07:00
Sébastien MB 97b5cd089d Allow environment variable resolution in configuration file 
- all string fields containing an environment variable like ${FOO} will
  be resolved
- supported format ${FOO}, ${FOO:-bar} and ${FOO-bar} to allow default
  values if variable is not present in environment
- add --no-env argument for CLI to disable the feature which is enabled
  by default

Resolves: #546
 2022-06-16 18:52:54 +02:00
Gavin Chappell a8b8d507b6
add a hook for sending push notifications via ntfy.sh 2022-06-09 21:10:38 +01:00
Dan Helfman 3561c93d74 Fix Healthchecks tests that leak global state, breaking downstream tests (discovered in #543).
continuous-integration/drone/push Build is passing Details
2022-06-09 11:05:44 -07:00
Dan Helfman 9aefb5179f Fix None find paths (#541).
continuous-integration/drone/push Build is passing Details
2022-06-03 15:20:05 -07:00
Dan Helfman d14f22e121 Add "borgmatic list --find" flag for searching for files across multiple archives (#541).
continuous-integration/drone/push Build is failing Details
2022-06-03 15:12:14 -07:00
Dan Helfman 80ec3e7d97 Deprecate "borgmatic list --successful" flag, as listing only non-checkpoint (successful) archives is now the default in newer versions of Borg.
continuous-integration/drone/push Build is passing Details
2022-06-02 20:35:39 -07:00
Dan Helfman 2bc91ac3d2 Add "generate-borgmatic-config --overwrite" flag to replace an existing destination file (#539).
continuous-integration/drone/push Build is passing Details
2022-05-29 16:03:55 -07:00
Dan Helfman 5b615d51a4 Add support for "borgmatic borg debug" command (#538).
continuous-integration/drone/push Build is passing Details
2022-05-29 15:43:03 -07:00
Dan Helfman 8fa90053cf Add "borgmatic check --force" flag to ignore configured check frequencies (#523). 2022-05-28 19:29:33 -07:00
Dan Helfman ad0e2e0d7c Tweak default check frequency to 1 month (#523). 2022-05-28 15:49:50 -07:00
Dan Helfman e76bfa555f Reduce the default consistency check frequency and support configuring the frequency independently for each check (#523). 2022-05-28 14:42:19 -07:00
Dan Helfman 8ddb7268eb Reuse "borg info" function.
continuous-integration/drone/push Build is passing Details
2022-05-27 13:51:11 -07:00
Dan Helfman cb5fe02ebd Fix broken Bash completion end-to-end test.
continuous-integration/drone/push Build is passing Details
2022-05-26 11:18:46 -07:00
Dan Helfman 77b84f8a48 Add Bash completion script so you can tab-complete the borgmatic command-line.
continuous-integration/drone/push Build is failing Details
2022-05-26 10:27:53 -07:00
Dan Helfman be59a3e574 Fix generate-borgmatic-config with "--source" flag to support more complex schema changes like the new Healthchecks configuration options (#536).
continuous-integration/drone/push Build is passing Details
2022-05-25 10:26:26 -07:00
Dan Helfman 54933ebef5 Change connection failures for monitoring hooks to be warnings instead of errors (#439).
continuous-integration/drone/push Build is passing Details
2022-05-24 15:50:04 -07:00
Dan Helfman 157e59ac88 Add Healthchecks monitoring hook "send_logs" option to enable/disable sending borgmatic logs to the Healthchecks server (#460).
continuous-integration/drone/push Build is passing Details
2022-05-24 14:44:33 -07:00
Dan Helfman 8b179e4647 Reverse logic of Healtchecks "skip_states" option to just "states" (#525).
continuous-integration/drone/push Build is failing Details
2022-05-24 14:09:42 -07:00
Dan Helfman 865eff7d98 Add Healthchecks monitoring hook "skip_states" option to disable pinging for particular monitoring states (#525).
continuous-integration/drone/push Build is failing Details
2022-05-24 13:59:28 -07:00
Dan Helfman b9741f4d0b Add Healthchecks monitoring hook "ping_body_limit" option to configure how many bytes of logs to send to the Healthchecks server (#294).
continuous-integration/drone/push Build is passing Details
2022-05-24 12:23:38 -07:00
Dan Helfman 02781662f8 Change monitoring hooks to specify the ping URL / integration key as a named option.
continuous-integration/drone/push Build is passing Details
2022-05-23 20:02:10 -07:00
Dan Helfman 32a1043468 Remove the error when "archive_name_format" is specified but a retention prefix isn't (#402).
continuous-integration/drone/push Build is passing Details
2022-05-23 16:11:24 -07:00
Dan Helfman 3e4aeec649 Warn when an unsupported variable is used in a hook command (#420).
continuous-integration/drone/push Build is passing Details
2022-05-23 15:27:54 -07:00
Dan Helfman 255cc6ec23 When deep merging common configuration, merge colliding list values by appending them (#531).
continuous-integration/drone/push Build is passing Details
2022-05-20 15:28:28 -07:00
Dan Helfman 788281cfb9 When a configuration include is a relative path, load it from either the current working directory or from the directory containing the file doing the including (#532).
continuous-integration/drone/push Build is passing Details
2022-05-19 17:15:05 -07:00
Dan Helfman 525266ede6 Deep merging when including common configuration (#381).
continuous-integration/drone/push Build is passing Details
2022-04-25 21:18:37 -07:00
Dan Helfman 0e6b425ac5 Fix "borgmatic borg key ..." to pass parameters to Borg in correct order (#515).
continuous-integration/drone/push Build is passing Details
2022-04-23 14:03:15 -07:00
Dan Helfman ed7fe5c6d0 Instead of executing "before" command hooks before all borgmatic actions run (and "after" hooks after), execute these hooks right before/after the corresponding action (#473).
continuous-integration/drone/push Build is passing Details
2022-04-21 22:08:25 -07:00
Dan Helfman 0c027a3050 Fix handling of TERM signal to exit borgmatic, not just forward the signal to Borg (#516).
continuous-integration/drone/push Build is passing Details
2022-04-03 13:12:48 -07:00
Dan Helfman 9f44bbad65 Fix borgmatic exit code (so it's zero) when initial Borg calls fail but later retries succeed (#517).
continuous-integration/drone/push Build is passing Details
2022-04-02 22:28:41 -07:00
Dan Helfman 976516d0e1 When loading a configuration file that is unreadable due to file permissions, warn instead of erroring (#444).
continuous-integration/drone/push Build is passing Details
2022-03-08 10:19:36 -08:00
Dan Helfman 574eb91921 Fix Borg usage error in the "compact" action when running "borgmatic --dry-run". Now, skip "compact" entirely during a dry run (#507).
continuous-integration/drone/push Build is passing Details
2022-03-07 21:46:12 -08:00
Dan Helfman 28fef3264b Fix handling of "patterns_from" and "exclude_from" options to error instead of warning when referencing unreadable files and running "create" action (#486).
continuous-integration/drone/push Build is passing Details
2022-03-07 15:32:07 -08:00
Dan Helfman 9161dbcb7d Removing unnecessary leading underscores from functions.
continuous-integration/drone/push Build is passing Details
2022-03-07 11:58:29 -08:00
Dan Helfman 4b3027e4fc Add test for new working_directory option (#431).
continuous-integration/drone/push Build is passing Details
2022-03-03 11:48:18 -08:00
Dan Helfman 0eb2634f9b Working directory option to support source directories with relative paths (#431).
continuous-integration/drone/push Build is failing Details 
Reviewed-on: #477
 2022-03-03 19:28:17 +00:00
Dan Helfman 9317cbaaf0 Code formatting.
continuous-integration/drone/push Build is passing Details
2022-02-10 10:23:34 -08:00
Dan Helfman 1b5f04b79f When using the "remote_rate_limit" option, tailor the flags passed to Borg depending on the Borg version (#394).
continuous-integration/drone/push Build is failing Details
2022-02-10 10:16:09 -08:00
Dan Helfman 948c86f62c When using the "numeric_owner" option with the "extract" action, tailor the flags passed to Borg depending on the Borg version (#394).
continuous-integration/drone/push Build is passing Details
2022-02-10 10:09:18 -08:00
Dan Helfman 7e7209322a When using the "numeric_owner" option, tailor the flags passed to Borg depending on the Borg version (#394).
continuous-integration/drone/push Build is passing Details
2022-02-10 09:51:13 -08:00
Dan Helfman 00a57fd947 Code formatting.
continuous-integration/drone/push Build is passing Details
2022-02-09 21:20:28 -08:00
Dan Helfman 6bf6ac310b When using the "bsd_flags" option, tailor the flags passed to Borg depending on the Borg version (#394).
continuous-integration/drone/push Build is failing Details
2022-02-09 21:11:00 -08:00
Dan Helfman 4b5af2770d When the "atime" option is used, tailor the flags passed to Borg depending on version (#394).
continuous-integration/drone/push Build is passing Details
2022-02-09 16:54:35 -08:00
Dan Helfman b525e70e1c Run "compact" action by default when no actions are specified (#394). 2022-02-09 14:33:12 -08:00
Dan Helfman e3f4bf0293 Build fix for ruamel.yaml.clib error.
continuous-integration/drone/push Build is failing Details
2022-02-08 12:52:45 -08:00
Dan Helfman 48f44d2f3d Add tests for compact action (#394). 2022-02-08 12:05:02 -08:00
Dan Helfman bec73245e9 Fix traceback when a YAML validation error occurs (#480, #482).
continuous-integration/drone/push Build is passing Details
2022-01-19 20:39:03 -08:00
fabianschilling b39f08694d Merge branch 'master' into pr-working-directory
continuous-integration/drone/pr Build is passing Details
2022-01-05 09:30:27 +00:00
Dan Helfman 2ee75546f5 Add MongoDB database hook documentation.
continuous-integration/drone/push Build is passing Details
2022-01-04 16:26:38 -08:00
Andrea Ghensi 7c6ce9399c fix integration tests and mongodb auth
continuous-integration/drone/pr Build is failing Details
2021-12-29 22:18:50 +01:00
Andrea Ghensi 6b7653484b Add mongodb dump hook
continuous-integration/drone/pr Build is failing Details
2021-12-26 01:00:58 +01:00
Fabian Schilling 85e0334826 Add missing working_directory arg to pass tests
continuous-integration/drone/pr Build is passing Details
2021-12-10 18:24:41 +01:00
Dan Helfman a1673d1fa1 Fix unicode error when restoring particular MySQL databases (#476).
continuous-integration/drone/push Build is passing Details
2021-12-08 16:40:25 -08:00
Dan Helfman 7a086d8430 Fix import ordering.
continuous-integration/drone/push Build was killed Details
2021-11-29 14:00:14 -08:00
Dan Helfman 0e8e9ced64 When command-line configuration override produces a parse error, error cleanly (#471).
continuous-integration/drone/push Build is failing Details
2021-11-29 12:49:21 -08:00
nebulon42 c3e76585fc
move mysqldump options to the beginning of the command due to MySQL bug 30994.
continuous-integration/drone/pr Build is passing Details
2021-11-26 17:16:03 +01:00
nebulon42 3729ba5ca3
add list_options setting, fixes #306
continuous-integration/drone/pr Build is passing Details
2021-11-20 15:43:58 +01:00
Dan Helfman 38ebfd2969 Rename retry_timeout to retry_wait and standardize log formatting (#28).
continuous-integration/drone/push Build is passing Details
2021-11-15 11:51:17 -08:00
cadamswaite 976a877a25 Formatting
continuous-integration/drone/pr Build is failing Details
2021-11-14 22:37:42 +00:00
cadamswaite b4117916b8 Add timeout and tests 2021-11-14 22:15:22 +00:00
cadamswaite 19cad89978 Add some tests for retry logic 2021-11-14 21:35:23 +00:00
Dan Helfman 449896f661 Fix error when configured source directories are not present on the filesystem at the time of backup (#387).
continuous-integration/drone/push Build is passing Details
2021-10-11 10:40:10 -07:00
Dan Helfman acb2ca79d9 Fix traceback that can occur when dumping a database (#440).
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2021-08-06 08:58:11 -07:00
Dan Helfman 80b33fbf8a Code style reformatting.
continuous-integration/drone/push Build is passing Details
2021-07-27 09:39:48 -07:00
Marek Szuba e8b8d86592 tests/integration/test_execute: use plain Python rather than xxd 
Removes this test's dependencies on vim and /dev/urandom.

Signed-off-by: Marek Szuba <marek.szuba@cern.ch>
 2021-07-27 13:50:16 +01:00
Dan Helfman 2a8692c64f Fix integration test to hopefully work on Alpine (#430).
continuous-integration/drone/push Build is failing Details
2021-07-25 22:50:00 -07:00
Dan Helfman 1709f57ff0 Fix hang when restoring a PostgreSQL "tar" format database dump (#430).
continuous-integration/drone/push Build is failing Details
2021-07-25 22:30:15 -07:00
Dan Helfman 54d57e1349 Add test for #407: Fix syslog logging on FreeBSD. 2021-06-23 10:21:45 -07:00
Dan Helfman 27d37b606b Better error messages! Switch the library used for validating configuration files (from pykwalify to jsonschema).
continuous-integration/drone/push Build is passing Details
2021-06-22 13:27:59 -07:00
Dan Helfman cf8882f2bc Run arbitrary Borg commands with new "borgmatic borg" action (#425).
continuous-integration/drone/push Build is passing Details
2021-06-17 20:41:44 -07:00
Dan Helfman 9fd28d2eed Fix error handling to error loudly when Borg gets killed due to running out of memory (#423)! 2021-06-08 11:43:55 -07:00
Dan Helfman d6e376d32d Fix end-to-end test broken by change in source directory examples.
continuous-integration/drone/push Build is passing Details
2021-04-18 17:54:54 -07:00
Dan Helfman d543109ef4 "Fix" build failure with Alpine Edge by switching from Edge to Alpine 3.13.
continuous-integration/drone/push Build is passing Details
2021-04-09 15:58:23 -07:00
Dan Helfman 4897a78fd3 Fix database tests broken by PostgreSQL upgrade in Alpine Edge.
continuous-integration/drone/push Build is passing Details
2020-12-24 22:23:09 -08:00
Dan Helfman ad7198ba66 Tweak to test failing on some machines.
continuous-integration/drone/push Build is passing Details
2020-11-26 16:22:42 -08:00
Dan Helfman c0ae01f5d5 Code formatting.
continuous-integration/drone/push Build is passing Details
2020-11-25 17:46:57 -08:00
Dan Helfman ccd1627175 Fix timing-related test error in Alpine Edge.
continuous-integration/drone/push Build is failing Details
2020-11-25 15:48:33 -08:00
Dan Helfman 1f4f28b4dc Drop support for Python 3.5. Only support black code formatter on Python 3.8+.
continuous-integration/drone/push Build is failing Details
2020-11-22 17:27:21 -08:00
Dan Helfman ea6cd53067 Update versions of test dependencies (test_requirements.txt and test containers).
continuous-integration/drone/push Build is failing Details
2020-11-22 14:48:07 -08:00
Dan Helfman 9b819f32f8 Fix traceback when upgrading old INI-style configuration with upgrade-borgmatic-config (#367).
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2020-11-18 15:01:10 -08:00
Dan Helfman b619bde037 Fix broken end-to-end test.
continuous-integration/drone/push Build is passing Details
2020-11-18 14:20:07 -08:00
François Poulain ebac02f118 Mention placeholders for repositories in documentation; fix #369 2020-11-18 20:22:41 +01:00
Dan Helfman 1ea04aedf0 Merge override values when specifying the "--override" flag multiple times (#361).
continuous-integration/drone/push Build is failing Details
2020-10-23 16:06:00 -07:00
Dan Helfman 0e978299cf Fix traceback when a database hook value is null in a configuration file (#355).
continuous-integration/drone/push Build is passing Details
2020-10-07 15:41:45 -07:00
networkjanitor 3c723e8d99 Merge branch 'master' into master 2020-08-12 21:11:45 +00:00
Dan Helfman 72bd96c656 Fix traceback when a configuration directory is non-readable due to directory permissions (#350).
continuous-integration/drone/push Build is passing Details
2020-08-12 11:32:00 -07:00
networkjanitor 893fca2816 Add before_extract and after_extract hooks 2020-08-01 16:08:32 +02:00
Dan Helfman b3fd1be5f6 Add "borgmatic export-tar" action to export an archive to a tar-formatted file or stream (#300).
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is passing Details
2020-07-22 21:53:55 -07:00
Dan Helfman 6f82c9979b Add #339 to NEWS and add test.
continuous-integration/drone/push Build is passing Details
2020-07-07 22:31:17 -07:00
Dan Helfman 37cc229749 Fix duplicate logging to Healthchecks and send "after_*" hooks output to Healthchecks (#328).
continuous-integration/drone/push Build is passing Details
continuous-integration/drone/tag Build is failing Details
2020-06-23 11:01:03 -07:00
Dan Helfman 17c2d109e5 Add tests for pass-through of BORG_* environment variables.
continuous-integration/drone/push Build is passing Details
2020-06-21 14:41:22 -07:00
Dan Helfman 32e15dc905 Add a few more mocks to PostgreSQL SSL tests.
continuous-integration/drone/push Build is passing Details
2020-06-20 14:39:16 -07:00
Dan Helfman f5ebca4907 Add SSL support to PostgreSQL database configuration (#331).
continuous-integration/drone/push Build is passing Details 
Reviewed-on: witten/borgmatic#331
 2020-06-20 21:24:14 +00:00
Edward Shornock d2d92b1f1a Add tests for the PostgreSQL SSL options 2020-06-20 23:32:24 +03:00
Dan Helfman 27cbe9dfc0 Fix for potential data loss (data not getting backed up) when borgmatic omitted configured source directories in certain situations (#333).
continuous-integration/drone/push Build is passing Details
2020-06-19 20:16:38 -07:00