Btrfs snapshotting (#251).

Reviewed-on: #946

.flake8

@@ -1 +0,0 @@
-select = Q0

.gitea/workflows/build.yaml

@@ -26,3 +26,5 @@ jobs:
           PASSWORD: "${{ secrets.REGISTRY_PASSWORD }}"
       - run: podman build --tag "$IMAGE_NAME" --file docs/Dockerfile --storage-opt "overlay.mount_program=/usr/bin/fuse-overlayfs" .
       - run: podman push "$IMAGE_NAME"
+      - run: scripts/export-docs-from-image
+      - run: curl --user "${{ secrets.REGISTRY_USERNAME }}:${{ secrets.REGISTRY_PASSWORD }}" --upload-file borgmatic-docs.tar.gz https://projects.torsion.org/api/packages/borgmatic-collective/generic/borgmatic-docs/$(head --lines=1 NEWS)/borgmatic-docs.tar.gz

NEWS

@@ -1,3 +1,99 @@
+1.9.4.dev0
+ * #251 (beta): Add a Btrfs hook for snapshotting and backing up Btrfs subvolumes. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/snapshot-your-filesystems/
+ * #926: Fix library error when running within a PyInstaller bundle.
+ * Reorganize data source and monitoring hooks to make developing new hooks easier.
+
+1.9.3
+ * #261 (beta): Add a ZFS hook for snapshotting and backing up ZFS datasets. See the documentation
+   for more information: https://torsion.org/borgmatic/docs/how-to/snapshot-your-filesystems/
+ * Remove any temporary copies of the manifest file created in support of the "bootstrap" action.
+ * Deprecate the "store_config_files" option at the global scope and move it under the "bootstrap"
+   hook. See the documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/extract-a-backup/#extract-the-configuration-files-used-to-create-an-archive
+ * Require the runtime directory to be an absolute path.
+ * Add a "--deleted" flag to the "repo-list" action for listing deleted archives that haven't
+   yet been compacted (Borg 2 only).
+ * Promote the "spot" check from a beta feature to stable.
+
+1.9.2
+ * #441: Apply the "umask" option to all relevant actions, not just some of them.
+ * #722: Remove the restriction that the "extract" and "mount" actions must match a single
+   repository. Now they work more like other actions, where each repository is applied in turn.
+ * #932: Fix the missing build backend setting in pyproject.toml to allow Fedora builds.
+ * #934: Update the logic that probes for the borgmatic streaming database dump, bootstrap
+   metadata, and check state directories to support more platforms and use cases. See the
+   documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#runtime-directory
+ * #934: Add the "RuntimeDirectory" and "StateDirectory" options to the sample systemd service
+   file to support the new runtime and state directory logic.
+ * #939: Fix borgmatic ignoring the "BORG_RELOCATED_REPO_ACCESS_IS_OK" and
+   "BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK" environment variables.
+ * Add a Pushover monitoring hook. See the documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#pushover-hook
+
+1.9.1
+ * #928: Fix the user runtime directory location on macOS (and possibly Cygwin).
+ * #930: Fix an error with the sample systemd service when no credentials are configured.
+ * #931: Fix an error when implicitly upgrading the check state directory from ~/.borgmatic to
+   ~/.local/state/borgmatic across filesystems.
+
+1.9.0
+ * #609: Fix the glob expansion of "source_directories" values to respect the "working_directory"
+   option.
+ * #609: BREAKING: Apply the "working_directory" option to all actions, not just "create". This
+   includes repository paths, destination paths, mount points, etc.
+ * #562: Deprecate the "borgmatic_source_directory" option in favor of "user_runtime_directory"
+   and "user_state_directory".
+ * #562: BREAKING: Move the default borgmatic streaming database dump and bootstrap metadata
+   directory from ~/.borgmatic to /run/user/$UID/borgmatic, which is more XDG-compliant. You can
+   override this location with the new "user_runtime_directory" option. Existing archives with
+   database dumps at the old location are still restorable. 
+ * #562, #638: Move the default check state directory from ~/.borgmatic to 
+   ~/.local/state/borgmatic. This is more XDG-compliant and also prevents these state files from
+   getting backed up (unless you explicitly include them). You can override this location with the
+   new "user_state_directory" option. After the first time you run the "check" action with borgmatic
+   1.9.0, you can safely delete the ~/.borgmatic directory.
+ * #838: BREAKING: With Borg 1.4+, store database dumps and bootstrap metadata in a "/borgmatic"
+   directory within a backup archive, so the path doesn't depend on the current user. This means
+   that you can now backup as one user and restore or bootstrap as another user, among other use
+   cases.
+ * #902: Add loading of encrypted systemd credentials. See the documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/provide-your-passwords/#using-systemd-service-credentials
+ * #911: Add a "key change-passphrase" action to change the passphrase protecting a repository key.
+ * #914: Fix a confusing apparent hang when when the repository location changes, and instead
+   show a helpful error message.
+ * #915: BREAKING: Rename repository actions like "rcreate" to more explicit names like
+   "repo-create" for compatibility with recent changes in Borg 2.0.0b10.
+ * #918: BREAKING: When databases are configured, don't auto-enable the "one_file_system" option,
+   as existing auto-excludes of special files should be sufficient to prevent Borg from hanging on
+   them. But if this change causes problems for you, you can always enable "one_file_system"
+   explicitly.
+ * #919: Clarify the command-line help for the "--config" flag.
+ * #919: Document a policy for versioning and breaking changes:
+   https://torsion.org/borgmatic/docs/how-to/upgrade/#versioning-and-breaking-changes
+ * #921: BREAKING: Change soft failure command hooks to skip only the current repository rather than
+   all repositories in the configuration file.
+ * #922: Replace setup.py (Python packaging metadata) with the more modern pyproject.toml.
+ * When using Borg 2, default the "archive_name_format" option to just "{hostname}", as Borg 2 does
+   not require unique archive names; identical archive names form a common "series" that can be
+   targeted together. See the Borg 2 documentation for more information:
+   https://borgbackup.readthedocs.io/en/2.0.0b13/changes.html#borg-1-2-x-1-4-x-to-borg-2-0
+ * Add support for Borg 2's "rclone:" repository URLs, so you can backup to 70+ cloud storage
+   services whether or not they support Borg explicitly.
+ * Add support for Borg 2's "sftp://" repository URLs.
+ * Update the "--match-archives" and "--archive" flags to support Borg 2 series names or archive
+   hashes.
+ * Add a "--match-archives" flag to the "prune" action.
+ * Add "--local-path" and "--remote-path" flags to the "config bootstrap" action for setting the
+   Borg executable paths used for bootstrapping.
+ * Add a "--user-runtime-directory" flag to the "config bootstrap" action for helping borgmatic
+   locate the bootstrap metadata stored in an archive.
+ * Add a Zabbix monitoring hook. See the documentation for more information:
+   https://torsion.org/borgmatic/docs/how-to/monitor-your-backups/#zabbix-hook
+ * Add a tarball of borgmatic's HTML documentation to the packages on the project page.
+
 1.8.14
  * #896: Fix an error in borgmatic rcreate/init on an empty repository directory with Borg 1.4.
  * #898: Add glob ("*") support to the "--repository" flag. Just quote any values containing

README.md

@@ -61,14 +61,19 @@ borgmatic is powered by [Borg Backup](https://www.borgbackup.org/).
 <a href="https://mariadb.com/"><img src="docs/static/mariadb.png" alt="MariaDB" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://www.mongodb.com/"><img src="docs/static/mongodb.png" alt="MongoDB" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://sqlite.org/"><img src="docs/static/sqlite.png" alt="SQLite" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+<a href="https://openzfs.org/"><img src="docs/static/openzfs.png" alt="OpenZFS" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+<a href="https://btrfs.readthedocs.io/"><img src="docs/static/btrfs.png" alt="Btrfs" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+<a href="https://rclone.org"><img src="docs/static/rclone.png" alt="rclone" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://healthchecks.io/"><img src="docs/static/healthchecks.png" alt="Healthchecks" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://uptime.kuma.pet/"><img src="docs/static/uptimekuma.png" alt="Uptime Kuma" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://cronitor.io/"><img src="docs/static/cronitor.png" alt="Cronitor" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://cronhub.io/"><img src="docs/static/cronhub.png" alt="Cronhub" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://www.pagerduty.com/"><img src="docs/static/pagerduty.png" alt="PagerDuty" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+<a href="https://www.pushover.net/"><img src="docs/static/pushover.png" alt="Pushover" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://ntfy.sh/"><img src="docs/static/ntfy.png" alt="ntfy" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://grafana.com/oss/loki/"><img src="docs/static/loki.png" alt="Loki" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://github.com/caronc/apprise/wiki"><img src="docs/static/apprise.png" alt="Apprise" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
+<a href="https://www.zabbix.com/"><img src="docs/static/zabbix.png" alt="Zabbix" height="40px" style="margin-bottom:20px; margin-right:20px;"></a>
 <a href="https://www.borgbase.com/?utm_source=borgmatic"><img src="docs/static/borgbase.png" alt="BorgBase" height="60px" style="margin-bottom:20px; margin-right:20px;"></a>
 
 

borgmatic/actions/borg.py

@@ -1,7 +1,7 @@
 import logging
 
 import borgmatic.borg.borg
-import borgmatic.borg.rlist
+import borgmatic.borg.repo_list
 import borgmatic.config.validate
 
 logger = logging.getLogger(__name__)
@@ -25,7 +25,7 @@ def run_borg(
         logger.info(
             f'{repository.get("label", repository["path"])}: Running arbitrary Borg command'
         )
-        archive_name = borgmatic.borg.rlist.resolve_archive_name(
+        archive_name = borgmatic.borg.repo_list.resolve_archive_name(
             repository['path'],
             borg_arguments.archive,
             config,

borgmatic/actions/change_passphrase.py

@@ -0,0 +1,38 @@
+import logging
+
+import borgmatic.borg.change_passphrase
+import borgmatic.config.validate
+
+logger = logging.getLogger(__name__)
+
+
+def run_change_passphrase(
+    repository,
+    config,
+    local_borg_version,
+    change_passphrase_arguments,
+    global_arguments,
+    local_path,
+    remote_path,
+):
+    '''
+    Run the "key change-passprhase" action for the given repository.
+    '''
+    if (
+        change_passphrase_arguments.repository is None
+        or borgmatic.config.validate.repositories_match(
+            repository, change_passphrase_arguments.repository
+        )
+    ):
+        logger.info(
+            f'{repository.get("label", repository["path"])}: Changing repository passphrase'
+        )
+        borgmatic.borg.change_passphrase.change_passphrase(
+            repository['path'],
+            config,
+            local_borg_version,
+            change_passphrase_arguments,
+            global_arguments,
+            local_path=local_path,
+            remote_path=remote_path,
+        )

borgmatic/actions/check.py

@@ -6,14 +6,17 @@ import logging
 import os
 import pathlib
 import random
+import shutil