||3 years ago|
|borgmatic||3 years ago|
|sample||3 years ago|
|scripts||3 years ago|
|static||4 years ago|
|tests||3 years ago|
|.gitignore||3 years ago|
|AUTHORS||3 years ago|
|LICENSE||7 years ago|
|MANIFEST.in||4 years ago|
|NEWS||3 years ago|
|README.md||3 years ago|
|setup.cfg||4 years ago|
|setup.py||3 years ago|
|test_requirements.txt||3 years ago|
|tox.ini||3 years ago|
borgmatic is a simple Python wrapper script for the Borg backup software that initiates a backup, prunes any old backups according to a retention policy, and validates backups for consistency. The script supports specifying your settings in a declarative configuration file rather than having to put them all on the command-line, and handles common errors.
Here's an example config file:
location: # List of source directories to backup. Globs are expanded. source_directories: - /home - /etc - /var/log/syslog* # Paths to local or remote repositories. repositories: - user@backupserver:sourcehostname.borg # Any paths matching these patterns are excluded from backups. exclude_patterns: - /home/*/.cache retention: # Retention policy for how many backups to keep in each category. keep_daily: 7 keep_weekly: 4 keep_monthly: 6 consistency: # List of consistency checks to run: "repository", "archives", or both. checks: - repository - archives
Want to see borgmatic in action? Check out the screencast.
To get up and running, follow the Borg Quick
Start to create
a repository on a local or remote host. Note that if you plan to run borgmatic
on a schedule with cron, and you encrypt your Borg repository with a
passphrase instead of a key file, you'll either need to set the borgmatic
encryption_passphrase configuration variable or set the
environment variable. See the repository encryption
of the Quick Start for more info.
Alternatively, the passphrase can be specified programatically by setting
either the borgmatic
encryption_passcommand configuration variable or the
BORG_PASSCOMMAND environment variable. See the Borg Security
for more info.
If the repository is on a remote host, make sure that your local root user has key-based ssh access to the desired user account on the remote host.
To install borgmatic, run the following command to download and install it:
sudo pip3 install --upgrade borgmatic
Note that your pip binary may have a different name than "pip3". Make sure you're using Python 3, as borgmatic does not support Python 2.
Other ways to install
- A borgmatic Docker image based on Alpine Linux.
- Another borgmatic Docker image based on Alpine Linux, updated automatically whenever the Alpine image updates.
- A borgmatic package for Fedora.
- A borgmatic package for Arch Linux.
- A borgmatic package for OpenBSD.
After you install borgmatic, generate a sample configuration file:
If that command is not found, then it may be installed in a location that's
not in your system
PATH. Try looking in
This generates a sample configuration file at /etc/borgmatic/config.yaml (by default). You should edit the file to suit your needs, as the values are just representative. All fields are optional except where indicated, so feel free to ignore anything you don't need.
You can also have a look at the full configuration schema for the authoritative set of all configuration options. This is handy if borgmatic has added new options since you originally created your configuration file.
Multiple configuration files
A more advanced usage is to create multiple separate configuration files and place each one in an /etc/borgmatic.d directory. For instance:
sudo mkdir /etc/borgmatic.d sudo generate-borgmatic-config --destination /etc/borgmatic.d/app1.yaml sudo generate-borgmatic-config --destination /etc/borgmatic.d/app2.yaml
With this approach, you can have entirely different backup policies for different applications on your system. For instance, you may want one backup configuration for your database data directory, and a different configuration for your user home directories.
When you set up multiple configuration files like this, borgmatic will run each one in turn from a single borgmatic invocation. This includes, by default, the traditional /etc/borgmatic/config.yaml as well.
And if you need even more customizability, you can specify alternate
configuration paths on the command-line with borgmatic's
borgmatic --help for more information.
If you find yourself performing prepraration tasks before your backup runs, or
cleanup work afterwards, borgmatic hooks may be of interest. They're simply
shell commands that borgmatic executes for you at various points, and they're
configured in the
hooks section of your configuration file.
For instance, you can specify
before_backup hooks to dump a database to file
before backing it up, and specify
after_backup hooks to delete the temporary
borgmatic hooks run once per configuration file.
before_backup hooks run
prior to backups of all repositories.
after_backup hooks run afterwards, but
not if an error occurs in a previous hook or in the backups themselves. And
on_error hooks if an error occurs.
An important security note about hooks: borgmatic executes all hook commands
with the user permissions of borgmatic itself. So to prevent potential shell
injection or privilege escalation, do not forget to set secure permissions
chmod 0700) on borgmatic configuration files and scripts invoked by hooks.
See the sample generated configuration file mentioned above for specifics about hook configuration syntax.
In general, all you should need to do to upgrade borgmatic is run the following:
sudo pip3 install --upgrade borgmatic
However, see below about special cases.
Upgrading from borgmatic 1.0.x
borgmatic changed its configuration file format in version 1.1.0 from INI-style to YAML. This better supports validation, and has a more natural way to express lists of values. To upgrade your existing configuration, first upgrade to the new version of borgmatic.
As of version 1.1.0, borgmatic no longer supports Python 2. If you were already running borgmatic with Python 3, then you can simply upgrade borgmatic in-place:
sudo pip3 install --upgrade borgmatic
But if you were running borgmatic with Python 2, uninstall and reinstall instead:
sudo pip uninstall borgmatic sudo pip3 install borgmatic
The pip binary names for different versions of Python can differ, so the above commands may need some tweaking to work on your machine.
Once borgmatic is upgraded, run:
That will generate a new YAML configuration file at /etc/borgmatic/config.yaml (by default) using the values from both your existing configuration and excludes files. The new version of borgmatic will consume the YAML configuration file instead of the old one.
Upgrading from atticmatic
You can ignore this section if you're not an atticmatic user (the former name of borgmatic).
borgmatic only supports Borg now and no longer supports Attic. So if you're an Attic user, consider switching to Borg. See the Borg upgrade command for more information. Then, follow the instructions above about setting up your borgmatic configuration files.
If you were already using Borg with atticmatic, then you can easily upgrade from atticmatic to borgmatic. Simply run the following commands:
sudo pip3 uninstall atticmatic sudo pip3 install borgmatic
That's it! borgmatic will continue using your /etc/borgmatic configuration files.
You can run borgmatic and start a backup simply by invoking it without arguments:
This will also prune any old backups as per the configured retention policy, and check backups for consistency problems due to things like file damage.
If you'd like to see the available command-line arguments, view the help:
Note that borgmatic prunes archives before creating an archive, so as to free up space for archiving. This means that when a borgmatic run finishes, there may still be prune-able archives. Not to worry, as they will get cleaned up at the start of the next run.
By default, the backup will proceed silently except in the case of errors. But if you'd like to to get additional information about the progress of the backup as it proceeds, use the verbosity option:
borgmatic --verbosity 1
Or, for even more progress spew:
borgmatic --verbosity 2
À la carte
If you want to run borgmatic with only pruning, creating, or checking enabled, the following optional flags are available:
borgmatic --prune borgmatic --create borgmatic --check
You can run with only one of these flags provided, or you can mix and match any number of them. This supports use cases like running consistency checks from a different cron job with a different frequency, or running pruning with a different verbosity level.
borgmatic --list borgmatic --info
You can include an optional
--json flag with
--info to get the output formatted as JSON.
If you want to run borgmatic automatically, say once a day, the you can configure a job runner to invoke it periodically.
If you're using cron, download the sample cron file. Then, from the directory where you downloaded it:
sudo mv borgmatic /etc/cron.d/borgmatic sudo chmod +x /etc/cron.d/borgmatic
You can modify the cron file if you'd like to run borgmatic more or less frequently.
sudo mv borgmatic.service borgmatic.timer /etc/systemd/system/ sudo systemctl enable borgmatic.timer sudo systemctl start borgmatic.timer
Feel free to modify the timer file based on how frequently you'd like borgmatic to run.
Support and contributing
You've got issues? Or an idea for a feature enhancement? We've got an issue tracker. In order to create a new issue or comment on an issue, you'll need to login first. Note that you can login with an existing GitHub account if you prefer.
Other questions or comments? Contact mailto:firstname.lastname@example.org.
If you'd like to contribute to borgmatic development, please feel free to submit a Pull Request or open an issue first to discuss your idea. We also accept Pull Requests on GitHub, if that's more your thing. In general, contributions are very welcome. We don't bite!
Start with PEP 8. But then, apply the following deviations from it:
- For strings, prefer single quotes over double quotes.
- Limit all lines to a maximum of 100 characters.
- Use trailing commas within multiline values or argument lists.
- For multiline constructs, put opening and closing delimeters on lines separate from their contents.
- Within multiline constructs, use standard four-space indentation. Don't align indentation with an opening delimeter.
borgmatic code uses the Black code formatter and Flake8 code checker, so certain code style requirements will be enforced when running automated tests. See the Black and Flake8 documentation for more information.
To get set up to hack on borgmatic, first clone master via HTTPS or SSH:
git clone https://projects.torsion.org/witten/borgmatic.git
git clone ssh://email@example.com:3022/witten/borgmatic.git
Then, install borgmatic "editable" so that you can easily run borgmatic commands while you're hacking on them to make sure your changes work.
cd borgmatic/ pip3 install --editable --user .
Note that this will typically install the borgmatic commands into
~/.local/bin, which may or may not be on your PATH. There are other ways to
install borgmatic editable as well, for instance into the system Python
install (so without
--user, as root), or even into a
virtualenv. How or where you install
borgmatic is up to you, but generally an editable install makes development
and testing easier.
Assuming you've cloned the borgmatic source code as described above, and
you're in the
borgmatic/ working copy, install tox, which is used for
setting up testing environments:
sudo pip3 install tox
Finally, to actually run tests, run:
cd borgmatic tox
Note that while running borgmatic itself only requires Python 3+, running borgmatic's tests require Python 3.6+.
If when running tests, you get an error from the Black code formatter about files that would be reformatted, you can ask Black to format them for you via the following:
tox -e black
Broken pipe with remote repository
When running borgmatic on a large remote repository, you may receive errors like the following, particularly while "borg check" is validating backups for consistency:
Write failed: Broken pipe borg: Error: Connection closed by remote host
This error can be caused by an ssh timeout, which you can rectify by adding the following to the ~/.ssh/config file on the client:
Host * ServerAliveInterval 120
This should make the client keep the connection alive while validating backups.
libyaml compilation errors
borgmatic depends on a Python YAML library (ruamel.yaml) that will optionally use a C YAML library (libyaml) if present. But if it's not installed, then when installing or upgrading borgmatic, you may see errors about compiling the YAML library. If so, not to worry. borgmatic should install and function correctly even without the C YAML library. And borgmatic won't be any faster with the C library present, so you don't need to go out of your way to install it.