retention prefix specified, but borgmatic still errors. #402

Closed
opened 2021-03-20 23:09:28 +00:00 by Filius-Patris · 4 comments

What I'm trying to do and why

I would like to set the archive name format to {now:%Y-%m-%d}. This requires setting the /retention/prefix key. The docs explicitely allow this to be an empty value, however bormgatic doesn't seem to recognize this.

Steps to reproduce (if a bug)

/etc/borgmatic/config.yaml:

location:
    source_directories:
        - /data
        - /hosting
        - /git

    repositories:
        - xxxx@xxxx.repo.borgbase.com:repo

    # exclude_patterns:
        # - '*.pyc'
        # - /home/*/.cache
        # - /etc/ssl

    exclude_caches: true

    exclude_if_present:
        - .nobackup
    keep_exclude_tags: true

storage:
    encryption_passphrase: <REDACTED>
    checkpoint_interval: 300
    compression: lz4
    archive_name_format: "{now:%Y-%m-%d}"

retention:
    keep_daily: 7
    keep_monthly: 4
    prefix: ""

hooks:
    before_everything:
        - pacman -Qe > /root/installed-packages.txt
    on_error:
        - echo "Error during prune/create/check."
    after_everything:
        - rm /root/installed-packages.txt

    postgresql_databases:
        - name: all
          username: postgres
          ssl_mode: disable

Actual behavior (if a bug)

[root@hostname ~]# borgmatic --verbosity 2 -c /etc/borgmatic/config.yaml create
Ensuring legacy configuration is upgraded
/etc/borgmatic/config.yaml: No valid configuration files found

summary:
/etc/borgmatic/config.yaml: Error parsing configuration file
An error occurred while parsing a configuration file at /etc/borgmatic/config.yaml:
If you provide an archive_name_format, you must also specify a retention prefix.
/etc/borgmatic/config.yaml: No valid configuration files found

Need some help? https://torsion.org/borgmatic/#issues

Expected behavior (if a bug)

... just accept it? And of course, prune everything that starts differently.

Environment

borgmatic version: 1.5.12

borgmatic installation method: pacman -S borgmatic

Borg version: borg 1.1.15

Python version: Python 3.9.2

Database version (if applicable): psql (PostgreSQL) 13.2

operating system and version: Linux 5.11.7-arch1-1 (Arch Linux, complete system update yesterday)

#### What I'm trying to do and why I would like to set the archive name format to `{now:%Y-%m-%d}`. This requires setting the /retention/prefix key. The docs explicitely allow this to be an empty value, however bormgatic doesn't seem to recognize this. #### Steps to reproduce (if a bug) /etc/borgmatic/config.yaml: ```yaml location: source_directories: - /data - /hosting - /git repositories: - xxxx@xxxx.repo.borgbase.com:repo # exclude_patterns: # - '*.pyc' # - /home/*/.cache # - /etc/ssl exclude_caches: true exclude_if_present: - .nobackup keep_exclude_tags: true storage: encryption_passphrase: <REDACTED> checkpoint_interval: 300 compression: lz4 archive_name_format: "{now:%Y-%m-%d}" retention: keep_daily: 7 keep_monthly: 4 prefix: "" hooks: before_everything: - pacman -Qe > /root/installed-packages.txt on_error: - echo "Error during prune/create/check." after_everything: - rm /root/installed-packages.txt postgresql_databases: - name: all username: postgres ssl_mode: disable ``` #### Actual behavior (if a bug) ```sh [root@hostname ~]# borgmatic --verbosity 2 -c /etc/borgmatic/config.yaml create Ensuring legacy configuration is upgraded /etc/borgmatic/config.yaml: No valid configuration files found summary: /etc/borgmatic/config.yaml: Error parsing configuration file An error occurred while parsing a configuration file at /etc/borgmatic/config.yaml: If you provide an archive_name_format, you must also specify a retention prefix. /etc/borgmatic/config.yaml: No valid configuration files found Need some help? https://torsion.org/borgmatic/#issues ``` #### Expected behavior (if a bug) ... just accept it? And of course, prune everything that starts differently. #### Environment **borgmatic version:** 1.5.12 **borgmatic installation method:** pacman -S borgmatic **Borg version:** borg 1.1.15 **Python version:** Python 3.9.2 **Database version (if applicable):** psql (PostgreSQL) 13.2 **operating system and version:** Linux 5.11.7-arch1-1 (Arch Linux, complete system update yesterday)
have you https://torsion.org/borgmatic/docs/how-to/set-up-backups/#validation ?

I think the problem is here: 77980511c6/borgmatic/config/validate.py (L71)

In Python, not prefix will return true if prefix is an empty string, while configuration reference says to leave prefix empty to disable the default, in which case --prefix option is not passed to borg.

The problem with this is that if you do specify an archive prefix, you have to specify the pruning prefix as well. This effectively forces you to use a static prefix, which is not always the case. E.g. I use simply '{now}' as a prefix for my backups, since all my remote backups come from local backups on my NAS and I keep different backups in separate borg repos.

I think the problem is here: https://projects.torsion.org/borgmatic-collective/borgmatic/src/commit/77980511c68f5f0d5ce91ab9ed53d2e29fd2734f/borgmatic/config/validate.py#L71 In Python, `not prefix` will return `true` if `prefix` is an empty string, while configuration reference says to leave `prefix` empty to disable the default, in which case `--prefix` option is not passed to `borg`. The problem with this is that if you do specify an archive prefix, you have to specify the pruning prefix as well. This effectively forces you to use a static prefix, which is not always the case. E.g. I use simply `'{now}'` as a prefix for my backups, since all my remote backups come from local backups on my NAS and I keep different backups in separate `borg` repos.
Owner

I apologize for the lengthy delay here. This feature was working "by design" with the goal of preventing users from shooting themselves in the foot with accidental pruning of archives they didn't intend to prune. But I think you've made it clear that: 1. That doesn't support your use case (an archive_name_format that can't be matched with a static prefix, and 2. It could be documented better.

So given your use case, I've completely removed the restriction and updated configuration comments accordingly. These changes are in master and will be part of the next release. Thanks for filing this!

I apologize for the lengthy delay here. This feature was working "by design" with the goal of preventing users from shooting themselves in the foot with accidental pruning of archives they didn't intend to prune. But I think you've made it clear that: 1. That doesn't support your use case (an `archive_name_format` that can't be matched with a static `prefix`, and 2. It could be documented better. So given your use case, I've completely removed the restriction and updated configuration comments accordingly. These changes are in master and will be part of the next release. Thanks for filing this!
Owner

Released in borgmatic 1.6.1!

Released in borgmatic 1.6.1!
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#402
No description provided.