mailcow backup with borgmatic fails #603

New Issue

barnybla · 2022-10-23T15:07:17Z

barnybla commented

2022-10-23 15:07:17 +00:00

What I'm trying to do and why

I try to initialize the repository on a hetzner storagebox

Steps to reproduce (if a bug)

Include (sanitized) borgmatic configuration files if applicable.

Actual behavior (if a bug)

Include (sanitized) --verbosity 2 output if applicable.

root@mail:/opt/mailcow-dockerized# docker exec f8a543785424 borgmatic --verbosity 2
Ensuring legacy configuration is upgraded
/etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found

summary:
/etc/borgmatic.d/config.yaml: Error parsing configuration file
An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml:
mapping values are not allowed in this context
in "/etc/borgmatic.d/config.yaml", line 3, column 9
/etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found

Expected behavior (if a bug)

Other notes / implementation ideas

Environment

borgmatic version: [1.7.2]

Use sudo borgmatic --version or sudo pip show borgmatic | grep ^Version

borgmatic installation method: [Docker container]

Borg version: [ 1.2.2]

Use sudo borg --version

Python version: [ 3.10.6]

Use python3 --version

Database version (if applicable): [Ver 15.1 Distrib 10.6.9-MariaDB]

Use psql --version or mysql --version on client and server.

operating system and version: [ubunto 22.04]

I have copied the config.yaml file to /opt/mailcow-dockerized/data/conf/borgmatic/etc. After I got the issue I copied the file also the the other shown postions. But the error was still there.

#### What I'm trying to do and why I try to initialize the repository on a hetzner storagebox #### Steps to reproduce (if a bug) Include (sanitized) borgmatic configuration files if applicable. #### Actual behavior (if a bug) Include (sanitized) `--verbosity 2` output if applicable. root@mail:/opt/mailcow-dockerized# docker exec f8a543785424 borgmatic --verbosity 2 Ensuring legacy configuration is upgraded /etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found summary: /etc/borgmatic.d/config.yaml: Error parsing configuration file An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml: mapping values are not allowed in this context in "/etc/borgmatic.d/config.yaml", line 3, column 9 /etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found #### Expected behavior (if a bug) #### Other notes / implementation ideas #### Environment **borgmatic version:** [1.7.2] Use `sudo borgmatic --version` or `sudo pip show borgmatic | grep ^Version` **borgmatic installation method:** [Docker container] **Borg version:** [ 1.2.2] Use `sudo borg --version` **Python version:** [ 3.10.6] Use `python3 --version` **Database version (if applicable):** [Ver 15.1 Distrib 10.6.9-MariaDB] Use `psql --version` or `mysql --version` on client and server. **operating system and version:** [ubunto 22.04] I have copied the config.yaml file to /opt/mailcow-dockerized/data/conf/borgmatic/etc. After I got the issue I copied the file also the the other shown postions. But the error was still there.

config.txt

724 B

witten commented

2022-10-23 16:34:22 +00:00

The key part of the error here is:

An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml:
mapping values are not allowed in this context
in "/etc/borgmatic.d/config.yaml", line 3, column 9

Meaning there's an unexpected key/value on line 3. Looking at your attached config file (thanks for including it!), however, I don't actually see any problem with it. And it appears to validate just fine with validate-borgmatic-config (once the needed environment variables are set). Are you sure that the contents of your config file on disk (at /etc/borgmatic.d/config.yaml within your Docker container) match what you've posted here?

Also, side note: You may want to change your encryption passphrase, since it's posted publicly in this config file.

The key part of the error here is: ``` An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml: mapping values are not allowed in this context in "/etc/borgmatic.d/config.yaml", line 3, column 9 ``` Meaning there's an unexpected key/value on line 3. Looking at your attached config file (thanks for including it!), however, I don't actually see any problem with it. And it appears to validate just fine with `validate-borgmatic-config` (once the needed environment variables are set). Are you sure that the contents of your config file on disk (at `/etc/borgmatic.d/config.yaml` within your Docker container) match what you've posted here? Also, side note: You may want to change your encryption passphrase, since it's posted publicly in this config file.

witten added the

question / support

label 2022-10-23 16:34:34 +00:00

barnybla commented

2022-10-23 16:53:53 +00:00

yes, both files are the same. I put the file in the docker container on the position shown in this description: https://docs.mailcow.email/de/third_party/borgmatic/third_party-borgmatic/. After the first time error I copied the file to the other positions in the error description. The column 9 is a :, and that must be right column [location:].

The passphrase is a changed one.

Thanks!

yes, both files are the same. I put the file in the docker container on the position shown in this description: https://docs.mailcow.email/de/third_party/borgmatic/third_party-borgmatic/. After the first time error I copied the file to the other positions in the error description. The column 9 is a :, and that must be right column [location:]. The passphrase is a changed one. Thanks!

witten commented

2022-10-23 16:58:21 +00:00

Oh, so you copied the entirety of the example file, including the source mailcow.conf and cat <<EOF > data/conf/borgmatic/etc/config.yaml portion? That's not a valid part of borgmatic's configuration syntax, and I bet that's the source of the error! Those are instructions to your shell in case you want to execute a command that writes out a borgmatic configuration file. So if you're not doing that, you can entirely cut the following lines:

source mailcow.conf
cat <<EOF > data/conf/borgmatic/etc/config.yaml
EOF (found at the end)

Hope that helps!

Oh, so you copied the entirety of the example file, including the `source mailcow.conf` and `cat <<EOF > data/conf/borgmatic/etc/config.yaml` portion? That's not a valid part of borgmatic's configuration syntax, and I bet that's the source of the error! Those are instructions to your shell in case you want to execute a command that writes out a borgmatic configuration file. So if you're not doing that, you can entirely cut the following lines: * `source mailcow.conf` * `cat <<EOF > data/conf/borgmatic/etc/config.yaml` * `EOF` (found at the end) Hope that helps!

barnybla commented

2022-10-31 15:11:37 +00:00

Hi witten,

now I had time to working back on my mailcow borgmatic backup. I had removed all the entries in the config.yaml file:

location:
    source_directories:
        - /mnt/source
    repositories:
        - ssh://u298873@u298873.your-storagebox.de/./backups/mail.bblana.de
    exclude_patterns:
        - '/mnt/source/postfix/public/'
        - '/mnt/source/postfix/private/'
        - '/mnt/source/rspamd/rspamd.sock'

storage:
    encryption_passphrase: "9jyj5Zo8YEFwgIF9"

retention:
    keep_hourly: 24
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 6
    prefix: ""

hooks:
    mysql_databases:
        - name: ${DBNAME}
          username: ${DBUSER}
          password: ${DBPASS}
          options: --default-character-set=utf8mb4

Now I get a new error:

root@mail:/opt/mailcow-dockerized/data/conf/borgmatic/etc# docker exec f8a543785424 borgmatic info
/etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found

summary:
/etc/borgmatic.d/config.yaml: Error parsing configuration file
An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml:
did not find expected <document start>
  in "/etc/borgmatic.d/config.yaml", line 2, column 5
/etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found

I this case the column is the s from source_directories:, that is a right entry. The removed entries ensures the correct MySQL credentials are pulled in from mailcow.conf. So I need that for the mysql db. I got the config.yaml from this site: https://docs.mailcow.email/third_party/borgmatic/third_party-borgmatic/#create-or-amend-docker-composeoverrideyml

Normaly this must be right.

Thanks for your help
Bernd

Hi witten, now I had time to working back on my mailcow borgmatic backup. I had removed all the entries in the config.yaml file: ``` location: source_directories: - /mnt/source repositories: - ssh://u298873@u298873.your-storagebox.de/./backups/mail.bblana.de exclude_patterns: - '/mnt/source/postfix/public/' - '/mnt/source/postfix/private/' - '/mnt/source/rspamd/rspamd.sock' storage: encryption_passphrase: "9jyj5Zo8YEFwgIF9" retention: keep_hourly: 24 keep_daily: 7 keep_weekly: 4 keep_monthly: 6 prefix: "" hooks: mysql_databases: - name: ${DBNAME} username: ${DBUSER} password: ${DBPASS} options: --default-character-set=utf8mb4 ``` Now I get a new error: ``` root@mail:/opt/mailcow-dockerized/data/conf/borgmatic/etc# docker exec f8a543785424 borgmatic info /etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found summary: /etc/borgmatic.d/config.yaml: Error parsing configuration file An error occurred while parsing a configuration file at /etc/borgmatic.d/config.yaml: did not find expected <document start> in "/etc/borgmatic.d/config.yaml", line 2, column 5 /etc/borgmatic/config.yaml /etc/borgmatic.d /root/.config/borgmatic/config.yaml /root/.config/borgmatic.d: No valid configuration files found ``` I this case the column is the s from source_directories:, that is a right entry. The removed entries ensures the correct MySQL credentials are pulled in from mailcow.conf. So I need that for the mysql db. I got the config.yaml from this site: [https://docs.mailcow.email/third_party/borgmatic/third_party-borgmatic/#create-or-amend-docker-composeoverrideyml](https://) Normaly this must be right. Thanks for your help Bernd

witten commented

2022-10-31 16:58:45 +00:00

I believe the error you're seeing is consistent with extra spacing before the location: section header—the parser literally can't find the start of the document because it's indented. So I recommend making sure all of your section headers are fully left-aligned with no extra spaces. In general, spacing and indentation is very important in a YAML document!

I believe the error you're seeing is consistent with extra spacing before the `location:` section header—the parser literally can't find the start of the document because it's indented. So I recommend making sure all of your section headers are fully left-aligned with no extra spaces. In general, spacing and indentation is very important in a YAML document!

witten commented

2023-01-27 00:03:54 +00:00

I'm closing this for now, but please feel free to continue the discussion or open another ticket.

witten closed this issue

2023-01-27 00:03:54 +00:00

Sign in to join this conversation.