Not a valid repository, check repo config #593

New Issue

ccigas · 2022-10-04T21:41:37Z

ccigas commented

2022-10-04 21:41:37 +00:00

What I'm trying to do and why

Install borgmatic for the first time to handle my backups

Steps to reproduce (if a bug)

sudo borgmatic create --verbosity 1 --list --stats

This is on my first ever install

Actual behavior (if a bug)

/etc/borgmatic/config.yaml: Error running configuration file
ssh://host@host.repo.borgbase.com/./repo: Error running actions for repository
/srv/repos/host/repo is not a valid repository. Check repo config.
Command 'borg create --info --stats ssh://host@host.repo.borgbase.com/./repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /home/username/borgtest' returned non-zero exit status 2.

Expected behavior (if a bug)

Other notes / implementation ideas

Environment

borgmatic version: 1.5.1

Use sudo borgmatic --version or sudo pip show borgmatic | grep ^Version

borgmatic installation method: sudo apt install

Borg version: 1.1.15

Use sudo borg --version

Python version: 3.8.10

Use python3 --version

Database version (if applicable): [version here]

Use psql --version or mysql --version on client and server.

operating system and version: Ubuntu 20.04.5

Just trying to learn borg and was suggested to use borgmatic off the start. After going through the installation method on the how to page and the first time trying to create a backup, I get stuck with the error mentioned.

Not sure what to do here, tried googling around but cant find anything related to it except similar errors for local paths, not that path on borgbase.

Hope someone can help me out. Thanks

#### What I'm trying to do and why Install borgmatic for the first time to handle my backups #### Steps to reproduce (if a bug) ``` sudo borgmatic create --verbosity 1 --list --stats ``` This is on my first ever install #### Actual behavior (if a bug) ``` /etc/borgmatic/config.yaml: Error running configuration file ssh://host@host.repo.borgbase.com/./repo: Error running actions for repository /srv/repos/host/repo is not a valid repository. Check repo config. Command 'borg create --info --stats ssh://host@host.repo.borgbase.com/./repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /home/username/borgtest' returned non-zero exit status 2. ``` #### Expected behavior (if a bug) #### Other notes / implementation ideas #### Environment **borgmatic version:** 1.5.1 Use `sudo borgmatic --version` or `sudo pip show borgmatic | grep ^Version` **borgmatic installation method:** sudo apt install **Borg version:** 1.1.15 Use `sudo borg --version` **Python version:** 3.8.10 Use `python3 --version` **Database version (if applicable):** [version here] Use `psql --version` or `mysql --version` on client and server. **operating system and version:** Ubuntu 20.04.5 Just trying to learn borg and was suggested to use borgmatic off the start. After going through the installation method on the how to page and the first time trying to create a backup, I get stuck with the error mentioned. Not sure what to do here, tried googling around but cant find anything related to it except similar errors for local paths, not that path on borgbase. Hope someone can help me out. Thanks

witten commented

2022-10-04 22:23:37 +00:00

Welcome to borgmatic! The problem you're seeing may be that the Borg repository hasn't been initialized/created yet. For instance, have you tried something like this before your create?

borgmatic init --verbosity 1

This creates the configured repository on disk so that you have a place to backup to.

Also note that the version of borgmatic you're using is pretty old (3 years!), so please considering upgrading if you'd like new features and bug fixes. I realize that OS repositories are often out of date, but you can install with pip or via another method.

Welcome to borgmatic! The problem you're seeing may be that the Borg repository hasn't been initialized/created yet. For instance, have you tried something like this before your `create`? ```borgmatic init --verbosity 1``` This creates the configured repository on disk so that you have a place to backup to. Also note that the version of borgmatic you're using is pretty old (3 years!), so please considering upgrading if you'd like new features and bug fixes. I realize that OS repositories are often out of date, but you can install with `pip` or via another method.

witten added the

question / support

label 2022-10-04 22:23:44 +00:00

ccigas commented

2022-10-05 00:43:03 +00:00

This is the response I get, even if no sudo.

ccigas@cooper01:~$ sudo borgmatic init --verbosity 1
usage: borgmatic init -e ENCRYPTION_MODE [--append-only] [--storage-quota STORAGE_QUOTA] [-h]
borgmatic init: error: the following arguments are required: -e/--encryption
Error parsing arguments: /usr/bin/borgmatic init --verbosity 1

This is the response I get, even if no sudo. ``` ccigas@cooper01:~$ sudo borgmatic init --verbosity 1 usage: borgmatic init -e ENCRYPTION_MODE [--append-only] [--storage-quota STORAGE_QUOTA] [-h] borgmatic init: error: the following arguments are required: -e/--encryption Error parsing arguments: /usr/bin/borgmatic init --verbosity 1 ```

witten commented

2022-10-05 02:30:43 +00:00

Oh, my bad! You'll need to specify an encryption type when initializing a repository. For example:

borgmatic init --verbosity 1 --encryption repokey

See the Borg documentation for more information about your encryption options:
https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-mode-tldr

Oh, my bad! You'll need to specify an encryption type when initializing a repository. For example: ```bash borgmatic init --verbosity 1 --encryption repokey ``` See the Borg documentation for more information about your encryption options: https://borgbackup.readthedocs.io/en/stable/usage/init.html#encryption-mode-tldr

ccigas commented

2022-10-05 03:41:20 +00:00

That command did the trick, thanks! But I am running into something else now. After running the init command, then the create command I listed in my OP, I logged off and connected somewhere else to my server and had to use the init command again. Is that normal? I assumed it was only once.

Also, after doing the init command again on the second login, I ran the create command again and borg reuploaded the single directory that I have in the config, just a test dir with a test text file. I thought out of the box that borg would only back up any changes? Am I missing something in the yaml to do this?

My goal is to backup my whole system to create a system image essentially.

Thanks for the help so far!

That command did the trick, thanks! But I am running into something else now. After running the init command, then the create command I listed in my OP, I logged off and connected somewhere else to my server and had to use the init command again. Is that normal? I assumed it was only once. Also, after doing the init command again on the second login, I ran the create command again and borg reuploaded the single directory that I have in the config, just a test dir with a test text file. I thought out of the box that borg would only back up any changes? Am I missing something in the yaml to do this? My goal is to backup my whole system to create a system image essentially. Thanks for the help so far!

witten commented

2022-10-05 03:52:46 +00:00

After running the init command, then the create command I listed in my OP, I logged off and connected somewhere else to my server and had to use the init command again. Is that normal? I assumed it was only once.

It's not normal! You should only have to initialize the repository once. Not sure what was going on there if the repository path didn't change within borgmatic's configuration file. Maybe the repository was created in a temporary directory or other location that got deleted between logins?

Also, after doing the init command again on the second login, I ran the create command again and borg reuploaded the single directory that I have in the config, just a test dir with a test text file. I thought out of the box that borg would only back up any changes? Am I missing something in the yaml to do this?

It sounds like after your second login, you created (inited) a completely new repository, so Borg was effectively starting over "from scratch." If you'd like to only backup any changes, you'd have to make sure you're using the same repository from one create to the next.

> After running the init command, then the create command I listed in my OP, I logged off and connected somewhere else to my server and had to use the init command again. Is that normal? I assumed it was only once. It's not normal! You should only have to initialize the repository once. Not sure what was going on there if the repository path didn't change within borgmatic's configuration file. Maybe the repository was created in a temporary directory or other location that got deleted between logins? > Also, after doing the init command again on the second login, I ran the create command again and borg reuploaded the single directory that I have in the config, just a test dir with a test text file. I thought out of the box that borg would only back up any changes? Am I missing something in the yaml to do this? It sounds like after your second login, you created (`init`ed) a completely new repository, so Borg was effectively starting over "from scratch." If you'd like to only backup any changes, you'd have to make sure you're using the same repository from one `create` to the next.

ccigas commented

2022-10-05 13:03:13 +00:00

Not sure how I wouldve done that if I did.

My source is set to /home/ccigas/borgtest
My repo is just copied from borgbase: ssh://host@host.repo.borgbase.com/./repo
I have my encryption passphrase filled in, my ssh_command filled in for the private key. And lastly is the default retention of 7 days, no other changes from the default config file.

So this morning I just tried doing the create command again, did not have to use the init command so thats a plus. But still seems like its creating multiple back ups? Is this because of archive name format always being different by using the default setting?

"{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}

So at the moment I ran the script three times on the test directory that hasnt changed once and when I list the archive there are three archives listed. Not really sure what I am doing wrong to be honest.

Not sure how I wouldve done that if I did. My source is set to /home/ccigas/borgtest My repo is just copied from borgbase: ssh://host@host.repo.borgbase.com/./repo I have my encryption passphrase filled in, my ssh_command filled in for the private key. And lastly is the default retention of 7 days, no other changes from the default config file. So this morning I just tried doing the create command again, did not have to use the init command so thats a plus. But still seems like its creating multiple back ups? Is this because of archive name format always being different by using the default setting? ``` "{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} ``` So at the moment I ran the script three times on the test directory that hasnt changed once and when I list the archive there are three archives listed. Not really sure what I am doing wrong to be honest.

witten commented

2022-10-05 16:38:01 +00:00

Borg creates a new archive everytime you call create. This serves as a snapshot of a particular point in time. However, if nothing has changed on disk, then the archive takes up almost no additional space on disk. The way you can verify this is by including the --files flag with create and looking at the list of files added/modified/etc. each time you run borgmatic. Hope this helps.

Borg creates a new archive everytime you call `create`. This serves as a snapshot of a particular point in time. However, if nothing has changed on disk, then the archive takes up almost no additional space on disk. The way you can verify this is by including the `--files` flag with `create` and looking at the list of files added/modified/etc. each time you run borgmatic. Hope this helps.

ccigas commented

2022-10-10 02:37:05 +00:00

Sorry I took so long to answer back. I ran the --files flag and this is my output. The line with the text file listed, does this mean that its working as expected since nothing changed in the directory or file?

~$ sudo borgmatic create --verbosity 1 --files
ssh://host@host.repo.borgbase.com/./repo: Creating archive
Enter passphrase for key '/home/ccigas/.ssh/id_ed25519':
Creating archive at "ssh://host@host.repo.borgbase.com/./repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}"
A /home/ccigas/borgtest/test.txt
Remote: Storage quota: 3.64 kB out of 10.00 GB used.
Remote: Storage quota: 5.45 kB out of 10.00 GB used.
Remote: compaction freed about 821 B repository space.

summary:
/etc/borgmatic/config.yaml: Successfully ran configuration file

Sorry I took so long to answer back. I ran the `--files` flag and this is my output. The line with the text file listed, does this mean that its working as expected since nothing changed in the directory or file? ``` ~$ sudo borgmatic create --verbosity 1 --files ssh://host@host.repo.borgbase.com/./repo: Creating archive Enter passphrase for key '/home/ccigas/.ssh/id_ed25519': Creating archive at "ssh://host@host.repo.borgbase.com/./repo::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}" A /home/ccigas/borgtest/test.txt Remote: Storage quota: 3.64 kB out of 10.00 GB used. Remote: Storage quota: 5.45 kB out of 10.00 GB used. Remote: compaction freed about 821 B repository space. summary: /etc/borgmatic/config.yaml: Successfully ran configuration file ```

witten commented

2022-10-10 17:00:37 +00:00

The A in the A /home/ccigas/borgtest/test.txt line means that text file was newly added. If you try running create again, it should no longer show up in the listing as it'll be unchanged. (borgmatic suppresses unchanged files from the listing.) Then, if you modify the file, it'll show up as M for modified.

Here are all the letter codes: https://borgbackup.readthedocs.io/en/stable/usage/create.html#item-flags

The `A` in the `A /home/ccigas/borgtest/test.txt` line means that text file was newly added. If you try running `create` *again*, it should no longer show up in the listing as it'll be unchanged. (borgmatic suppresses unchanged files from the listing.) Then, if you modify the file, it'll show up as `M` for modified. Here are all the letter codes: https://borgbackup.readthedocs.io/en/stable/usage/create.html#item-flags

ccigas commented

2022-10-15 23:56:28 +00:00

The A in the A /home/ccigas/borgtest/test.txt line means that text file was newly added. If you try running create again, it should no longer show up in the listing as it'll be unchanged. (borgmatic suppresses unchanged files from the listing.) Then, if you modify the file, it'll show up as M for modified.

Here are all the letter codes: https://borgbackup.readthedocs.io/en/stable/usage/create.html#item-flags

Just got back into this, I did the same command again as in my previous reply and I still see an A before the text file location. Not sure what I am doing wrong. Should I just post my config?

> The `A` in the `A /home/ccigas/borgtest/test.txt` line means that text file was newly added. If you try running `create` *again*, it should no longer show up in the listing as it'll be unchanged. (borgmatic suppresses unchanged files from the listing.) Then, if you modify the file, it'll show up as `M` for modified. > > Here are all the letter codes: https://borgbackup.readthedocs.io/en/stable/usage/create.html#item-flags Just got back into this, I did the same command again as in my previous reply and I still see an A before the text file location. Not sure what I am doing wrong. Should I just post my config?

witten commented

2022-10-16 03:11:48 +00:00

Sure, feel free to post your config (with any secrets or sensitive information redacted). I'm not sure what might be causing the behavior you're seeing though. Even if your repository is append-only on Borgbase, that still wouldn't cause your test file to be "added" repeatedly.

ccigas commented

2022-10-18 00:59:31 +00:00

Sure, feel free to post your config (with any secrets or sensitive information redacted). I'm not sure what might be causing the behavior you're seeing though. Even if your repository is append-only on Borgbase, that still wouldn't cause your test file to be "added" repeatedly.

Thanks, posted it to pastebin so its not making this too long.
https://pastebin.com/gm9j1isC

> Sure, feel free to post your config (with any secrets or sensitive information redacted). I'm not sure what might be causing the behavior you're seeing though. Even if your repository is append-only on Borgbase, that still wouldn't cause your test file to be "added" repeatedly. Thanks, posted it to pastebin so its not making this too long. https://pastebin.com/gm9j1isC

witten commented

2022-10-18 04:55:37 +00:00

I don't see anything in your configuration file that might lead to the behavior you're seeing, but I did find this hightly relevant Borg FAQ entry: I am seeing ‘A’ (added) status for an unchanged file!?

It sounds like the behavior you're seeing is perfectly normal (but confusing!), and nothing is being duplicated in storage.

I don't see anything in your configuration file that might lead to the behavior you're seeing, but I did find this hightly relevant Borg FAQ entry: [I am seeing ‘A’ (added) status for an unchanged file!?](https://borgbackup.readthedocs.io/en/stable/faq.html#a-status-oddity) It sounds like the behavior you're seeing is perfectly normal (but confusing!), and nothing is being duplicated in storage.

ccigas commented

2022-10-19 01:31:03 +00:00

Oh nice thanks! Time to learn how to do a full OS backup and restore now then.

Thanks for the help!

Oh nice thanks! Time to learn how to do a full OS backup and restore now then. Thanks for the help!

witten commented

2022-10-19 02:52:47 +00:00

Sure thing!

witten closed this issue

2022-10-19 02:52:48 +00:00

Sign in to join this conversation.