Auto-init repo in case it's not existing #117

New Issue

varac · 2018-12-11T13:21:30Z

varac commented

2018-12-11 13:21:30 +00:00

#110 (Support for initializing repositories) adds the feature to manually initialize the repo. In automated setups it would be great if borgmatic automatically checks before a backup if the repo exists, and if not, initilizes it with the parameters from the given config.

varac commented

2018-12-11 13:23:33 +00:00

Sorry for some reason I can't edit the above issue description anymore (it's too bold!).

varac commented

2018-12-11 13:24:23 +00:00

To properly link to the above mentioned issue: #110

witten commented

2018-12-12 03:55:38 +00:00

What were you thoughts on this comment?

My only concern with automatic initialization (with or without a prompt) is that it might not be desired behavior in certain circumstances, e.g. when not run in interactive mode. I’m thinking when you’re running borgmatic --create from a cron job, which is pretty common.

Also, could you talk a little about the use case where auto-initialization would be useful for you? Thanks.

What were you thoughts on this comment? > My only concern with automatic initialization (with or without a prompt) is that it might not be desired behavior in certain circumstances, e.g. when not run in interactive mode. I’m thinking when you’re running borgmatic --create from a cron job, which is pretty common. Also, could you talk a little about the use case where auto-initialization would be useful for you? Thanks.

thomasleveil commented

2018-12-12 07:34:14 +00:00

Also, could you talk a little about the use case where auto-initialization would be useful for you?

For my part, I've built a docker image with borg+bormatic and some shell script to do auto-initialization. I put that docker image in all our team docker-compose projects to backup docker volumes data.
Auto-initialization is very good for the ease of setting up backup on our projects. The documentation for my team members is very light: just append the backup container to the docker project with a few environment variables and it's done.

Having Bormatic taking care of initialization would ease the maintenance of this backup docker image: fewer shell scripts

> Also, could you talk a little about the use case where auto-initialization would be useful for you? For my part, I've built a docker image with borg+bormatic and some shell script to do auto-initialization. I put that docker image in all our team docker-compose projects to backup docker volumes data. Auto-initialization is very good for the ease of setting up backup on our projects. The documentation for my team members is very light: just append the backup container to the docker project with a few environment variables and it's done. Having Bormatic taking care of initialization would ease the maintenance of this backup docker image: fewer shell scripts

varac commented

2018-12-12 08:41:11 +00:00

@witten: Before using borgmatic we hat a backup script that would check if the repo existed and if not, it would get initialised:

attic list $REPOSITORY > /dev/null || attic init $REPOSITORY

I'd like to offload this task to borgmatic, with a similar reason as @thomasleveil: The less logic you need to bake in your custom scripts, the better. This should be a job of borgmatic.

@witten: Before using borgmatic we hat a backup script that would check if the repo existed and if not, it would get initialised: attic list $REPOSITORY > /dev/null || attic init $REPOSITORY I'd like to offload this task to borgmatic, with a similar reason as @thomasleveil: The less logic you need to bake in your custom scripts, the better. This should be a job of borgmatic.

witten commented

2018-12-23 23:47:30 +00:00

What do you all think about the edge case with auto-initialization where the repository originally exists, and then somehow accidentally gets deleted? At that point, the repository will get automatically recreated completely silently, and the user will not realize that they've actually had a major loss of archives.

Any ideas on ways to mitigate that?

Other questions: borgmatic --init does accept a few optional parameters and one required parameter (--encryption). How will the values of those get determined automatically?

What do you all think about the edge case with auto-initialization where the repository originally exists, and then somehow accidentally gets deleted? At that point, the repository will get automatically recreated completely silently, and the user will not realize that they've actually had a major loss of archives. Any ideas on ways to mitigate that? Other questions: `borgmatic --init` does accept a few optional parameters and one required parameter (`--encryption`). How will the values of those get determined automatically?

thomasleveil commented

2018-12-24 11:20:44 +00:00

What do you all think about the edge case with auto-initialization where the repository originally exists, and then somehow accidentally gets deleted? At that point, the repository will get automatically recreated completely silently, and the user will not realize that they’ve actually had a major loss of archives.

Any ideas on ways to mitigate that?

Well, if the repository data is lost, it does not harm to recreate it and push new backups to it. At least it ensures our data is not only existing in our app for too long.

To me, the responsibility of borgmatic is to make backups and upload them. If the repository does not exists, I have other monitoring tools which have for responsibility to check my borg repositories (last update time, size, etc). So I don't care if borgmatic is not alerting me if it had to recreate the repository.
However, borgmatic should log that it did create the repository.

Other questions: borgmatic --init does accept a few optional parameters and one required parameter (--encryption). How will the values of those get determined automatically?

In my current setup, this is done with environment variables, but I guess borgmatic could make use of its yaml config file for reading those values

> What do you all think about the edge case with auto-initialization where the repository originally exists, and then somehow accidentally gets deleted? At that point, the repository will get automatically recreated completely silently, and the user will not realize that they’ve actually had a major loss of archives. > > Any ideas on ways to mitigate that? Well, if the repository data is lost, it does not harm to recreate it and push new backups to it. At least it ensures our data is not only existing in our app for too long. To me, the responsibility of borgmatic is to make backups and upload them. If the repository does not exists, I have other monitoring tools which have for responsibility to check my borg repositories (last update time, size, etc). So I don't care if borgmatic is not alerting me if it had to recreate the repository. However, borgmatic should log that it did create the repository. > Other questions: borgmatic --init does accept a few optional parameters and one required parameter (--encryption). How will the values of those get determined automatically? In my current setup, this is done with environment variables, but I guess borgmatic could make use of its yaml config file for reading those values

witten commented

2018-12-24 19:09:07 +00:00

Okay. Here's another idea for the user interface for this feature: Make use of the existing borgmatic command-line flags for repository initialization to make this feature opt-in. For instance, instead of just:

borgmatic --create

You would be able to do:

borgmatic --init --encryption repokey --create

The idea being that if the repository doesn't exist, it'll create it with the requested encryption mode and any other initialization flags. But if it already exists, it'll log that fact and continue on with the backup as per usual.
Thoughts?

Okay. Here's another idea for the user interface for this feature: Make use of the existing borgmatic command-line flags for repository initialization to make this feature opt-in. For instance, instead of just: ```bash borgmatic --create ``` You would be able to do: ```bash borgmatic --init --encryption repokey --create ``` The idea being that if the repository doesn't exist, it'll create it with the requested encryption mode and any other initialization flags. But if it already exists, it'll log that fact and continue on with the backup as per usual. Thoughts?

thomasleveil commented

2018-12-24 20:33:12 +00:00

That sounds reasonable

witten added the

design finalized

label 2018-12-24 20:43:09 +00:00

witten commented

2018-12-24 20:43:28 +00:00

Thanks!

witten referenced this issue from a commit

2018-12-25 06:33:32 +00:00

With --init command-line flag, if a repository already exists, proceed without erroring (#117).

witten commented

2018-12-25 06:45:39 +00:00

Implemented and released in borgmatic 1.2.13. The usage is as described above: Just mix and match --init and --create or other command-line flags to get the behavior you're looking for. Enjoy!

Implemented and released in borgmatic 1.2.13. The usage is as described above: Just mix and match `--init` and `--create` or other command-line flags to get the behavior you're looking for. Enjoy!

witten closed this issue

2018-12-25 06:45:39 +00:00

varac commented

2018-12-25 18:04:39 +00:00

Awesome, thx !

Sign in to join this conversation.