When running multiple configs, do not fail at first error #116

New Issue

Closed

opened 2018-12-09 19:27:43 +00:00 by nicoulaj · 6 comments

nicoulaj commented 2018-12-09 19:27:43 +00:00

Contributor

Copy Link

I put multiple configs in /etc/borgmatic.d, and my systemd service runs them all at once (by just calling borgmatic).

The issue is, if one failure occurs borgmatic just stops here. I would expect borgmatic to try them all.

Ideally it could also display a summary of which failed and which succeeded at the end.

I put multiple configs in `/etc/borgmatic.d`, and my systemd service runs them all at once (by just calling borgmatic). The issue is, if one failure occurs borgmatic just stops here. I would expect borgmatic to try them all. Ideally it could also display a summary of which failed and which succeeded at the end.

nicoulaj changed title from When running multiple configs, do not fail at first to When running multiple configs, do not fail at first error 2018-12-09 19:28:11 +00:00

witten commented 2018-12-09 22:37:07 +00:00

Owner

Copy Link

Thanks for reporting. Right now, any exception due to calling Borg gets immediately bubbled up to main() and printed to the console. So one way I could see to implement your request would be: Catch any exception that occurs when calling Borg on a given configuration file, add the exception to a list, and then proceed to the next configuration file. Finally, upon exit, print out any accumulated exceptions from that list to the console.

Anyway, could you say a little more about your use case? For instance, what are your different borgmatic config files used for? Why does one of them sometimes fail? And why do you want the other configurations to continue running even if the one fails? Thanks.

Thanks for reporting. Right now, any exception due to calling Borg gets immediately bubbled up to `main()` and printed to the console. So one way I could see to implement your request would be: Catch any exception that occurs when calling Borg on a given configuration file, add the exception to a list, and then proceed to the next configuration file. Finally, upon exit, print out any accumulated exceptions from that list to the console. Anyway, could you say a little more about your use case? For instance, what are your different borgmatic config files used for? Why does one of them sometimes fail? And why do you want the other configurations to continue running even if the one fails? Thanks.

nicoulaj commented 2018-12-10 13:36:02 +00:00

Author

Contributor

Copy Link

Anyway, could you say a little more about your use case? For instance, what are your different borgmatic config files used for?

It's just for separating different data volumes (eg: work documents, personal documents, system config). I want to keep them isolate for several reasons:

• minimize risk if borg repo somehow gets corrupted
• use different retention policies for each repo
• repos can have different lifecycles (eg: system can be reinstalled, documents continue forever)
• smaller archives for browsing/extracting borg repos

Why does one of them sometimes fail? And why do you want the other configurations to continue running even if the one fails?

In this case it was just bad configuration, but I upload the borg repo to backblaze B2 as a post step, so you could imagine it can fail for various reasons (connectivity issues, storage quota reached...)

Anyway, maybe I am doing it wrong, I could have 1 systemd timer+service per repo, but I want them to run sequentially, so it's a little more annoying to manage dans just being able to drop configs in /etc/borgmatic.d and have it run automatically.

> Anyway, could you say a little more about your use case? For instance, what are your different borgmatic config files used for? It's just for separating different data volumes (eg: work documents, personal documents, system config). I want to keep them isolate for several reasons: * minimize risk if borg repo somehow gets corrupted * use different retention policies for each repo * repos can have different lifecycles (eg: system can be reinstalled, documents continue forever) * smaller archives for browsing/extracting borg repos > Why does one of them sometimes fail? And why do you want the other configurations to continue running even if the one fails? In this case it was just bad configuration, but I upload the borg repo to backblaze B2 as a post step, so you could imagine it can fail for various reasons (connectivity issues, storage quota reached...) Anyway, maybe I am doing it wrong, I could have 1 systemd timer+service per repo, but I want them to run sequentially, so it's a little more annoying to manage dans just being able to drop configs in `/etc/borgmatic.d` and have it run automatically.

nicoulaj commented 2018-12-10 13:40:22 +00:00

Author

Contributor

Copy Link

Unrelated: I think you have a config issue with your gitea, tried to edit my previous comment and cannot save it, the Firefox Javascript console shows:

ReferenceError: issuesTribute is not defined[Learn More] index.js:690:17
initRepository/<https://projects.torsion.org/js/index.js:690:17
dispatch https://projects.torsion.org/vendor/plugins/jquery/jquery.min.js:3:12392
add/r.handle https://projects.torsion.org/vendor/plugins/jquery/jquery.min.js:3:9156
Source map error: request failed with status 404
Resource URL: https://projects.torsion.org/vendor/plugins/tribute/tribute.min.js
Source Map URL: tribute.min.js.map[Learn More]

Unrelated: I think you have a config issue with your gitea, tried to edit my previous comment and cannot save it, the Firefox Javascript console shows: ``` ReferenceError: issuesTribute is not defined[Learn More] index.js:690:17 initRepository/<https://projects.torsion.org/js/index.js:690:17 dispatch https://projects.torsion.org/vendor/plugins/jquery/jquery.min.js:3:12392 add/r.handle https://projects.torsion.org/vendor/plugins/jquery/jquery.min.js:3:9156 Source map error: request failed with status 404 Resource URL: https://projects.torsion.org/vendor/plugins/tribute/tribute.min.js Source Map URL: tribute.min.js.map[Learn More] ```

witten commented 2018-12-11 01:43:53 +00:00

Owner

Copy Link

Nope, your use case makes sense.. I just wanted to make sure I understood it before trying to implement anything.

As for the Gitea error, it sounds like you're hitting this: https://github.com/go-gitea/gitea/issues/4755 :(

Nope, your use case makes sense.. I just wanted to make sure I understood it before trying to implement anything. As for the Gitea error, it sounds like you're hitting this: https://github.com/go-gitea/gitea/issues/4755 :(

witten added the

design finalized

label 2018-12-11 01:44:01 +00:00

witten referenced this issue from a commit 2018-12-25 23:29:16 +00:00

When running multiple configuration files, attempt all of them even if one errors (#116).

witten commented 2018-12-25 23:31:14 +00:00

Owner

Copy Link

Implemented in master. This enhancement will go out as part of the next release.

Implemented in master. This enhancement will go out as part of the next release.

witten closed this issue 2018-12-25 23:31:14 +00:00

witten commented 2018-12-26 05:14:26 +00:00

Owner

Copy Link

Just released in borgmatic 1.2.14.

Just released in borgmatic 1.2.14.

👍 1

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#116

No description provided.