Option to let borgmatic fail if a source_directory is not present #501

New Issue

Closed

opened 2022-02-21 09:44:52 +00:00 by level420 · 5 comments

level420 commented 2022-02-21 09:44:52 +00:00

Copy Link

What I'm trying to do and why

In some situations it is desireable to let borgmatic create fail with an exit code gt. 0 if a given source_directory entry is not present on the file system.

This is when having a misconfiguration like a typo in the path like /erc vs. /etc.

Other notes / implementation ideas

This could be done either by an additional failonmissingsourcepath or similar command line parameter or an attribute in the borgmatic yaml-File.

Environment

borgmatic version: 1.5.22

borgmatic installation method: pip3 install --upgrade borgmatic

Borg version: 1.1.17

Python version: 3.6.8

operating system and version: AlmaLinux release 8.5 (Arctic Sphynx)

#### What I'm trying to do and why In some situations it is desireable to let `borgmatic create` fail with an exit code gt. 0 if a given `source_directory` entry is not present on the file system. This is when having a misconfiguration like a typo in the path like `/erc` vs. `/etc`. #### Other notes / implementation ideas This could be done either by an additional `failonmissingsourcepath` or similar command line parameter or an attribute in the borgmatic yaml-File. #### Environment **borgmatic version:** 1.5.22 **borgmatic installation method:** pip3 install --upgrade borgmatic **Borg version:** 1.1.17 **Python version:** 3.6.8 **operating system and version:** AlmaLinux release 8.5 (Arctic Sphynx)

witten commented 2022-03-05 22:52:38 +00:00

Owner

Copy Link

This makes sense to me as a feature. Thanks for filing it! There was a brief period of time when borgmatic did error on missing source directories, but that was an unintentional regression (see #387). I could see having a configuration option as you suggest to optionally turn this behavior back on.

This makes sense to me as a feature. Thanks for filing it! There was a brief period of time when borgmatic did error on missing source directories, but that was an unintentional regression (see #387). I could see having a configuration option as you suggest to optionally turn this behavior back on.

witten added the

good first issue

label 2023-02-04 17:40:03 +00:00

diivi commented 2023-03-15 19:43:48 +00:00

Collaborator

Copy Link

I'd like to work on this, do you want Borgmatic to fail only when a parameter like the mentioned failonmissingsourcepath is provided? I believe it should fail with an exit code > 0 everytime the source_directories entry is not present.

I am thinking of editing the create.py::create_archive() function and adding a for loop to check if all directories exist in the provided source_directories(and throw an error if even one of those does not exist), before this line:

    sources = deduplicate_directories(
        map_directories_to_devices(
            expand_directories(
                tuple(location_config.get('source_directories', ())) + borgmatic_source_directories
            )
        ),
        additional_directory_devices=map_directories_to_devices(
            expand_directories(pattern_root_directories(location_config.get('patterns')))
        ),
    )

Could you help me understand if this would be the correct approach to do this? Thanks!

I'd like to work on this, do you want Borgmatic to fail only when a parameter like the mentioned `failonmissingsourcepath` is provided? I believe it should fail with an exit code > 0 everytime the `source_directories` entry is not present. I am thinking of editing the `create.py::create_archive()` function and adding a for loop to check if all directories exist in the provided `source_directories`(and throw an error if even one of those does not exist), before this line: ```python sources = deduplicate_directories( map_directories_to_devices( expand_directories( tuple(location_config.get('source_directories', ())) + borgmatic_source_directories ) ), additional_directory_devices=map_directories_to_devices( expand_directories(pattern_root_directories(location_config.get('patterns'))) ), ) ``` Could you help me understand if this would be the correct approach to do this? Thanks!

witten commented 2023-03-16 17:52:43 +00:00

Owner

Copy Link

Given that Borg's default behavior is not to fail on missing directories, I think that should probably continue to be borgmatic's default as well. I can think of use cases where a source directory is only present depending on whether a device or network share is mounted, whether a particular application is running, etc. But I'm open to arguments for the other approach, as well.

If there is an option controlling this behavior, I'd suggest calling it something like source_directories_must_exist in the location section.

I am thinking of editing the create.py::create_archive() function and adding a for loop to check if all directories exist in the provided source_directories(and throw an error if even one of those does not exist), before this line:

Yes, that sounds like the perfect place to do that! It might make sense to put that logic into a separate function though and then call it from there.

Given that Borg's default behavior is *not* to fail on missing directories, I think that should probably continue to be borgmatic's default as well. I can think of use cases where a source directory is only present depending on whether a device or network share is mounted, whether a particular application is running, etc. But I'm open to arguments for the other approach, as well. If there is an option controlling this behavior, I'd suggest calling it something like `source_directories_must_exist` in the `location` section. > I am thinking of editing the create.py::create_archive() function and adding a for loop to check if all directories exist in the provided source_directories(and throw an error if even one of those does not exist), before this line: Yes, that sounds like the perfect place to do that! It might make sense to put that logic into a separate function though and then call it from there.

👍 1

witten referenced this issue from a commit 2023-03-18 21:08:56 +00:00

Add "source_directories_must_exist" option to NEWS (#501).

witten commented 2023-03-18 21:11:10 +00:00

Owner

Copy Link

Thanks to @diivi, this is now implemented in master as a source_directories_must_exist option and will be part of the next release!

Thanks to @diivi, this is now [implemented](https://github.com/borgmatic-collective/borgmatic/pull/53) in master as a `source_directories_must_exist` option and will be part of the next release!

🎉 1

witten closed this issue 2023-03-18 21:11:10 +00:00

witten referenced this issue 2023-03-27 03:14:46 +00:00

Hook "on_warning" #606

witten referenced this issue 2023-03-27 19:40:15 +00:00

Provide more influence on the decision about if a backup is to be considered a success or failure. #447

witten commented 2023-03-27 23:22:31 +00:00

Owner

Copy Link

Released in borgmatic 1.7.10!

Released in borgmatic 1.7.10!

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

3 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#501

No description provided.