borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 53 Pull Requests 1 Actions Packages Releases 121 Activity

"Permission denied" error does not cause hard failure, but rather turns operation successful #486

New Issue
Closed
opened 2021-12-30 22:27:45 +00:00 by Alexander-Shukaev · 4 comments
Alexander-Shukaev commented 2021-12-30 22:27:45 +00:00
Copy Link

What I'm trying to do and why

Running borgmatic with exclude_from.

Steps to reproduce (if a bug)

Run borgmatic with exclude_from option, where a file would have such permissions that borgmatic would not be able to read it.

Actual behavior (if a bug)

borgmatic.service[186908]: ...: Creating archive
borgmatic.service[186908]: Traceback (most recent call last):
borgmatic.service[186908]:   File "/usr/bin/borg", line 33, in <module>
borgmatic.service[186908]:     sys.exit(load_entry_point('borgbackup==1.1.17', 'console_scripts', 'borg')())
borgmatic.service[186908]:   File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4692, in main
borgmatic.service[186908]:     args = archiver.get_args(sys.argv, os.environ.get('SSH_ORIGINAL_COMMAND'))
borgmatic.service[186908]:   File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4489, in get_args
borgmatic.service[186908]:     result = self.parse_args(argv[1:])
borgmatic.service[186908]:   File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4535, in parse_args
borgmatic.service[186908]:     args = parser.parse_args(args or ['-h'])
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1821, in parse_args
borgmatic.service[186908]:     args, argv = self.parse_known_args(args, namespace)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1854, in parse_known_args
borgmatic.service[186908]:     namespace, args = self._parse_known_args(args, namespace)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 2045, in _parse_known_args
borgmatic.service[186908]:     positionals_end_index = consume_positionals(start_index)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 2022, in consume_positionals
borgmatic.service[186908]:     take_action(action, args)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1931, in take_action
borgmatic.service[186908]:     action(self, namespace, argument_values, option_string)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1210, in __call__
borgmatic.service[186908]:     subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1854, in parse_known_args
borgmatic.service[186908]:     namespace, args = self._parse_known_args(args, namespace)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 2063, in _parse_known_args
borgmatic.service[186908]:     start_index = consume_optional(start_index)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 2003, in consume_optional
borgmatic.service[186908]:     take_action(action, args, option_string)
borgmatic.service[186908]:   File "/usr/lib/python3.10/argparse.py", line 1931, in take_action
borgmatic.service[186908]:     action(self, namespace, argument_values, option_string)
borgmatic.service[186908]:   File "/usr/lib/python3.10/site-packages/borg/patterns.py", line 56, in __call__
borgmatic.service[186908]:     with open(filename) as f:
borgmatic.service[186908]: PermissionError: [Errno 13] Permission denied: '/.../excludes'

After that, it would go ahead and treat create operation successful and even run after_backup hook instead of on_error one. I actually had failing backups going unnoticed for a while because of that. I have corresponding email notifications set up on both hooks, and in this case it was fooling me to believe that everything is OK, since the emails would reflect success rather than failure. I only realized that something is wrong when I accidentally noticed security not being incremented anymore for a while.

Expected behavior (if a bug)

All fatal errors should unconditinally fail the backup process, report proper exit status, and run corresponding hook. I guess some review of other code paths, not just this one, might be needed.

Environment

borgmatic version: 1.5.21

Borg version: 1.1.17

Python version: 3.10.1

#### What I'm trying to do and why Running `borgmatic` with `exclude_from`. #### Steps to reproduce (if a bug) Run `borgmatic` with `exclude_from` option, where a file would have such permissions that `borgmatic` would not be able to read it. #### Actual behavior (if a bug) ``` borgmatic.service[186908]: ...: Creating archive borgmatic.service[186908]: Traceback (most recent call last): borgmatic.service[186908]: File "/usr/bin/borg", line 33, in <module> borgmatic.service[186908]: sys.exit(load_entry_point('borgbackup==1.1.17', 'console_scripts', 'borg')()) borgmatic.service[186908]: File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4692, in main borgmatic.service[186908]: args = archiver.get_args(sys.argv, os.environ.get('SSH_ORIGINAL_COMMAND')) borgmatic.service[186908]: File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4489, in get_args borgmatic.service[186908]: result = self.parse_args(argv[1:]) borgmatic.service[186908]: File "/usr/lib/python3.10/site-packages/borg/archiver.py", line 4535, in parse_args borgmatic.service[186908]: args = parser.parse_args(args or ['-h']) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1821, in parse_args borgmatic.service[186908]: args, argv = self.parse_known_args(args, namespace) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1854, in parse_known_args borgmatic.service[186908]: namespace, args = self._parse_known_args(args, namespace) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 2045, in _parse_known_args borgmatic.service[186908]: positionals_end_index = consume_positionals(start_index) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 2022, in consume_positionals borgmatic.service[186908]: take_action(action, args) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1931, in take_action borgmatic.service[186908]: action(self, namespace, argument_values, option_string) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1210, in __call__ borgmatic.service[186908]: subnamespace, arg_strings = parser.parse_known_args(arg_strings, None) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1854, in parse_known_args borgmatic.service[186908]: namespace, args = self._parse_known_args(args, namespace) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 2063, in _parse_known_args borgmatic.service[186908]: start_index = consume_optional(start_index) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 2003, in consume_optional borgmatic.service[186908]: take_action(action, args, option_string) borgmatic.service[186908]: File "/usr/lib/python3.10/argparse.py", line 1931, in take_action borgmatic.service[186908]: action(self, namespace, argument_values, option_string) borgmatic.service[186908]: File "/usr/lib/python3.10/site-packages/borg/patterns.py", line 56, in __call__ borgmatic.service[186908]: with open(filename) as f: borgmatic.service[186908]: PermissionError: [Errno 13] Permission denied: '/.../excludes' ``` After that, it would go ahead and treat `create` operation successful and even run `after_backup` hook instead of `on_error` one. I actually had failing backups going unnoticed for a while because of that. I have corresponding email notifications set up on both hooks, and in this case it was fooling me to believe that everything is OK, since the emails would reflect success rather than failure. I only realized that something is wrong when I accidentally noticed `security` not being incremented anymore for a while. #### Expected behavior (if a bug) All fatal errors should unconditinally fail the backup process, report proper exit status, and run corresponding hook. I guess some review of other code paths, not just this one, might be needed. #### Environment **borgmatic version:** 1.5.21 **Borg version:** 1.1.17 **Python version:** 3.10.1
witten commented 2022-03-07 19:31:48 +00:00
Owner
Copy Link

I've reproduced this problem locally. Thank you for reporting it!

I've reproduced this problem locally. Thank you for reporting it!
witten added the
bug
 label 2022-03-07 19:31:56 +00:00
witten commented 2022-03-07 19:41:05 +00:00
Owner
Copy Link

So I think what's going on here is Borg is returning a "warning" exit code instead of an "error" exit code, which borgmatic intentionally does not error on. One fix for this specific issue could be for borgmatic to try to read the file before handing it off to Borg.. and error if there is a permission denied error.

So I think what's going on here is Borg is returning a "warning" exit code instead of an "error" exit code, which borgmatic intentionally does not error on. One fix for this specific issue could be for borgmatic to try to read the file before handing it off to Borg.. and error if there is a permission denied error.
witten commented 2022-03-07 23:32:38 +00:00
Owner
Copy Link

I implemented this fix for both exclude_from and patterns_from in borgmatic master. This will be part of the next release. Thanks for reporting it!

I implemented this fix for both `exclude_from` and `patterns_from` in borgmatic master. This will be part of the next release. Thanks for reporting it!
witten commented 2022-03-14 20:47:57 +00:00
Owner
Copy Link

Released in borgmatic 1.5.24.

Released in borgmatic 1.5.24.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#486
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?