borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 53 Pull Requests 1 Actions Packages Releases 121 Activity

jsonschema 3.2.0 traceback with 1.5.16 when including the umask stanza in the hooks section #437

New Issue
Closed
opened 2021-07-27 16:04:57 +00:00 by mihailim · 3 comments
mihailim commented 2021-07-27 16:04:57 +00:00
Copy Link

What I'm trying to do and why

Trying to set umask to 0027 when running before_backup hook scripts.

Steps to reproduce (if a bug)

location:
    source_directories:
        - /various
    exclude_patterns:
        - /root/.cache
    repositories:
        - /path/to/repo.borg
        - ssh://hostspec/remoterepo.borg
storage:
    encryption_passphrase: "redacted"
    checkpoint_interval: 600
    compression: zstd,11
    ssh_command: ssh -i /path/to/pkey
    borg_cache_directory: /path/to/cachedir
    lock_wait: 60
    archive_name_format: '{hostname}-repo-{now}'
retention:
    keep_daily: 730
    keep_weekly: 520
    keep_monthly: 120
    keep_yearly: 10
    prefix: '{hostname}-repo-'
consistency:
    checks:
        - archives
    check_last: 3
    prefix: '{hostname}-repo-'
hooks:
    umask: 0027
    before_backup:
        - /path/to/hookscript

Actual behavior (if a bug)

jsonschema complains with jsonschema.exceptions.UndefinedTypeCheck: Type 'scalar' is unknown to this type checker (see full traceback below). Using the latest stable jsonschema 3.2.0 under Python 3.6.9.

This seems to be caused by borgmatic/config/schema.yaml line 790, where umask is defined as type: scalar which isn't known to jsonschema.

Full traceback:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/jsonschema/_types.py", line 91, in is_type
    fn = self._type_checkers[type]
  File "/usr/lib/python3/dist-packages/pyrsistent/_pmap.py", line 71, in __getitem__
    return PMap._getitem(self._buckets, key)
  File "/usr/lib/python3/dist-packages/pyrsistent/_pmap.py", line 68, in _getitem
    raise KeyError(key)
KeyError: 'scalar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 357, in is_type
    return self.TYPE_CHECKER.is_type(instance, type)
  File "/usr/lib/python3/dist-packages/jsonschema/_types.py", line 93, in is_type
    raise UndefinedTypeCheck(type)
jsonschema.exceptions.UndefinedTypeCheck: Type 'scalar' is unknown to this type checker

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/borgmatic", line 11, in <module>
    load_entry_point('borgmatic==1.5.16', 'console_scripts', 'borgmatic')()
  File "/usr/lib/python3/dist-packages/borgmatic/commands/borgmatic.py", line 779, in main
    configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides)
  File "/usr/lib/python3/dist-packages/borgmatic/commands/borgmatic.py", line 579, in load_configurations
    config_filename, validate.schema_filename(), overrides
  File "/usr/lib/python3/dist-packages/borgmatic/config/validate.py", line 114, in parse_configuration
    validation_errors = tuple(validator.iter_errors(config))
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 286, in properties
    schema_path=property,
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 344, in descend
    for error in self.iter_errors(instance, schema):
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 286, in properties
    schema_path=property,
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 344, in descend
    for error in self.iter_errors(instance, schema):
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 272, in type
    if not any(validator.is_type(instance, type) for type in types):
  File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 272, in <genexpr>
    if not any(validator.is_type(instance, type) for type in types):
  File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 359, in is_type
    raise exceptions.UnknownType(type, instance, self.schema)
jsonschema.exceptions.UnknownType: 
Unknown type 'scalar' for validator with schema:

Expected behavior (if a bug)

No crash.

Other notes / implementation ideas

Can be worked around by setting the umask in the hook scripts and removing that stanza from the borgmatic config YAML, of course.

Environment

borgmatic version: 1.5.16
borgmatic installation method: Debian package (self-packaged as before, based on and backported from the upstream sid package)
Borg version: 1.1.15
Python version: 3.6.9
operating system and version: Ubuntu 18.04 LTS

#### What I'm trying to do and why Trying to set `umask` to `0027` when running `before_backup` hook scripts. #### Steps to reproduce (if a bug) location: source_directories: - /various exclude_patterns: - /root/.cache repositories: - /path/to/repo.borg - ssh://hostspec/remoterepo.borg storage: encryption_passphrase: "redacted" checkpoint_interval: 600 compression: zstd,11 ssh_command: ssh -i /path/to/pkey borg_cache_directory: /path/to/cachedir lock_wait: 60 archive_name_format: '{hostname}-repo-{now}' retention: keep_daily: 730 keep_weekly: 520 keep_monthly: 120 keep_yearly: 10 prefix: '{hostname}-repo-' consistency: checks: - archives check_last: 3 prefix: '{hostname}-repo-' hooks: umask: 0027 before_backup: - /path/to/hookscript #### Actual behavior (if a bug) jsonschema complains with `jsonschema.exceptions.UndefinedTypeCheck: Type 'scalar' is unknown to this type checker` (see full traceback below). Using the latest stable jsonschema 3.2.0 under Python 3.6.9. This seems to be caused by borgmatic/config/schema.yaml line 790, where `umask` is defined as `type: scalar` which isn't known to jsonschema. Full traceback: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/jsonschema/_types.py", line 91, in is_type fn = self._type_checkers[type] File "/usr/lib/python3/dist-packages/pyrsistent/_pmap.py", line 71, in __getitem__ return PMap._getitem(self._buckets, key) File "/usr/lib/python3/dist-packages/pyrsistent/_pmap.py", line 68, in _getitem raise KeyError(key) KeyError: 'scalar' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 357, in is_type return self.TYPE_CHECKER.is_type(instance, type) File "/usr/lib/python3/dist-packages/jsonschema/_types.py", line 93, in is_type raise UndefinedTypeCheck(type) jsonschema.exceptions.UndefinedTypeCheck: Type 'scalar' is unknown to this type checker During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/bin/borgmatic", line 11, in <module> load_entry_point('borgmatic==1.5.16', 'console_scripts', 'borgmatic')() File "/usr/lib/python3/dist-packages/borgmatic/commands/borgmatic.py", line 779, in main configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides) File "/usr/lib/python3/dist-packages/borgmatic/commands/borgmatic.py", line 579, in load_configurations config_filename, validate.schema_filename(), overrides File "/usr/lib/python3/dist-packages/borgmatic/config/validate.py", line 114, in parse_configuration validation_errors = tuple(validator.iter_errors(config)) File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors for error in errors: File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 286, in properties schema_path=property, File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 344, in descend for error in self.iter_errors(instance, schema): File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors for error in errors: File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 286, in properties schema_path=property, File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 344, in descend for error in self.iter_errors(instance, schema): File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 328, in iter_errors for error in errors: File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 272, in type if not any(validator.is_type(instance, type) for type in types): File "/usr/lib/python3/dist-packages/jsonschema/_validators.py", line 272, in <genexpr> if not any(validator.is_type(instance, type) for type in types): File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 359, in is_type raise exceptions.UnknownType(type, instance, self.schema) jsonschema.exceptions.UnknownType: Unknown type 'scalar' for validator with schema: #### Expected behavior (if a bug) No crash. #### Other notes / implementation ideas Can be worked around by setting the umask in the hook scripts and removing that stanza from the borgmatic config YAML, of course. #### Environment **borgmatic version:** 1.5.16 **borgmatic installation method:** Debian package (self-packaged as before, based on and backported from the upstream sid package) **Borg version:** 1.1.15 **Python version:** 3.6.9 **operating system and version:** Ubuntu 18.04 LTS
witten commented 2021-07-27 16:28:00 +00:00
Owner
Copy Link

Ugh, good catch! Thank you so much for taking the time to file this.

Ugh, good catch! Thank you so much for taking the time to file this.
❤️ 1
witten added the
bug
 label 2021-07-27 16:28:08 +00:00
witten commented 2021-07-27 17:05:24 +00:00
Owner
Copy Link

Fix is in master. You can use it to modify your schema manually if you need an immediate fix. About to do a bug fix release for this as well.

Fix is in master. You can use it to modify your schema manually if you need an immediate fix. About to do a bug fix release for this as well.
witten commented 2021-07-27 17:20:05 +00:00
Owner
Copy Link

Okay, the fix is released in borgmatic 1.5.17. Thanks again!

Okay, the fix is released in borgmatic 1.5.17. Thanks again!
❤️ 1
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#437
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?