Override location.repositories yields error for remote repository #471

Closed
opened 9 months ago by tikeyknax · 4 comments

What I'm trying to do and why

Trying to override the location.repositories on the commandline.

I have a borgmatic config set up to backup files from my laptop to my NAS. This works fine and is a really nice backup solution. Now, I have an external HDD attached to the NAS that I only connect to the NAS (and to the power) on certain occasions. So I want to run the same backup as I usually run on the laptop to backup my files on this external HDD. For this, I want to override the location.repositories value.

I could, of course, duplicate my config and put the new location there. However, I'm using several config files to backup my files with different settings, so I would have to copy and change them all.

Steps to reproduce (if a bug)

borgmatic --override location.repositories=["backupuser@my_nas:/media/backup_ext_sd/my_repo.borg"]

Actual behavior (if a bug)

Traceback (most recent call last):
File "/usr/local/bin/borgmatic", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 798, in main
configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 597, in load_configurations
configs[config_filename] = validate.parse_configuration(
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/validate.py", line 110, in parse_configuration
override.apply_overrides(config, overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 68, in apply_overrides
overrides = parse_overrides(raw_overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 54, in parse_overrides
return tuple(
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 55, in
(tuple(raw_keys.split('.')), convert_value_type(value))
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 30, in convert_value_type
return ruamel.yaml.YAML(typ='safe').load(io.StringIO(value))
File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/main.py", line 434, in load
return constructor.get_single_data()
File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 119, in get_single_data
node = self.composer.get_single_node()
File "_ruamel_yaml.pyx", line 703, in _ruamel_yaml.CParser.get_single_node
File "_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event
ruamel.yaml.scanner.ScannerError: while scanning a plain scalar
in "", line 1, column 2
found unexpected ':'
in "", line 1, column 21

Expected behavior (if a bug)

It might very well be that I'm just using the override functionality wrongly. I have not found an example for my use case in the documentation.

Environment

borgmatic version: 1.5.21

borgmatic installation method: pip3

Borg version: borg 1.1.15

Python version: 3.8.5

operating system and version: Linux tikey-TUXEDO 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

(base) tikey@tikey-TUXEDO:~$ pip3 list | grep ruamel
ruamel.yaml 0.17.17
ruamel.yaml.clib 0.2.2

#### What I'm trying to do and why Trying to override the location.repositories on the commandline. I have a borgmatic config set up to backup files from my laptop to my NAS. This works fine and is a really nice backup solution. Now, I have an external HDD attached to the NAS that I only connect to the NAS (and to the power) on certain occasions. So I want to run the same backup as I usually run on the laptop to backup my files on this external HDD. For this, I want to override the location.repositories value. I could, of course, duplicate my config and put the new location there. However, I'm using several config files to backup my files with different settings, so I would have to copy and change them all. #### Steps to reproduce (if a bug) borgmatic --override location.repositories=["backupuser@my_nas:/media/backup_ext_sd/my_repo.borg"] #### Actual behavior (if a bug) Traceback (most recent call last): File "/usr/local/bin/borgmatic", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 798, in main configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 597, in load_configurations configs[config_filename] = validate.parse_configuration( File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/validate.py", line 110, in parse_configuration override.apply_overrides(config, overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 68, in apply_overrides overrides = parse_overrides(raw_overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 54, in parse_overrides return tuple( File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 55, in <genexpr> (tuple(raw_keys.split('.')), convert_value_type(value)) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 30, in convert_value_type return ruamel.yaml.YAML(typ='safe').load(io.StringIO(value)) File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/main.py", line 434, in load return constructor.get_single_data() File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 119, in get_single_data node = self.composer.get_single_node() File "_ruamel_yaml.pyx", line 703, in _ruamel_yaml.CParser.get_single_node File "_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event ruamel.yaml.scanner.ScannerError: while scanning a plain scalar in "<file>", line 1, column 2 found unexpected ':' in "<file>", line 1, column 21 #### Expected behavior (if a bug) It might very well be that I'm just using the override functionality wrongly. I have not found an example for my use case in the documentation. #### Environment **borgmatic version:** 1.5.21 **borgmatic installation method:** pip3 **Borg version:** borg 1.1.15 **Python version:** 3.8.5 **operating system and version:** Linux tikey-TUXEDO 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux (base) tikey@tikey-TUXEDO:~$ pip3 list | grep ruamel ruamel.yaml 0.17.17 ruamel.yaml.clib 0.2.2
Owner

Quick note that it may be easier to accomplish occasional backups to a removable drive with the approach described in the borgmatic documentation rather than a command-line override.

Having said that, the command-line override approach should work too! And certainly shouldn't be blowing up with a traceback if it does error. It's interesting because I don't get an error with the same override on my machine. I'm on the same version of ruamel.yaml but a different ruamel.yaml.clib (0.2.6). I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?

Quick note that it may be easier to accomplish occasional backups to a removable drive with [the approach described in the borgmatic documentation](https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/) rather than a command-line override. Having said that, the command-line override approach should work too! And certainly shouldn't be blowing up with a traceback if it does error. It's interesting because I don't get an error with the same override on my machine. I'm on the same version of `ruamel.yaml` but a different `ruamel.yaml.clib` (0.2.6). I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?
Owner

Also, I've just pushed a change to master so that the error message produced in this case is much nicer instead of an ugly traceback. But I believe it will still error on your system.

Also, I've just pushed a change to master so that the error message produced in this case is much nicer instead of an ugly traceback. But I believe it will still error on your system.
Poster

I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?

Hey, thanks for taking a look into this issue and sorry for the late reply. Your hint with the quotes was spot on. Single quotes alone are not enough but if I use double quotes + single quotes, it seems to work.

borgmatic --override location.repositories=["'backupuser@my_nas:/media/backup_ext_sd/my_repo.borg'"]

Thanks for your help.

> I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes? Hey, thanks for taking a look into this issue and sorry for the late reply. Your hint with the quotes was spot on. Single quotes alone are not enough but if I use double quotes + single quotes, it seems to work. ``` borgmatic --override location.repositories=["'backupuser@my_nas:/media/backup_ext_sd/my_repo.borg'"] ``` Thanks for your help.
Owner

Glad to hear that did it!

Glad to hear that did it!
witten closed this issue 8 months ago
witten added the
question / support
label 8 months ago
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#471
Loading…
There is no content yet.