borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 53 Pull Requests 2 Actions Packages Releases 121 Activity

Override location.repositories yields error for remote repository #471

New Issue
Closed
opened 2021-11-27 09:07:37 +00:00 by tikeyknax · 4 comments
tikeyknax commented 2021-11-27 09:07:37 +00:00
Copy Link

What I'm trying to do and why

Trying to override the location.repositories on the commandline.

I have a borgmatic config set up to backup files from my laptop to my NAS. This works fine and is a really nice backup solution. Now, I have an external HDD attached to the NAS that I only connect to the NAS (and to the power) on certain occasions. So I want to run the same backup as I usually run on the laptop to backup my files on this external HDD. For this, I want to override the location.repositories value.

I could, of course, duplicate my config and put the new location there. However, I'm using several config files to backup my files with different settings, so I would have to copy and change them all.

Steps to reproduce (if a bug)

borgmatic --override location.repositories=["backupuser@my_nas:/media/backup_ext_sd/my_repo.borg"]

Actual behavior (if a bug)

Traceback (most recent call last):
File "/usr/local/bin/borgmatic", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 798, in main
configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 597, in load_configurations
configs[config_filename] = validate.parse_configuration(
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/validate.py", line 110, in parse_configuration
override.apply_overrides(config, overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 68, in apply_overrides
overrides = parse_overrides(raw_overrides)
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 54, in parse_overrides
return tuple(
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 55, in
(tuple(raw_keys.split('.')), convert_value_type(value))
File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 30, in convert_value_type
return ruamel.yaml.YAML(typ='safe').load(io.StringIO(value))
File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/main.py", line 434, in load
return constructor.get_single_data()
File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 119, in get_single_data
node = self.composer.get_single_node()
File "_ruamel_yaml.pyx", line 703, in _ruamel_yaml.CParser.get_single_node
File "_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event
ruamel.yaml.scanner.ScannerError: while scanning a plain scalar
in "", line 1, column 2
found unexpected ':'
in "", line 1, column 21

Expected behavior (if a bug)

It might very well be that I'm just using the override functionality wrongly. I have not found an example for my use case in the documentation.

Environment

borgmatic version: 1.5.21

borgmatic installation method: pip3

Borg version: borg 1.1.15

Python version: 3.8.5

operating system and version: Linux tikey-TUXEDO 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

(base) tikey@tikey-TUXEDO:~$ pip3 list | grep ruamel
ruamel.yaml 0.17.17
ruamel.yaml.clib 0.2.2

#### What I'm trying to do and why Trying to override the location.repositories on the commandline. I have a borgmatic config set up to backup files from my laptop to my NAS. This works fine and is a really nice backup solution. Now, I have an external HDD attached to the NAS that I only connect to the NAS (and to the power) on certain occasions. So I want to run the same backup as I usually run on the laptop to backup my files on this external HDD. For this, I want to override the location.repositories value. I could, of course, duplicate my config and put the new location there. However, I'm using several config files to backup my files with different settings, so I would have to copy and change them all. #### Steps to reproduce (if a bug) borgmatic --override location.repositories=["backupuser@my_nas:/media/backup_ext_sd/my_repo.borg"] #### Actual behavior (if a bug) Traceback (most recent call last): File "/usr/local/bin/borgmatic", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 798, in main configs, parse_logs = load_configurations(config_filenames, global_arguments.overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/commands/borgmatic.py", line 597, in load_configurations configs[config_filename] = validate.parse_configuration( File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/validate.py", line 110, in parse_configuration override.apply_overrides(config, overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 68, in apply_overrides overrides = parse_overrides(raw_overrides) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 54, in parse_overrides return tuple( File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 55, in <genexpr> (tuple(raw_keys.split('.')), convert_value_type(value)) File "/usr/local/lib/python3.8/dist-packages/borgmatic/config/override.py", line 30, in convert_value_type return ruamel.yaml.YAML(typ='safe').load(io.StringIO(value)) File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/main.py", line 434, in load return constructor.get_single_data() File "/usr/local/lib/python3.8/dist-packages/ruamel/yaml/constructor.py", line 119, in get_single_data node = self.composer.get_single_node() File "_ruamel_yaml.pyx", line 703, in _ruamel_yaml.CParser.get_single_node File "_ruamel_yaml.pyx", line 904, in _ruamel_yaml.CParser._parse_next_event ruamel.yaml.scanner.ScannerError: while scanning a plain scalar in "<file>", line 1, column 2 found unexpected ':' in "<file>", line 1, column 21 #### Expected behavior (if a bug) It might very well be that I'm just using the override functionality wrongly. I have not found an example for my use case in the documentation. #### Environment **borgmatic version:** 1.5.21 **borgmatic installation method:** pip3 **Borg version:** borg 1.1.15 **Python version:** 3.8.5 **operating system and version:** Linux tikey-TUXEDO 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux (base) tikey@tikey-TUXEDO:~$ pip3 list | grep ruamel ruamel.yaml 0.17.17 ruamel.yaml.clib 0.2.2
witten commented 2021-11-29 20:21:59 +00:00
Owner
Copy Link

Quick note that it may be easier to accomplish occasional backups to a removable drive with the approach described in the borgmatic documentation rather than a command-line override.

Having said that, the command-line override approach should work too! And certainly shouldn't be blowing up with a traceback if it does error. It's interesting because I don't get an error with the same override on my machine. I'm on the same version of ruamel.yaml but a different ruamel.yaml.clib (0.2.6). I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?

Quick note that it may be easier to accomplish occasional backups to a removable drive with [the approach described in the borgmatic documentation](https://torsion.org/borgmatic/docs/how-to/backup-to-a-removable-drive-or-an-intermittent-server/) rather than a command-line override. Having said that, the command-line override approach should work too! And certainly shouldn't be blowing up with a traceback if it does error. It's interesting because I don't get an error with the same override on my machine. I'm on the same version of `ruamel.yaml` but a different `ruamel.yaml.clib` (0.2.6). I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?
witten commented 2021-11-29 20:50:42 +00:00
Owner
Copy Link

Also, I've just pushed a change to master so that the error message produced in this case is much nicer instead of an ugly traceback. But I believe it will still error on your system.

Also, I've just pushed a change to master so that the error message produced in this case is much nicer instead of an ugly traceback. But I believe it will still error on your system.
tikeyknax commented 2021-12-11 21:06:34 +00:00
Author
Copy Link

I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes?

Hey, thanks for taking a look into this issue and sorry for the late reply. Your hint with the quotes was spot on. Single quotes alone are not enough but if I use double quotes + single quotes, it seems to work.

borgmatic --override location.repositories=["'backupuser@my_nas:/media/backup_ext_sd/my_repo.borg'"]

Thanks for your help.

> I also wonder if your shell is eating those quotes. Have you tried single quotes instead of double quotes? Hey, thanks for taking a look into this issue and sorry for the late reply. Your hint with the quotes was spot on. Single quotes alone are not enough but if I use double quotes + single quotes, it seems to work. ``` borgmatic --override location.repositories=["'backupuser@my_nas:/media/backup_ext_sd/my_repo.borg'"] ``` Thanks for your help.
witten commented 2021-12-11 21:16:58 +00:00
Owner
Copy Link

Glad to hear that did it!

Glad to hear that did it!
witten added the
question / support
 label 2021-12-11 21:17:05 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#471
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?