wrong ssh_command used for multiple configurations #323

Closed
opened 2020-06-06 13:54:38 +00:00 by MaggeO · 4 comments

What I'm trying to do and why

I have multiple configuration files inside /etc/borgmatic.d/ with different ssh_command configs for internal and external networks

  1. For internal backups I didn't specify any special ssh_command, so that the default "ssh_command: ssh" should be used.
    # Command to use instead of just "ssh". This can be used to specify ssh options.
    # ssh_command:
  1. For external backups I have to specify the port to be used for the ssh_command, so my config looks like this:
    # Command to use instead of just "ssh". This can be used to specify ssh options.
    ssh_command: ssh -p 22222

Steps to reproduce (if a bug)

Create multiple config files with different ssh_command configs.

Actual behavior (if a bug)

Now it seems, that for some of my internal configs, which don't have any special ssh_command config, the wrong config us used. And then it fails cause on the used port is no ssh connection possible. This is not the case for every "internal" config. Perhaps it depends on the order the config are processed. So after the external config, which specifies the ssh_command with a different port, this ssh_comand will be used for every subsequent config file, even if there is no special ssh_command configured.

Include (sanitized) --verbosity 2 output if applicable.

user@backupserver:/mnt/BorgBackup/: Pruning archives
borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup
using builtin fallback logging configuration
35 self tests completed in 0.10 seconds
SSH command line: ['ssh', '-p', '22222', 'user@backupserver', '/usr/local/bin/borg', 'serve', '--umask=077', '--debug']
Remote: ssh: connect to host backupserver port 22222: Connection refused
Connection closed by remote host. Is borg working on the server?
Traceback (most recent call last):
  File "borg/archiver.py", line 4529, in main
  File "borg/archiver.py", line 4461, in run
  File "borg/archiver.py", line 146, in wrapper
  File "borg/remote.py", line 576, in __init__
borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server?
Platform: Linux ucs 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64
Linux: debian 9.12
Borg: 1.1.11  Python: CPython 3.5.9 msgpack: 0.5.6
PID: 74791  CWD: /etc/borgmatic.d
sys.argv: ['borg', 'prune', '--keep-weekly', '4', '--keep-monthly', '6', '--keep-daily', '7', '--prefix', '{hostname}-User_Maildata-', '--remote-path', '/usr/local/bin/borg', '--debug', '--show-rc', 'user@backupserver:/mnt/BorgBackup']
SSH_ORIGINAL_COMMAND: None
terminating with error status, rc 2
user@backupserver:/mnt/BorgBackup: Error running actions for repository
Command 'borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup' returned non-zero exit status 2
/etc/borgmatic.d/UCS_User_Maildata.yaml: No commands to run for on-error hook
/etc/borgmatic.d/UCS_User_Maildata.yaml: Error running configuration file
/etc/borgmatic.d/UCS_Server_Config.yaml: No commands to run for post-everything hook
/etc/borgmatic.d/UCS_Server_Config_Wendlingen.yaml: No commands to run for post-everything hook
/etc/borgmatic.d/UCS_User_Maildata.yaml: No commands to run for post-everything hook

summary:
/etc/borgmatic.d/UCS_Server_Config.yaml: Successfully ran configuration file
/etc/borgmatic.d/UCS_Server_Config_Wendlingen.yaml: Successfully ran configuration file
/etc/borgmatic.d/UCS_User_Maildata.yaml: Error running configuration file
user@backupserver:/mnt/BorgBackup: Error running actions for repository
using builtin fallback logging configuration
35 self tests completed in 0.10 seconds
SSH command line: ['ssh', '-p', '22222', 'user@backupserver', '/usr/local/bin/borg', 'serve', '--umask=077', '--debug']
Remote: ssh: connect to host syno1.bs.private port 22222: Connection refused
Connection closed by remote host. Is borg working on the server?
Traceback (most recent call last):
  File "borg/archiver.py", line 4529, in main
  File "borg/archiver.py", line 4461, in run
  File "borg/archiver.py", line 146, in wrapper
  File "borg/remote.py", line 576, in __init__
borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server?
Platform: Linux ucs 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64
Linux: debian 9.12
Borg: 1.1.11  Python: CPython 3.5.9 msgpack: 0.5.6
PID: 74791  CWD: /etc/borgmatic.d
sys.argv: ['borg', 'prune', '--keep-weekly', '4', '--keep-monthly', '6', '--keep-daily', '7', '--prefix', '{hostname}-User_Maildata-', '--remote-path', '/usr/local/bin/borg', '--debug', '--show-rc', 'user@backupserver:/mnt/BorgBackup']
SSH_ORIGINAL_COMMAND: None
terminating with error status, rc 2
Command 'borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup' returned non-zero exit status 2

Expected behavior (if a bug)

Use the standard ssh_command (ssh) if there is no special ssh_command configured. Even for multiple configuration files with different configurations.

Other notes / implementation ideas

If I specify the "default" ssh command also explicitly in the internal configuration, I didn't get the error any more.

    # Command to use instead of just "ssh". This can be used to specify ssh options.
    ssh_command: ssh

Environment

borgmatic version: [1.5.5]

borgmatic installation method: [sudo pip3 install --user --upgrade borgmatic]

Borg version: [borg 1.1.11]

Python version: [Python 3.5.3]

operating system and version: [Debian 9]

#### What I'm trying to do and why I have multiple configuration files inside /etc/borgmatic.d/ with different ssh_command configs for internal and external networks 1. For internal backups I didn't specify any special ssh_command, so that the default "ssh_command: ssh" should be used. ``` # Command to use instead of just "ssh". This can be used to specify ssh options. # ssh_command: ``` 2. For external backups I have to specify the port to be used for the ssh_command, so my config looks like this: ``` # Command to use instead of just "ssh". This can be used to specify ssh options. ssh_command: ssh -p 22222 ``` #### Steps to reproduce (if a bug) Create multiple config files with different ssh_command configs. #### Actual behavior (if a bug) Now it seems, that for some of my internal configs, which don't have any special ssh_command config, the wrong config us used. And then it fails cause on the used port is no ssh connection possible. This is not the case for every "internal" config. Perhaps it depends on the order the config are processed. So after the external config, which specifies the ssh_command with a different port, this ssh_comand will be used for every subsequent config file, even if there is no special ssh_command configured. Include (sanitized) `--verbosity 2` output if applicable. ``` user@backupserver:/mnt/BorgBackup/: Pruning archives borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup using builtin fallback logging configuration 35 self tests completed in 0.10 seconds SSH command line: ['ssh', '-p', '22222', 'user@backupserver', '/usr/local/bin/borg', 'serve', '--umask=077', '--debug'] Remote: ssh: connect to host backupserver port 22222: Connection refused Connection closed by remote host. Is borg working on the server? Traceback (most recent call last): File "borg/archiver.py", line 4529, in main File "borg/archiver.py", line 4461, in run File "borg/archiver.py", line 146, in wrapper File "borg/remote.py", line 576, in __init__ borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server? Platform: Linux ucs 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 Linux: debian 9.12 Borg: 1.1.11 Python: CPython 3.5.9 msgpack: 0.5.6 PID: 74791 CWD: /etc/borgmatic.d sys.argv: ['borg', 'prune', '--keep-weekly', '4', '--keep-monthly', '6', '--keep-daily', '7', '--prefix', '{hostname}-User_Maildata-', '--remote-path', '/usr/local/bin/borg', '--debug', '--show-rc', 'user@backupserver:/mnt/BorgBackup'] SSH_ORIGINAL_COMMAND: None terminating with error status, rc 2 user@backupserver:/mnt/BorgBackup: Error running actions for repository Command 'borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup' returned non-zero exit status 2 /etc/borgmatic.d/UCS_User_Maildata.yaml: No commands to run for on-error hook /etc/borgmatic.d/UCS_User_Maildata.yaml: Error running configuration file /etc/borgmatic.d/UCS_Server_Config.yaml: No commands to run for post-everything hook /etc/borgmatic.d/UCS_Server_Config_Wendlingen.yaml: No commands to run for post-everything hook /etc/borgmatic.d/UCS_User_Maildata.yaml: No commands to run for post-everything hook summary: /etc/borgmatic.d/UCS_Server_Config.yaml: Successfully ran configuration file /etc/borgmatic.d/UCS_Server_Config_Wendlingen.yaml: Successfully ran configuration file /etc/borgmatic.d/UCS_User_Maildata.yaml: Error running configuration file user@backupserver:/mnt/BorgBackup: Error running actions for repository using builtin fallback logging configuration 35 self tests completed in 0.10 seconds SSH command line: ['ssh', '-p', '22222', 'user@backupserver', '/usr/local/bin/borg', 'serve', '--umask=077', '--debug'] Remote: ssh: connect to host syno1.bs.private port 22222: Connection refused Connection closed by remote host. Is borg working on the server? Traceback (most recent call last): File "borg/archiver.py", line 4529, in main File "borg/archiver.py", line 4461, in run File "borg/archiver.py", line 146, in wrapper File "borg/remote.py", line 576, in __init__ borg.remote.ConnectionClosedWithHint: Connection closed by remote host. Is borg working on the server? Platform: Linux ucs 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1 (2020-01-20) x86_64 Linux: debian 9.12 Borg: 1.1.11 Python: CPython 3.5.9 msgpack: 0.5.6 PID: 74791 CWD: /etc/borgmatic.d sys.argv: ['borg', 'prune', '--keep-weekly', '4', '--keep-monthly', '6', '--keep-daily', '7', '--prefix', '{hostname}-User_Maildata-', '--remote-path', '/usr/local/bin/borg', '--debug', '--show-rc', 'user@backupserver:/mnt/BorgBackup'] SSH_ORIGINAL_COMMAND: None terminating with error status, rc 2 Command 'borg prune --keep-weekly 4 --keep-monthly 6 --keep-daily 7 --prefix {hostname}-User_Maildata- --remote-path /usr/local/bin/borg --debug --show-rc user@backupserver:/mnt/BorgBackup' returned non-zero exit status 2 ``` #### Expected behavior (if a bug) Use the standard ssh_command (ssh) if there is no special ssh_command configured. Even for multiple configuration files with different configurations. #### Other notes / implementation ideas If I specify the "default" ssh command also explicitly in the internal configuration, I didn't get the error any more. ``` # Command to use instead of just "ssh". This can be used to specify ssh options. ssh_command: ssh ``` #### Environment **borgmatic version:** [1.5.5] **borgmatic installation method:** [sudo pip3 install --user --upgrade borgmatic] **Borg version:** [borg 1.1.11] **Python version:** [Python 3.5.3] **operating system and version:** [Debian 9]
Owner

Thank you so much for the detailed ticket and bug description. I think I know exactly what's going on here: The ssh_command option is passed to Borg via an environment variable. And while that environment variable is set separately based on the value in each configuration file, it isn't ever removed if the ssh_command option isn't present in a configuration file. And so the environment variable from the previous configuration file is still in effect.

So the fix is to make sure environment variables get reset between borgmatic configuration files / Borg invocation.

Thank you so much for the detailed ticket and bug description. I think I know exactly what's going on here: The `ssh_command` option is passed to Borg via an environment variable. And while that environment variable is set separately based on the value in each configuration file, it isn't ever removed if the `ssh_command` option isn't present in a configuration file. And so the environment variable from the previous configuration file is still in effect. So the fix is to make sure environment variables get reset between borgmatic configuration files / Borg invocation.
witten added the
bug
label 2020-06-06 20:17:52 +00:00
Owner

Okay, this is fixed! It'll be released soon, and I'll note here when that happens.

Okay, this is fixed! It'll be released soon, and I'll note here when that happens.
Owner

Okay, this is released in borgmatic 1.5.6. Please let me know if you have any other issues!

Okay, this is released in borgmatic 1.5.6. Please let me know if you have any other issues!
Author

Hi, thanks for the very fast fix.

Hi, thanks for the very fast fix.
Sign in to join this conversation.
No Milestone
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#323
No description provided.