"borgmatic check" regression in 1.7.3 #597

New Issue

Closed

opened 2022-10-14 12:55:29 +00:00 by kimheino · 4 comments

kimheino commented 2022-10-14 12:55:29 +00:00

Contributor

Copy Link

Commit ae036aebd7 changes execute.py:

-            command, shell=shell, env=environment, cwd=working_directory
+            command, stderr=subprocess.STDOUT, shell=shell, env=environment, cwd=working_directory

This change will break borgmatic check if there is any output to stderr from ssh:

#> borgmatic check -v 2

Ensuring legacy configuration is upgraded
borg --version --debug --show-rc
/etc/borgmatic.d/storagebox.yaml: No commands to run for pre-actions hook
/etc/borgmatic.d/storagebox.yaml: No commands to run for pre-check hook
ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg: Running consistency checks
borg info --json ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg
ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg: Error running actions for repository
Cannot determine Borg repository ID for ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg
/etc/borgmatic.d/storagebox.yaml: Running command for on-error hook

Root cause is RHEL9's default crypto policy which prohibits SHA1. Hetzner's Storagebox has RSA key with SHA1:

#> ssh uXXXXXX-sub14@u311936.your-storagebox.de -p 23
client_global_hostkeys_private_confirm: server gave bad signature for RSA key 1: error in libcrypto
+------------------------------------------------------------------+
| Welcome to your Storage Box.                                     |
...
uXXXXXX-sub14 /home >

That one extra line ("client_global_hostkeys_private_confirm") is outputted to stderr by local ssh client, not remote server. It will obviously break JSON parsing in borgmatic check.

There are workarounds to get rid of that stderr warning, for example:

1. Manually add remote's RSA key to known_hosts (<- this is what I did)
2. Allow SHA1: update-crypto-policies --set DEFAULT:SHA1

I still would recommend to revert that change, or partially revert it if JSON parsing is used.

Borgmatic 1.7.3 is currently in EPEL9-testing and this problem will be hit by multiple users in 6 days:

https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-64cd98ca6b

Commit ae036aebd76 changes execute.py: ``` - command, shell=shell, env=environment, cwd=working_directory + command, stderr=subprocess.STDOUT, shell=shell, env=environment, cwd=working_directory ``` This change will break `borgmatic check` if there is any output to stderr from ssh: ``` #> borgmatic check -v 2 Ensuring legacy configuration is upgraded borg --version --debug --show-rc /etc/borgmatic.d/storagebox.yaml: No commands to run for pre-actions hook /etc/borgmatic.d/storagebox.yaml: No commands to run for pre-check hook ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg: Running consistency checks borg info --json ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg: Error running actions for repository Cannot determine Borg repository ID for ssh://uXXXXXX-sub14@uXXXXXX.your-storagebox.de:23/./borg /etc/borgmatic.d/storagebox.yaml: Running command for on-error hook ``` Root cause is RHEL9's default crypto policy which prohibits SHA1. Hetzner's Storagebox has RSA key with SHA1: ``` #> ssh uXXXXXX-sub14@u311936.your-storagebox.de -p 23 client_global_hostkeys_private_confirm: server gave bad signature for RSA key 1: error in libcrypto +------------------------------------------------------------------+ | Welcome to your Storage Box. | ... uXXXXXX-sub14 /home > ``` That one extra line ("client_global_hostkeys_private_confirm") is outputted to stderr by local ssh client, not remote server. It will obviously break JSON parsing in `borgmatic check`. There are workarounds to get rid of that stderr warning, for example: 1) Manually add remote's RSA key to known_hosts (<- this is what I did) 2) Allow SHA1: `update-crypto-policies --set DEFAULT:SHA1` I still would recommend to revert that change, or partially revert it if JSON parsing is used. Borgmatic 1.7.3 is currently in EPEL9-testing and this problem will be hit by multiple users in 6 days: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-64cd98ca6b

witten commented 2022-10-14 16:47:03 +00:00

Owner

Copy Link

Wow, thank you for reporting this and diving into the cause!

Wow, thank you for reporting this and diving into the cause!

witten added the

bug

label 2022-10-14 16:47:10 +00:00

witten commented 2022-10-14 17:09:24 +00:00

Owner

Copy Link

It looks like this isn't quite as straightforward as backing out the change you've helpfully pinpointed, as doing so would break the new special file detection feature. So I may need to introduce some branching logic to do one thing for check and a different thing when detecting special files.

It looks like this isn't *quite* as straightforward as backing out the change you've helpfully pinpointed, as doing so would break the new special file detection feature. So I may need to introduce some branching logic to do one thing for `check` and a different thing when detecting special files.

witten referenced this issue from a commit 2022-10-14 23:22:32 +00:00

Fix regression in which "check" action errored on certain systems (#597, #598).

witten commented 2022-10-14 23:29:48 +00:00

Owner

Copy Link

Fix released in borgmatic 1.7.4. Please give it a shot and let me know if it works for you. Thanks!

Fix released in borgmatic 1.7.4. Please give it a shot and let me know if it works for you. Thanks!

witten closed this issue 2022-10-14 23:29:48 +00:00

witten referenced this issue 2022-10-14 23:30:02 +00:00

Cannot determine Borg repository ID #598

kimheino commented 2022-10-15 17:57:24 +00:00

Author

Contributor

Copy Link

Works for me. Thanks for quick fix.

Works for me. Thanks for quick fix.

👍 1

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

config-command-line

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels

blocked

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

No Label

bug

Milestone

No items

No Milestone

Assignees

Clear assignees

diivi import_bot witten

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#597

No description provided.