Borgmatic 2.0 executes commands multiple times #1060

New issue

Closed

opened 2025-04-08 04:36:05 +00:00 by vitoyucepi · 4 comments

vitoyucepi commented 2025-04-08 04:36:05 +00:00

Copy link

What I'm trying to do and why

While migrating the borgmatic configs from 1.9.14 to 2.0.1, I ran into the problem with the new hooks.

Steps to reproduce

I am using the python:3.13-slim container image to reproduce the problem.

1. apt-get update;
   apt-get install --no-install-recommends -y pkg-config libssl-dev liblz4-dev libzstd-dev libxxhash-dev libacl1-dev gcc;
   pip install borgmatic==2.0.1 borgbackup;
   

2. 1.conf

   source_directories:
     - /root
   
   repositories:
     - path: /tmp/test1
   
   skip_actions:
     - prune
   
   before_backup:
     - "echo 'Starting a backup.'"
   after_backup:
     - "echo 'Finished a backup.'"
   
   before_prune:
     - "echo 'Starting pruning.'"
   after_prune:
     - "echo 'Finished pruning.'"
   
   before_compact:
     - "echo 'Starting compaction.'"
   after_compact:
     - "echo 'Finished compaction.'"
   
   before_check:
     - "echo 'Starting checks.'"
   after_check:
     - "echo 'Finished checks.'"
   
   before_extract:
     - "echo 'Starting extracting.'"
   after_extract:
     - "echo 'Finished extracting.'"
   

3. 2.conf

   source_directories:
     - /root
   
   repositories:
     - path: /tmp/test2
   
   skip_actions:
     - prune
   
   commands:
     - before: action
       when: [create]
       run: ["echo 'Starting create.'"]
     - after: action
       when: [create]
       run: ["echo 'Finished create.'"]
     - before: action
       when: [prune]
       run: ["echo 'Starting prune.'"]
     - after: action
       when: [prune]
       run: ["echo 'Finished prune.'"]
     - before: action
       when: [compact]
       run: ["echo 'Starting compact.'"]
     - after: action
       when: [compact]
       run: ["echo 'Finished compact.'"]
     - before: action
       when: [check]
       run: ["echo 'Starting check.'"]
     - after: action
       when: [check]
       run: ["echo 'Finished check.'"]
   

4. export BORG_PASSPHRASE=1;
   borgmatic -c 1.conf init -e repokey;
   borgmatic -c 2.conf init -e repokey;
   borgmatic -c 1.conf;
   borgmatic -c 2.conf;
   

Actual behavior

When I use borgmatic==1.9.14, the output is

Starting a backup.
Finished a backup.
Starting compaction.
Finished compaction.
Starting checks.
Finished checks.

When I use borgmatic==2.0.1, the output is

Starting create.
Starting prune.
Starting compact.
Starting check.
Finished create.
Finished prune.
Finished compact.
Finished check.
Starting create.
Starting prune.
Starting compact.
Starting check.
Finished create.
Finished prune.
Finished compact.
Finished check.
Starting create.
Starting prune.
Starting compact.
Starting check.
Finished create.
Finished prune.
Finished compact.
Finished check.
Starting create.
Starting prune.
Starting compact.
Starting check.
Finished create.
Finished prune.
Finished compact.
Finished check.

Expected behavior

Borgmatic should not run hooks multiple times.

Other notes / implementation ideas

No response

borgmatic version

2.0.1

borgmatic installation method

pip

Borg version

1.4.0

Python version

Python 3.13.2

Database version (if applicable)

N/A

Operating system and version

Debian 12

### What I'm trying to do and why While migrating the borgmatic configs from 1.9.14 to 2.0.1, I ran into the problem with the new hooks. ### Steps to reproduce I am using the `python:3.13-slim` container image to reproduce the problem. 1. ```sh apt-get update; apt-get install --no-install-recommends -y pkg-config libssl-dev liblz4-dev libzstd-dev libxxhash-dev libacl1-dev gcc; pip install borgmatic==2.0.1 borgbackup; ``` 2. `1.conf` ```conf source_directories: - /root repositories: - path: /tmp/test1 skip_actions: - prune before_backup: - "echo 'Starting a backup.'" after_backup: - "echo 'Finished a backup.'" before_prune: - "echo 'Starting pruning.'" after_prune: - "echo 'Finished pruning.'" before_compact: - "echo 'Starting compaction.'" after_compact: - "echo 'Finished compaction.'" before_check: - "echo 'Starting checks.'" after_check: - "echo 'Finished checks.'" before_extract: - "echo 'Starting extracting.'" after_extract: - "echo 'Finished extracting.'" ``` 1. `2.conf` ```conf source_directories: - /root repositories: - path: /tmp/test2 skip_actions: - prune commands: - before: action when: [create] run: ["echo 'Starting create.'"] - after: action when: [create] run: ["echo 'Finished create.'"] - before: action when: [prune] run: ["echo 'Starting prune.'"] - after: action when: [prune] run: ["echo 'Finished prune.'"] - before: action when: [compact] run: ["echo 'Starting compact.'"] - after: action when: [compact] run: ["echo 'Finished compact.'"] - before: action when: [check] run: ["echo 'Starting check.'"] - after: action when: [check] run: ["echo 'Finished check.'"] ``` 1. ```sh export BORG_PASSPHRASE=1; borgmatic -c 1.conf init -e repokey; borgmatic -c 2.conf init -e repokey; borgmatic -c 1.conf; borgmatic -c 2.conf; ``` ### Actual behavior When I use `borgmatic==1.9.14`, the output is ``` Starting a backup. Finished a backup. Starting compaction. Finished compaction. Starting checks. Finished checks. ``` When I use `borgmatic==2.0.1`, the output is ``` Starting create. Starting prune. Starting compact. Starting check. Finished create. Finished prune. Finished compact. Finished check. Starting create. Starting prune. Starting compact. Starting check. Finished create. Finished prune. Finished compact. Finished check. Starting create. Starting prune. Starting compact. Starting check. Finished create. Finished prune. Finished compact. Finished check. Starting create. Starting prune. Starting compact. Starting check. Finished create. Finished prune. Finished compact. Finished check. ``` ### Expected behavior Borgmatic should not run hooks multiple times. ### Other notes / implementation ideas _No response_ ### borgmatic version 2.0.1 ### borgmatic installation method pip ### Borg version 1.4.0 ### Python version Python 3.13.2 ### Database version (if applicable) N/A ### Operating system and version Debian 12

👍 1

witten commented 2025-04-08 05:09:16 +00:00

Owner

Copy link

Thanks for the detailed ticket and repro steps! I've got a repro here, so I'll let you know what I find.

Thanks for the detailed ticket and repro steps! I've got a repro here, so I'll let you know what I find.

witten added the

bug

label 2025-04-08 05:09:21 +00:00

witten commented 2025-04-08 05:32:44 +00:00

Owner

Copy link

Wow, this is embarrassing. The whole list of action command hooks get run once for each action that borgmatic's executed with (implicitly or explicitly). That's why you're seeing all the duplication. I don't think I ever noticed this during development because I was mostly manual testing with single actions on the command-line. And the automated tests don't cover this particular use case (yet).

Wow, this is embarrassing. The whole list of action command hooks get run once for each action that borgmatic's executed with (implicitly or explicitly). That's why you're seeing all the duplication. I don't think I ever noticed this during development because I was mostly manual testing with single actions on the command-line. And the automated tests don't cover this particular use case (yet).

witten referenced this issue from a commit 2025-04-08 06:01:00 +00:00

Fix command hooks getting run too many times when multiple borgmatic actions are executed (#1060).

witten commented 2025-04-08 06:02:34 +00:00

Owner

Copy link

This is fixed in main and will be part of the next release.

Side note: This also uncovered that action command hooks execute before/after actions listed in skip_actions. That should probably be remedied as well.

Thanks again!

This is fixed in main and will be part of the next release. Side note: This also uncovered that `action` command hooks execute before/after actions listed in `skip_actions`. That should probably be remedied as well. Thanks again!

🎉 1

witten closed this issue 2025-04-08 06:02:45 +00:00

witten referenced this issue from a commit 2025-04-08 06:11:41 +00:00

Don't run action command hooks for actions listed in the "skip_actions" option (#1060).

witten commented 2025-04-08 19:14:14 +00:00

Owner

Copy link

Released in borgmatic 2.0.2!

Released in borgmatic 2.0.2!

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

parallel-borgmatic

command-hook-steps

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

blocked

  

breaking

  

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

borgmatic-collective/borgmatic#1060

No description provided.