Before everything hook running twice with include directive #1124

New issue

Closed

opened 2025-07-31 15:33:48 +00:00 by dot-mike · 4 comments

dot-mike commented 2025-07-31 15:33:48 +00:00

Copy link

What I'm trying to do and why

I'm trying to have separate repository config for an application-backup. To take backup I need to generate the backup first, so I implemented a before-hook for everything.

The issue is that the before-hook for everything runs twice, for each config-file as defined in .config/borgmatic.d/

Steps to reproduce

Use config files below, replace the variables as necessary

Common config file not used by borgmatic directly, only included
$ cat .config/borgmatic/gitea-common.yaml

verbosity: 2

commands:
  - before: everything
    when: [create]
    run:
      - "bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'"
  - after: everything
    states: [finish]
    run:
      - "bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'"
keep_within: '7d'
keep_daily:   7
keep_weekly:  4
keep_monthly: 6

borgmatic.d config files:

cat .config/borgmatic.d/gitea-local.yaml

<<: !include /var/services/homes/backup/.config/borgmatic/gitea-common.yaml

remote_path: /usr/local/bin/borg

repositories:
  - path: ssh://backupseerver/volume1/borgbackup/gitea-app
    label: localbackup

cat .config/borgmatic.d/gitea-remote.yaml

<<: !include /var/services/homes/backup/.config/borgmatic/gitea-common.yaml

encryption_passphrase: <secret>

repositories:
  - path: ssh://remoteserver/./borg-app-gitea
    label: remoteserver

Actual behavior

See that prepare_ is created twice, one for each config file.

backup@server:~$ borgmatic
Running before create everything command hook
PGDATA=*** bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'
Running before create everything command hook
PGDATA=*** bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'

Expected behavior

Only one prepare file should be created as we use everything for the hook. As stated in the docs:

everything runs before or after all configuration files. Errors here do not trigger error hooks or the fail state in monitoring hooks. This replaces the deprecated before_everything and after_everything.

Other notes / implementation ideas

No response

borgmatic version

2.0.3

borgmatic installation method

pip

Borg version

borg 1.4.1

Python version

Python 3.12.11

Database version (if applicable)

No response

Operating system and version

N/A

### What I'm trying to do and why I'm trying to have separate repository config for an application-backup. To take backup I need to generate the backup first, so I implemented a before-hook for everything. The issue is that the before-hook for everything runs twice, for each config-file as defined in `.config/borgmatic.d/` ### Steps to reproduce Use config files below, replace the variables as necessary Common config file not used by borgmatic directly, only included $ cat .config/borgmatic/gitea-common.yaml ```bash verbosity: 2 commands: - before: everything when: [create] run: - "bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'" - after: everything states: [finish] run: - "bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)'" keep_within: '7d' keep_daily: 7 keep_weekly: 4 keep_monthly: 6 ``` borgmatic.d config files: cat .config/borgmatic.d/gitea-local.yaml ``` <<: !include /var/services/homes/backup/.config/borgmatic/gitea-common.yaml remote_path: /usr/local/bin/borg repositories: - path: ssh://backupseerver/volume1/borgbackup/gitea-app label: localbackup ``` cat .config/borgmatic.d/gitea-remote.yaml ``` <<: !include /var/services/homes/backup/.config/borgmatic/gitea-common.yaml encryption_passphrase: <secret> repositories: - path: ssh://remoteserver/./borg-app-gitea label: remoteserver ``` ### Actual behavior See that `prepare_` is created twice, one for each config file. backup@server:~$ borgmatic Running before create everything command hook PGDATA=*** bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)' Running before create everything command hook PGDATA=*** bash -c 'touch /tmp/borg-repro/prepare_$(date +%Y%m%d%H%M%S%N)' ### Expected behavior Only one prepare file should be created as we use `everything` for the hook. As stated in the docs: > `everything` runs before or after all configuration files. Errors here do not trigger error hooks or the fail state in monitoring hooks. This replaces the deprecated before_everything and after_everything. ### Other notes / implementation ideas _No response_ ### borgmatic version 2.0.3 ### borgmatic installation method pip ### Borg version borg 1.4.1 ### Python version Python 3.12.11 ### Database version (if applicable) _No response_ ### Operating system and version N/A

dot-mike changed title from Before everything hook running twice to Before everything hook running twice with include directive 2025-07-31 16:02:47 +00:00

dot-mike commented 2025-07-31 19:32:10 +00:00

Author

Copy link

Looks like I've hit this issue: #1080 (comment) and should be solved in 2.0.4

Looks like I've hit this issue: https://projects.torsion.org/borgmatic-collective/borgmatic/issues/1080#issuecomment-11030 and should be solved in 2.0.4

dot-mike closed this issue 2025-07-31 19:32:10 +00:00

witten commented 2025-08-01 23:50:10 +00:00

Owner

Copy link

Yup, good sleuthing! If upgrading to 2.0.4+ doesn't do it for you, please let me know.

Yup, good sleuthing! If upgrading to 2.0.4+ doesn't do it for you, please let me know.

witten added the

question / support

label 2025-08-01 23:50:20 +00:00

dot-mike commented 2025-08-02 00:32:34 +00:00

Author

Copy link

2.0.4+ solved it, thanks!

2.0.4+ solved it, thanks!

👍 1

witten commented 2025-08-02 02:44:18 +00:00

Owner

Copy link

Glad to hear it!

Glad to hear it!

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

parallel-borgmatic

command-hook-steps

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

blocked

  

breaking

  

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

borgmatic-collective/borgmatic#1124

No description provided.