substitution problems with included files #745

New Issue

Closed

opened 2023-08-17 00:25:29 +00:00 by alsoeric · 4 comments

alsoeric commented 2023-08-17 00:25:29 +00:00

Copy Link

What I'm trying to do and why

I want constants defined in the backup-specific configuration file to substitute in the common configuration file. for example

in the common configuration file:
archive_name_format: '{backup_prefix}-{now:%Y-%m-%d_%H:%M}'

in the backup-specific config file

constants:
  backup_prefix: home
<<: !include /etc/borgmatic/config.yaml    

I would expect:

archive_name_format: 'home-{now:%Y-%m-%d_%H:%M}'

but get
archive_name_format: '{backup_prefix}-{now:%Y-%m-%d_%H:%M}'

Steps to reproduce

No response

Actual behavior

No response

Expected behavior

No response

Other notes / implementation ideas

No response

borgmatic version

No response

borgmatic installation method

No response

Borg version

1.8.1

Python version

Python 3.10.12

Database version (if applicable)

No response

Operating system and version

PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy

### What I'm trying to do and why I want constants defined in the backup-specific configuration file to substitute in the common configuration file. for example in the common configuration file: `archive_name_format: '{backup_prefix}-{now:%Y-%m-%d_%H:%M}'` in the backup-specific config file ``` constants: backup_prefix: home <<: !include /etc/borgmatic/config.yaml ``` I would expect: `archive_name_format: 'home-{now:%Y-%m-%d_%H:%M}'` but get `archive_name_format: '{backup_prefix}-{now:%Y-%m-%d_%H:%M}'` ### Steps to reproduce _No response_ ### Actual behavior _No response_ ### Expected behavior _No response_ ### Other notes / implementation ideas _No response_ ### borgmatic version _No response_ ### borgmatic installation method _No response_ ### Borg version 1.8.1 ### Python version Python 3.10.12 ### Database version (if applicable) _No response_ ### Operating system and version PRETTY_NAME="Ubuntu 22.04.3 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.3 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy

witten commented 2023-08-17 15:57:21 +00:00

Owner

Copy Link

Thanks for filing this. Unfortunately this is a documented limitation of constants currently:

Constants don't work across includes or separate configuration files though.

I do think your use case is a valuable one though, and it's probably worth revising the feature to work across includes.

Thanks for filing this. Unfortunately this is a [documented](https://torsion.org/borgmatic/docs/how-to/make-per-application-backups/#constant-interpolation) limitation of constants currently: > Constants don't work across includes or separate configuration files though. I do think your use case is a valuable one though, and it's probably worth revising the feature to work across includes.

RKI027 commented 2023-10-13 13:18:32 +00:00

Copy Link

I have something that works with the following setup:

config.yaml

constants:
    !include common/constants.yaml

<<: !include common/common.yaml

#...more config using constants

common/constants.yaml

host: example.com
repo: myrepo

common/common.yaml

exclude_from:
    - common/excludes

ssh_command: ssh -p2023 -i ....

#...more config NOT using constants

Is it a happy accident that it works or should I be extra careful when upgrading to a future version? (currently using 1.8.2)

I have something that works with the following setup: `config.yaml` ```yaml constants: !include common/constants.yaml <<: !include common/common.yaml #...more config using constants ``` `common/constants.yaml` ```yaml host: example.com repo: myrepo ``` `common/common.yaml` ```yaml exclude_from: - common/excludes ssh_command: ssh -p2023 -i .... #...more config NOT using constants ``` Is it a happy accident that it works or should I be extra careful when upgrading to a future version? (currently using 1.8.2)

witten commented 2023-10-13 16:11:31 +00:00

Owner

Copy Link

If I'm understanding correctly, common.yaml doesn't use any of your constants while config.yaml does. So it makes sense that this works currently. The only tricky part is that you're including the constants themselves, which is currently supported. As long as you don't introduce any use of constants within common.yaml or similar includes, then you should be fine!

If I'm understanding correctly, `common.yaml` doesn't use any of your constants while `config.yaml` does. So it makes sense that this works currently. The only tricky part is that you're including the constants themselves, which is currently supported. As long as you don't introduce any use of constants within `common.yaml` or similar includes, then you should be fine!

👍 1

witten referenced this issue 2023-11-14 21:59:46 +00:00

Using constants in a config for placeholders in an included file. #788

witten commented 2023-11-20 05:17:01 +00:00

Owner

Copy Link

This has been implemented as part of borgmatic 1.8.5, just released. Thanks again for the suggestion!

This has been implemented as part of borgmatic 1.8.5, just released. Thanks again for the suggestion!

witten closed this issue 2023-11-20 05:17:01 +00:00

witten referenced this issue from a commit 2023-11-20 05:22:40 +00:00

Constants support includes and command-line overrides (#745, #782)

witten referenced this issue from a commit 2023-11-20 05:22:40 +00:00

Update documentation about configuration includes and constants (#745).

witten referenced this issue 2023-12-24 19:21:14 +00:00

Overriding hostname constant doesn't affect default archive_name_format #763

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

spot-check

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

3 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#745

No description provided.