Feature Request: Allow setting BORG_FILES_CACHE_TTL from config #618

New Issue

Closed

opened 2022-12-02 08:51:28 +00:00 by sur5r · 6 comments

sur5r commented 2022-12-02 08:51:28 +00:00

Copy Link

What I'm trying to do and why

I'm backing up a large file server over a rather slow line. The server contains two sections: active data and archived data. About once a year, stuff is moved from the active area to the archive area. Given the archive area is huge and only growing, it is only backed up once a month. As BORG_FILES_CACHE_TTL defaults to 20, the metadata from the archive area is gone when it's being backup` up again.

Currently, I'm injecting BORG_FILES_CACHE_TTL via systemd. For consistency reasons, I would like to be able to do so from the borgmatic config instead.

Other notes / implementation ideas

I think a key files_cache_ttl next to the existing files_cache would make sense.

Environment

borgmatic version: 1.5.20
borgmatic installation method: Debian package
Borg version: 1.2.2
Python version: 3.9.2
operating system and version: Debian bullseye

#### What I'm trying to do and why I'm backing up a large file server over a rather slow line. The server contains two sections: active data and archived data. About once a year, stuff is moved from the active area to the archive area. Given the archive area is huge and only growing, it is only backed up once a month. As `BORG_FILES_CACHE_TTL` defaults to 20, the metadata from the archive area is gone when it's being backup` up again. Currently, I'm injecting `BORG_FILES_CACHE_TTL` via systemd. For consistency reasons, I would like to be able to do so from the borgmatic config instead. #### Other notes / implementation ideas I think a key `files_cache_ttl` next to the existing `files_cache` would make sense. #### Environment **borgmatic version:** 1.5.20 **borgmatic installation method:** Debian package **Borg version:** 1.2.2 **Python version:** 3.9.2 **operating system and version:** Debian bullseye

sur5r commented 2022-12-02 09:54:57 +00:00

Author

Copy Link

I will try and implement this here: https://projects.torsion.org/sur5r/borgmatic/src/branch/filescachettl

Comments and feedback very welcome.

I will try and implement this here: https://projects.torsion.org/sur5r/borgmatic/src/branch/filescachettl Comments and feedback very welcome.

witten commented 2022-12-03 18:22:15 +00:00

Owner

Copy Link

Lookin' good! Thanks for taking the time to file this and start on the implementation. Here are a few points of feedback:

• I recommend grouping this option along with the other Borg options that get turned into environment variables, so the user can find them all in one place. Look for borg_keys_directory and similar settings.
• Also maybe call this borg_files_cache_ttl to emphasize that this corresponds to the environment variable of the same name.
• Would it be possible to put the environment setting logic in the make_environment() function along with the setting of other Borg environment variables? In fact you probably don't even need to change that function at all.. You can likely just edit the OPTION_TO_ENVIRONMENT_VARIABLE mapping in borgmatic/borg/environment.py.

When you're ready, please feel free to submit this as a pull request. Thanks!

Lookin' good! Thanks for taking the time to file this and start on the implementation. Here are a few points of feedback: * I recommend grouping this option along with the other Borg options that get turned into environment variables, so the user can find them all in one place. Look for `borg_keys_directory` and similar settings. * Also maybe call this `borg_files_cache_ttl` to emphasize that this corresponds to the environment variable of the same name. * Would it be possible to put the environment setting logic in the `make_environment()` function along with the setting of other Borg environment variables? In fact you probably don't even need to change that function at all.. You can likely just edit the `OPTION_TO_ENVIRONMENT_VARIABLE` mapping in `borgmatic/borg/environment.py`. When you're ready, please feel free to submit this as a pull request. Thanks!

witten added the

good first issue

label 2023-02-04 17:15:54 +00:00

Soumik_Dutta referenced this issue 2023-03-17 20:20:23 +00:00

add option to set borg_files_cache_ttl in config #654

witten referenced this issue from a commit 2023-03-18 02:24:42 +00:00

Add option to set borg_files_cache_ttl in config (#618).

witten commented 2023-03-18 02:26:06 +00:00

Owner

Copy Link

@Soumik_Dutta swooped in and implemented this! The feature will be part of the next release.

@Soumik_Dutta swooped in and implemented this! The feature will be part of the next release.

witten closed this issue 2023-03-18 02:26:06 +00:00

sur5r commented 2023-03-18 20:02:08 +00:00

Author

Copy Link

I completely forgot about this. Thanks everyone!

I completely forgot about this. Thanks everyone!

witten commented 2023-03-18 20:18:31 +00:00

Owner

Copy Link

Apparently we did too. 😄

Apparently we did too. 😄

witten commented 2023-03-27 23:22:05 +00:00

Owner

Copy Link

Released in borgmatic 1.7.10!

Released in borgmatic 1.7.10!

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No Label

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No Milestone

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#618

No description provided.