Suggestion: Limiting archive listings according to what's specified in the config file #334

New issue

Closed

opened 2020-06-21 19:19:54 +00:00 by s1shed · 3 comments

s1shed commented 2020-06-21 19:19:54 +00:00

Contributor

Copy link

What I'm trying to do and why

I'm listing archives while using multiple config files which all have the same respository defined.

On one system I have several config files in /etc/borgmatic.d/, one for each application. If I type borgmatic list, it will list all archives, one for each config file. If I have 5 config files that all point to the same repository, I'll get the same output printed five times. Of course, I realize that I can print this all once with borgmatic --config /etc/borgmatic.d/app1.yaml.

Would it make sense that when borgmatic list is run that the output is limited to what is specified in the key storage.archive_name_format in the configfile that is currently in use (perhaps by specifying a parameter)? Maybe a new key in the config file? Or, if location.repositories is identical in all config files, de-duplicate the list and print the listing just once?

I can get the output I want by specifying a config file and `-P '{hostname}-appname' but perhaps if it wouldn't be a PITA others would find what I'm suggesting to be useful.

(I'd not be opposed to marking this "won't fix" and closing)

#### What I'm trying to do and why I'm listing archives while using multiple config files which all have the same respository defined. On one system I have several config files in `/etc/borgmatic.d/`, one for each application. If I type `borgmatic list`, it will list all archives, one for each config file. If I have 5 config files that all point to the same repository, I'll get the same output printed five times. Of course, I realize that I can print this all once with `borgmatic --config /etc/borgmatic.d/app1.yaml`. Would it make sense that when `borgmatic list` is run that the output is limited to what is specified in the key `storage.archive_name_format` in the configfile that is currently in use (perhaps by specifying a parameter)? Maybe a new key in the config file? Or, if `location.repositories` is identical in all config files, de-duplicate the list and print the listing just once? I can get the output I want by specifying a config file and `-P '{hostname}-appname' but perhaps if it wouldn't be a PITA others would find what I'm suggesting to be useful. (I'd not be opposed to marking this "won't fix" and closing)

👍 1

witten commented 2020-06-24 20:46:16 +00:00

Owner

Copy link

Interesting idea! Thanks for submitting it. Given that storage.archive_name_format isn't necessarily a prefix (and can contain things like {now} as part of the pattern), it would probably be necessary for the user to specify a separate prefix specific to listing. Maybe an output.prefix option. Then, that value could be passed to borg list --prefix as you suggest.

Or, if location.repositories is identical in all config files, de-duplicate the list and print the listing just once?

The main impediment I can see to this approach is that multiple configuration files may have differences in options that impact borgmatic list. Examples: local_path, remote_path, and lock_wait. So to be truly accurate, the de-duplication would have to take into account whether all of repository, local_path, remote_path, and lock_wait are identical across configuration files. Annoying, but doable. And may actually be preferrable to the user having to specific a prefix as above.

Let me know your thoughts.

Interesting idea! Thanks for submitting it. Given that `storage.archive_name_format` isn't necessarily a prefix (and can contain things like `{now}` as part of the pattern), it would probably be necessary for the user to specify a separate prefix specific to `list`ing. Maybe an `output.prefix` option. Then, that value could be passed to `borg list --prefix` as you suggest. > Or, if location.repositories is identical in all config files, de-duplicate the list and print the listing just once? The main impediment I can see to this approach is that multiple configuration files may have differences in options that impact `borgmatic list`. Examples: `local_path`, `remote_path`, and `lock_wait`. So to be truly accurate, the de-duplication would have to take into account whether all of repository, `local_path`, `remote_path`, and `lock_wait` are identical across configuration files. Annoying, but doable. And may actually be preferrable to the user having to specific a prefix as above. Let me know your thoughts.

s1shed commented 2020-06-27 09:59:11 +00:00

Author

Contributor

Copy link

Interesting idea! Thanks for submitting it. Given that storage.archive_name_format isn't necessarily a prefix (and can contain things like {now} as part of the pattern), it would probably be necessary for the user to specify a separate prefix specific to listing. Maybe an output.prefix option. Then, that value could be passed to borg list --prefix as you suggest.

The main benefit of having a configurable option would be that each config file could have its own prefix defined and one run of borgmatic list would cover it all. This is a possibility that I also considered, like the constraint that can be defined for pruning archives.

Or, if location.repositories is identical in all config files, de-duplicate the list and print the listing just once?

The main impediment I can see to this approach is that multiple configuration files may have differences in options that impact borgmatic list. Examples: local_path, remote_path, and lock_wait. So to be truly accurate, the de-duplication would have to take into account whether all of repository, local_path, remote_path, and lock_wait are identical across configuration files. Annoying, but doable. And may actually be preferrable to the user having to specific a prefix as above.

Let me know your thoughts.

This would definitely take care of the sort of situation that I described in this issue but I don't know if the annoying effort would be worth it.

> Interesting idea! Thanks for submitting it. Given that `storage.archive_name_format` isn't necessarily a prefix (and can contain things like `{now}` as part of the pattern), it would probably be necessary for the user to specify a separate prefix specific to `list`ing. Maybe an `output.prefix` option. Then, that value could be passed to `borg list --prefix` as you suggest. The main benefit of having a configurable option would be that each config file could have its own prefix defined and one run of `borgmatic list` would cover it all. This is a possibility that I also considered, like the constraint that can be defined for pruning archives. > > Or, if location.repositories is identical in all config files, de-duplicate the list and print the listing just once? > > The main impediment I can see to this approach is that multiple configuration files may have differences in options that impact `borgmatic list`. Examples: `local_path`, `remote_path`, and `lock_wait`. So to be truly accurate, the de-duplication would have to take into account whether all of repository, `local_path`, `remote_path`, and `lock_wait` are identical across configuration files. Annoying, but doable. And may actually be preferrable to the user having to specific a prefix as above. > > Let me know your thoughts. This would definitely take care of the sort of situation that I described in this issue but I don't know if the annoying effort would be worth it.

👍 1

witten commented 2023-03-29 18:54:06 +00:00

Owner

Copy link

I think this is still an important issue for users with the same repository defined in multiple borgmatic config files, but I'm closing this ticket as a dup of #479 as that one has more recent discussion. Please feel free to follow along there.

I think this is still an important issue for users with the same repository defined in multiple borgmatic config files, but I'm closing this ticket as a dup of #479 as that one has more recent discussion. Please feel free to follow along there.

witten closed this issue 2023-03-29 18:54:06 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

parallel-borgmatic

command-hook-steps

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

blocked

  

breaking

  

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

borgmatic-collective/borgmatic#334

No description provided.