Configuration of logging in YAML config file #793

New Issue

atelszewski · 2023-11-20T10:15:43Z

atelszewski commented

2023-11-20 10:15:43 +00:00

What I'd like to do and why

Hi,

as of now, if one wants borgmatic's logs written to a file, command line arguments describing the desired logging settings need to be added to borgmatic call.

I would like to be able to just run borgmatic create and have the backup, together with log files, created.

What I mean is to add the ability to configure logging in the YAML config file.

Additionally, I would like to have the log files be created with a particular umask (0600).

Thank you for your support. :-)

--
Best regards,
Andrzej Telszewski

Other notes / implementation ideas

No response

### What I'd like to do and why Hi, as of now, if one wants `borgmatic`'s logs written to a file, command line arguments describing the desired logging settings need to be added to `borgmatic` call. I would like to be able to _just_ run `borgmatic create` and have the backup, together with log files, created. What I mean is to add the ability to configure _logging_ in the YAML config file. Additionally, I would like to have the log files be created with a particular umask (`0600`). Thank you for your support. :-) -- Best regards, Andrzej Telszewski ### Other notes / implementation ideas _No response_

witten commented

2023-11-20 16:51:02 +00:00

Thanks for filing this! The main reason borgmatic's log configuration can only be done on the command-line today is that logging begins before the configuration files are even loaded!

It's possible though that if logging options were added to borgmatic configuration, then any log messages could be buffered/delayed until configuration files are loaded and logging options are parsed. However that still wouldn't help with logging any errors that occur as part of loading configuration files. Maybe that's good enough?

There would also be the wrinkle of what to do if multiple configuration files are loaded and they have different logging configurations in each. Maybe the solution there is just to warn or error. Or maybe even different logging options per configuration file could actually be supported.

The umask features makes sense to me.

Thanks for filing this! The main reason borgmatic's log configuration can only be done on the command-line today is that logging begins before the configuration files are even loaded! It's possible though that if logging options were added to borgmatic configuration, then any log messages could be buffered/delayed until configuration files are loaded and logging options are parsed. However that still wouldn't help with logging any errors that occur as part of loading configuration files. Maybe that's good enough? There would also be the wrinkle of what to do if multiple configuration files are loaded and they have different logging configurations in each. Maybe the solution there is just to warn or error. Or maybe even different logging options per configuration file could actually be supported. The umask features makes sense to me.

atelszewski commented

2023-11-21 10:40:19 +00:00

Hi,

It's possible though that if logging options were added to borgmatic configuration, then any log messages could be buffered/delayed until configuration files are loaded and logging options are parsed.

That sounds correct.

However that still wouldn't help with logging any errors that occur as part of loading configuration files.

That's a similar situation that we have with daemonizing programs.
If log file cannot be created (i.e. config parsed), then there will normally be no sign of failure in daemon start-up.

The solution I see here is that, if borgmatic is unable to parse config files, it should print the message to stderr.
When run by hand, the user will see the message directly.
When run in an automated way, it will be the supervisor (e.g. cron) that should log the error message.

If everything with parsing the config files went correct, then the buffered messages would be logged to log files.
If it didn't, the messages would be printed to stderr.

There would also be the wrinkle of what to do if multiple configuration files are loaded and they have different logging configurations in each. Maybe the solution there is just to warn or error.

If there is different configuration for the same thing, I would just error out.

Or maybe even different logging options per configuration file could actually be supported.

Here we come to the same thing mentioned above.

If I'm not mistaken, borgmatic is repo-centric (or maybe rather label-centric?)
I mean, it will parse and merge config files, to ultimately find out what repose are there and will run actions for each of them.

Would it maybe make more sense to have per-repo logging configuration, and not per-config file?
And then, if across multiple config files the logging config for particular repo is different, then just error out?

Also, I would like to have a global config, if one wants to log everything (all repos) to a single log file.

--
Best regards,
Andrzej Telszewski

Hi, > It's possible though that if logging options were added to borgmatic configuration, then any log messages could be buffered/delayed until configuration files are loaded and logging options are parsed. That sounds correct. > However that still wouldn't help with logging any errors that occur as part of loading configuration files. That's a similar situation that we have with daemonizing programs. If log file cannot be created (i.e. config parsed), then there will normally be no _sign_ of failure in daemon start-up. The solution I see here is that, if `borgmatic` is unable to parse config files, it should print the message to _stderr_. When run by hand, the user will see the message directly. When run in an automated way, it will be the supervisor (e.g. `cron`) that should log the error message. If everything with parsing the config files went correct, then the _buffered_ messages would be logged to log files. If it didn't, the messages would be printed to _stderr_. > There would also be the wrinkle of what to do if multiple configuration files are loaded and they have different logging configurations in each. Maybe the solution there is just to warn or error. If there is different configuration for _the same thing_, I would just error out. > Or maybe even different logging options per configuration file could actually be supported. Here we come to _the same thing_ mentioned above. If I'm not mistaken, `borgmatic` is _repo_-centric (or maybe rather _label_-centric?) I mean, it will parse and merge config files, to ultimately find out what repose are there and will run actions for each of them. Would it maybe make more sense to have per-repo logging configuration, and not per-config file? And then, if across multiple config files the logging config for particular repo is different, then just error out? Also, I would like to have a global config, if one wants to log everything (all repos) to a single log file. -- Best regards, Andrzej Telszewski

witten commented

2023-12-04 19:28:06 +00:00

If I'm not mistaken, borgmatic is repo-centric (or maybe rather label-centric?)
I mean, it will parse and merge config files, to ultimately find out what repose are there and will run actions for each of them.

borgmatic is actually configuration-file centric. A top-level configuration file can include one or more repositories, and configuration files are not normally merged unless an explicit include is performed. So for instance a setup might look like this:

/etc/borgmatic.d/config1.yaml
- repository a
- repository b
- a bunch of options that apply to both repo a and b
/etc/borgmatic.d/config2.yaml
- repository c
- a bunch of options that apply (only) to repo c

Anyway, yes, we could just error out if there is conflicting logging config. And/or, as you suggest, have a global configuration.

> If I'm not mistaken, borgmatic is repo-centric (or maybe rather label-centric?) I mean, it will parse and merge config files, to ultimately find out what repose are there and will run actions for each of them. borgmatic is actually configuration-file centric. A top-level configuration file can include one or more repositories, and configuration files are not normally merged unless an explicit include is performed. So for instance a setup might look like this: * /etc/borgmatic.d/config1.yaml * repository a * repository b * a bunch of options that apply to both repo a and b * /etc/borgmatic.d/config2.yaml * repository c * a bunch of options that apply (only) to repo c Anyway, yes, we could just error out if there is conflicting logging config. And/or, as you suggest, have a global configuration.

witten added the

new feature area

label 2023-12-04 19:28:13 +00:00

atelszewski commented

2023-12-08 13:02:25 +00:00

Hi,

thank you for the explanation. :-)

On the topic of the log file name, I find it would be useful to be able to name the log file after the archive name.
The result I would like to achieve is to have a log of a particular create run.

--
Best regards,
Andrzej Telszewski

Hi, thank you for the explanation. :-) On the topic of the log file name, I find it would be useful to be able to name the log file after the archive name. The result I would like to achieve is to have a log of a particular `create` run. -- Best regards, Andrzej Telszewski

👍 1

witten referenced this issue

2024-03-09 16:53:09 +00:00

Set borgmatic command-line flag defaults in config file #303

witten commented

2024-03-09 16:53:21 +00:00

Related: #303.

witten referenced this issue

2024-03-09 16:57:06 +00:00

Set borgmatic command-line flag defaults in config file #303

Sign in to join this conversation.