Log various facts about system and dependencies at startup under high verbosity #714

New Issue

2rs2ts · 2023-06-18T19:47:39Z

2rs2ts commented

2023-06-18 19:47:39 +00:00

What I'm trying to do and why

In order to make it easier to file issues, it would be useful if borgmatic's logs under high verbosity (--log-verbosity 2) would include facts about the system, including:

Version of borgmatic, borg, and Python itself

Perhaps any other dependencies could also be logged, however I'm not sure how useful that would be for the various Python libraries, nor whether that would have to be derived from a list that's maintained manually (which would make it a huge pain in the butt.)
Install path for the above (to help identify install method)
OS name and version
Database version (if applicable,) however I'm not sure how to find this information out since I don't use any databases in my backups.
User that is running borgmatic (for detecting problems arising from running borgmatic as a user that doesn't have access to, e.g., the backup targets.)
Ownership info of borgmatic config folder (for similar problems as the above)
Perhaps other things, please comment below if you have good ideas!

I think this information should be printed near the startup sequence, though depending on what we output, it would probably have to be split into two parts: printing what you can before reading the config file, and then printing what you can only know after having read the config file. If we print only after reading the config file, then we'll miss out on any helpful information for debugging issues that result in either being unable to read the file, or in misinterpreting it.

This idea came up in the discussion in #713.

#### What I'm trying to do and why In order to make it easier to file issues, it would be useful if borgmatic's logs under high verbosity (`--log-verbosity 2`) would include facts about the system, including: * Version of borgmatic, borg, and Python itself Perhaps any other dependencies could also be logged, however I'm not sure how useful that would be for the various Python libraries, nor whether that would have to be derived from a list that's maintained manually (which would make it a huge pain in the butt.) * Install path for the above (to help identify install method) * OS name and version * Database version (if applicable,) however I'm not sure how to find this information out since I don't use any databases in my backups. * User that is running borgmatic (for detecting problems arising from running borgmatic as a user that doesn't have access to, e.g., the backup targets.) * Ownership info of borgmatic config folder (for similar problems as the above) * Perhaps other things, please comment below if you have good ideas! I think this information should be printed near the startup sequence, though depending on what we output, it would probably have to be split into two parts: printing what you can before reading the config file, and then printing what you can only know after having read the config file. If we print only after reading the config file, then we'll miss out on any helpful information for debugging issues that result in either being unable to read the file, or in misinterpreting it. This idea came up in the discussion in #713.

witten commented

2023-06-20 05:48:32 +00:00

Thank you for filing this! These are all great ideas. I think even if borgmatic only included a subset of these items to start with, it would be a huge improvement over the status quo.

witten added the

good first issue

label 2023-06-20 05:48:40 +00:00

witten added the

new feature area

label 2023-06-28 18:35:29 +00:00

witten removed the

new feature area

label 2023-06-28 18:36:49 +00:00

IBims1NicerTobi commented

2023-08-28 19:46:18 +00:00

We could start by getting the distro and the version from /etc/os-release for linux as almost all distros have that file afaik. Borg otherwise supports many more unix like operating systems. I will have a look at neofetch as they already have a lot of them convered in a single bash script that might be really helpful.

The borgmatic, borg and python version should be easy to get. Python and borgmatic can be determined internally, borg version should be fetched from running the borg executable and getting the stdout from that.

The config file paths are already logged by the backup itself but it should not be too hard to get the config file paths at startup. It might be a bit challenging to get all the include paths from all of the config files.

Database versions might be doable by writing an integration for every database hook, but that might require a new function in every file that can be called at startup to get the version before the actual hook is called.

I would like to work on this as it seems really helpful

We could start by getting the distro and the version from /etc/os-release for linux as almost all distros have that file afaik. Borg otherwise supports many more unix like operating systems. I will have a look at neofetch as they already have a lot of them convered in a single bash script that might be really helpful. The borgmatic, borg and python version should be easy to get. Python and borgmatic can be determined internally, borg version should be fetched from running the borg executable and getting the stdout from that. The config file paths are already logged by the backup itself but it should not be too hard to get the config file paths at startup. It might be a bit challenging to get all the include paths from all of the config files. Database versions might be doable by writing an integration for every database hook, but that might require a new function in every file that can be called at startup to get the version before the actual hook is called. I would like to work on this as it seems really helpful

witten commented

2023-08-28 21:23:31 +00:00

This all sounds like a great place to start, and if you want to work on this that'd be welcome! Some additional details that may be helpful:

local_borg_version is already available as a variable within borgmatic/commands/borgmatic.py:run_configuration() and gets passed around to various other functions.
You can get borgmatic's version from importlib_metadata.version('borgmatic'), although take care in how importlib_metadata is imported. See borgmatic/commands/borgmatic.py for an example.

And your idea about extending the database hooks sounds reasonable to me. In general though, I'd suggest erring on the side of a basic implementation as a first pass at this feature, rather than exhaustive and complex. We can always iterate later on.

This all sounds like a great place to start, and if you want to work on this that'd be welcome! Some additional details that may be helpful: - `local_borg_version` is already available as a variable within `borgmatic/commands/borgmatic.py:run_configuration()` and gets passed around to various other functions. - You can get borgmatic's version from `importlib_metadata.version('borgmatic')`, although take care in how `importlib_metadata` is imported. See `borgmatic/commands/borgmatic.py` for an example. And your idea about extending the database hooks sounds reasonable to me. In general though, I'd suggest erring on the side of a basic implementation as a first pass at this feature, rather than exhaustive and complex. We can always iterate later on.

IBims1NicerTobi commented

2023-08-29 09:51:55 +00:00

I have looked at some of the options for getting os and env info and the best choice seems to be using pythons platform for that. It basically ticks all the boxes so far:

Maintained without changing any time soon
Cross Platform support (Mac OS in our case)
Support for /etc/os-release and /usr/lib/os-release

I have looked at some of the options for getting os and env info and the best choice seems to be using pythons platform for that. It basically ticks all the boxes so far: - Maintained without changing any time soon - Cross Platform support (Mac OS in our case) - Support for /etc/os-release and /usr/lib/os-release

IBims1NicerTobi commented

2023-08-29 10:01:46 +00:00

One question that remains is: Where will we implement this? Does borgmatic log this every time someone runs with loglevel 2? I personally think for the os and version info it would be good to implement a new flag just like --version but name something like --version-env that outputs all off the platform info.

witten commented

2023-08-29 21:10:04 +00:00

I have looked at some of the options for getting os and env info and the best choice seems to be using pythons platform for that.

Makes total sense to me!

One question that remains is: Where will we implement this? Does borgmatic log this every time someone runs with loglevel 2? I personally think for the os and version info it would be good to implement a new flag just like --version but name something like --version-env that outputs all off the platform info.

I think either would be fine, although one of the benefits of doing it at the start of every run with verbosity level 2 is that it would tend to make its way into logs that people post on tickets, which might make it easier to debug random issues. On the other hand, an explicit flag would have the benefit of not mucking up the other logs, and we could always say in the ticket prompt to run --version-env or whatever. I might have a slight bias for logging this info automatically every run, but I could see arguments either way.

> I have looked at some of the options for getting os and env info and the best choice seems to be using pythons platform for that. Makes total sense to me! > One question that remains is: Where will we implement this? Does borgmatic log this every time someone runs with loglevel 2? I personally think for the os and version info it would be good to implement a new flag just like --version but name something like --version-env that outputs all off the platform info. I think either would be fine, although one of the benefits of doing it at the start of every run with verbosity level 2 is that it would tend to make its way into logs that people post on tickets, which might make it easier to debug random issues. On the other hand, an explicit flag would have the benefit of not mucking up the other logs, and we could always say in the ticket prompt to run `--version-env` or whatever. I might have a slight bias for logging this info automatically every run, but I could see arguments either way.

IBims1NicerTobi commented

2023-09-01 03:31:34 +00:00

Ok so I have most of this done, the only roadblock I have hit is how to deal with borg versions. The issue is that a user can have as many different borg versions installed as they want (not that it is realistic). Should we just log all of the borg versions? In my opinion it would make the most sense to call every version of borg that is in all of the configs and keep the result and in the end build a log output with a list of configs and the corresponding borg version of the version is not the same in all of the config files.
Stuff I have working:

Python version and implementation
OS (distro and kernel for linux, version for windows and macos)
borgmatic version
Example form my machine (currently shows up as the first entries in the log):
Borgmatic version: 1.8.2.dev0
Architecture: x86_64
Python: CPython 3.11.5
Operating system: Linux
Kernel: 6.1.49-1-lts
Name: Arch Linux
Id: arch
Pretty name: Arch Linux
Build id: rolling
Ansi color: 38;2;23;147;209
Home url: https://archlinux.org/
Documentation url: https://wiki.archlinux.org/
Support url: https://bbs.archlinux.org/
Bug report url: https://bugs.archlinux.org/
Privacy policy url: https://terms.archlinux.org/docs/privacy-policy/
Logo: archlinux-logo

Ok so I have most of this done, the only roadblock I have hit is how to deal with borg versions. The issue is that a user can have as many different borg versions installed as they want (not that it is realistic). Should we just log all of the borg versions? In my opinion it would make the most sense to call every version of borg that is in all of the configs and keep the result and in the end build a log output with a list of configs and the corresponding borg version of the version is not the same in all of the config files. Stuff I have working: - Python version and implementation - OS (distro and kernel for linux, version for windows and macos) - borgmatic version Example form my machine (currently shows up as the first entries in the log): `Borgmatic version: 1.8.2.dev0` `Architecture: x86_64` `Python: CPython 3.11.5` `Operating system: Linux` `Kernel: 6.1.49-1-lts` `Name: Arch Linux` `Id: arch` `Pretty name: Arch Linux` `Build id: rolling` `Ansi color: 38;2;23;147;209` `Home url: https://archlinux.org/` `Documentation url: https://wiki.archlinux.org/` `Support url: https://bbs.archlinux.org/` `Bug report url: https://bugs.archlinux.org/` `Privacy policy url: https://terms.archlinux.org/docs/privacy-policy/` `Logo: archlinux-logo`

witten commented

2023-09-01 17:47:35 +00:00

That looks like a great start, although I'm not sure we need all those URLs and logos and stuff. What do you think about plucking out just one or two key fields from the /etc/os-release values and displaying those?

As for dealing with Borg versions, I'd suggest making this easy on yourself and reducing code complexity: In borgmatic/borg/version.py:local_borg_version(), which is called once per configuration file, simply log the version (with the relevant config file prefix)! That way, the appropriate Borg version will be displayed along with each configuration file's logs.

This approach could work for other types of version logging that's potentially per-configuration file as well if you include them as part of this ticket: Databases, etc.

The main downside I see is that you don't have all the versions of absolutely everything collected at the tippy top of borgmatic's logs, but IMO that's okay. You can have the "global" versions at the top, and then per-configuration versions as each configuration file is loaded and processed.

And the downside of the other approach, building up all the borg versions and displaying them at the end after all configuration files are processed, is you won't get to see the Borg version(s) if an error occurs and borgmatic exits early!

That looks like a great start, although I'm not sure we need all those URLs and logos and stuff. What do you think about plucking out just one or two [key fields](https://www.linux.org/docs/man5/os-release.html) from the `/etc/os-release` values and displaying those? As for dealing with Borg versions, I'd suggest making this easy on yourself and reducing code complexity: In `borgmatic/borg/version.py:local_borg_version()`, which is called once per configuration file, simply log the version (with the relevant config file prefix)! That way, the appropriate Borg version will be displayed along with each configuration file's logs. This approach could work for other types of version logging that's potentially per-configuration file as well if you include them as part of this ticket: Databases, etc. The main downside I see is that you don't have all the versions of absolutely everything collected at the tippy top of borgmatic's logs, but IMO that's okay. You can have the "global" versions at the top, and then per-configuration versions as each configuration file is loaded and processed. And the downside of the other approach, building up all the borg versions and displaying them at the end after all configuration files are processed, is you won't get to see the Borg version(s) if an error occurs and borgmatic exits early!

IBims1NicerTobi commented

2023-09-02 17:10:33 +00:00

I would probably only print PRETTY_NAME if available, maybe NAME and VERSION as fallback and nothing else otherwise as the distro probably matters least for debugging and it should be fine to miss out on some information related to that.
Also since I have done some work by now I would probably open a pull request and move the discussion there so we can all also have a look at the code.

I would probably only print `PRETTY_NAME` if available, maybe `NAME` and `VERSION` as fallback and nothing else otherwise as the distro probably matters least for debugging and it should be fine to miss out on some information related to that. Also since I have done some work by now I would probably open a pull request and move the discussion there so we can all also have a look at the code.

IBims1NicerTobi referenced this issue

2023-09-02 17:28:22 +00:00

WIP: Added debug log entries for borg, borgmatic and os version #750

witten commented

2023-09-04 17:33:59 +00:00

Sounds good. I will have a look when I get a chance!

witten referenced this issue from a commit

2023-11-25 03:59:41 +00:00

Add debug message that logs borg version for every config (#714).

Sign in to join this conversation.