#151 Create automated binaries for borgmatic

Closed
opened 8 months ago by dverhelst · 9 comments

Hi,

Enterprise linux operating systems don’t easily support Python3/Pip3. Even from a deployment perspective, providing a binary makes sense for a python3 package.

Working on https://github.com/FiaasCo/borgbackup taught us deploying a binary of borg made much more sense.

Are there plans to implement a packaging system (such as https://github.com/cmarquardt/borgmatic-binary) for borgmatic?

Can we get this into a feature and start working on it?

Please share your thoughts.

Hi, Enterprise linux operating systems don't easily support Python3/Pip3. Even from a deployment perspective, providing a binary makes sense for a python3 package. Working on https://github.com/FiaasCo/borgbackup taught us deploying a binary of borg made much more sense. Are there plans to implement a packaging system (such as https://github.com/cmarquardt/borgmatic-binary) for borgmatic? Can we get this into a feature and start working on it? Please share your thoughts.
witten commented 8 months ago
Owner

You’ve probably seen it already, but there are a bunch of third-party distribution-specific options for installing borgmatic.

The borgmatic-binary project you linked to though looks like exactly what you’re asking about here. Is the issue that it’s not updated on every borgmatic release, and you’d like something more official or automated?

Also, could you say more about your use case, your Linux distribution, etc? Is the problem that pip just isn’t available at all, or is it an outdated version? Do you normally install things with the enterprise Linux’s package manager, or do you usually install binaries?

Thanks.

You've probably seen it already, but there are a bunch of third-party distribution-specific options for [installing borgmatic](https://torsion.org/borgmatic/docs/how-to/set-up-backups/#other-ways-to-install). The borgmatic-binary project you linked to though looks like exactly what you're asking about here. Is the issue that it's not updated on every borgmatic release, and you'd like something more official or automated? Also, could you say more about your use case, your Linux distribution, etc? Is the problem that pip just isn't available at all, or is it an outdated version? Do you normally install things with the enterprise Linux's package manager, or do you usually install binaries? Thanks.
dverhelst commented 8 months ago
Poster

The borgmatic-binary would be perfect if it was implemented into borgmatic just as borg does for releasing cross-platform binaries. That way I am able to quickly run borg on a variety of platform/cpus and deploy at scale.

Having widespread packages available for different distributions really is a good thing, but from an automation point of view, it requires adding 1 or more different repositories, packages, .. just to be able to get it to work. Every distribution is almost a custom-job to get it done (ever tried ‘managing’ python3/pip3 packages on CentOS?). (note: i’m all in favor of using regular packages (rpm/deb/…) but in some cases it causes more trouble than it fixes things if they are not in the main repositories of the distributions)

Having to go through that hassle vs. being able to just take a binary is just a much cleaner option from an automation point of view. It also allows for easy upgrades/downgrades. Borgmatic will be instantly available (platforms,cpus) and much easier to start playing with.

I would be willing to help out and try to implement this with pleasure provided we decide on what needs to be done. (I’m a automation/cicd fanatic, but I lack python dev skills).

The alternative will be to spawn a separate disconnected system to get these binaries built and deployed (much like the person who created the borgmatic-binary project). This is rarely a good thing.

The borgmatic-binary would be perfect if it was implemented into borgmatic just as borg does for releasing cross-platform binaries. That way I am able to quickly run borg on a variety of platform/cpus and deploy at scale. Having widespread packages available for different distributions really is a good thing, but from an automation point of view, it requires adding 1 or more different repositories, packages, .. just to be able to get it to work. Every distribution is almost a custom-job to get it done (ever tried 'managing' python3/pip3 packages on CentOS?). (note: i'm all in favor of using regular packages (rpm/deb/...) but in some cases it causes more trouble than it fixes things if they are not in the main repositories of the distributions) Having to go through that hassle vs. being able to just take a binary is just a much cleaner option from an automation point of view. It also allows for easy upgrades/downgrades. Borgmatic will be instantly available (platforms,cpus) and much easier to start playing with. I would be willing to help out and try to implement this with pleasure provided we decide on what needs to be done. (I'm a automation/cicd fanatic, but I lack python dev skills). The alternative will be to spawn a separate disconnected system to get these binaries built and deployed (much like the person who created the borgmatic-binary project). This is rarely a good thing.

Hello,

I would also like to have a pre-compiled binary for borgmatic!

It would help with running it on my Unraid server. Currently, I need to re-install it every reboot because packages installed do not persist after a reboot.

With a pre-compiled binary like I am using with Borg, I could just create a symbolic link at boot time rather than have to re-install via pip3 every boot…

Thank you!

Hello, I would also like to have a pre-compiled binary for borgmatic! It would help with running it on my Unraid server. Currently, I need to re-install it every reboot because packages installed do not persist after a reboot. With a pre-compiled binary like I am using with Borg, I could just create a symbolic link at boot time rather than have to re-install via pip3 every boot... Thank you!
witten commented 8 months ago
Owner

Binaries seems like a reasonable thing to have for use cases as described here, but also a fair amount of work to set up and maintain compared to the platform-agnostic packages that are built and distributed for borgmatic today. For instance, the very meager CI infrastructure that I’m running today may not be sufficient for building binaries.

Given that this need sounds like it’s for an enterprise Linux distribution in at least one case (maybe less so for Unraid), does a binary distribution of borgmatic + Borg sound like something your employer would purchase a license for? If so, that would certainly help me find the time to work on this sooner rather than later. (I’m assuming you’d be running enterprise Linux in a company of some sort with a budget for such things. If not, let me know!)

Binaries seems like a reasonable thing to have for use cases as described here, but also a fair amount of work to set up and maintain compared to the platform-agnostic packages that are built and distributed for borgmatic today. For instance, the very meager CI infrastructure that I'm running today may not be sufficient for building binaries. Given that this need sounds like it's for an enterprise Linux distribution in at least one case (maybe less so for Unraid), does a binary distribution of borgmatic + Borg sound like something your employer would purchase a license for? If so, that would certainly help me find the time to work on this sooner rather than later. (I'm assuming you'd be running enterprise Linux in a company of some sort with a budget for such things. If not, let me know!)
dverhelst commented 6 months ago
Poster

This request is not for an Enterprise-need, but it’s based on experiences on both open-source and enterprise. It’s the sheer variety of distro’s and older packages, obsolete python’s, … that makes using borgmatic difficult. Imagine a mix of:

  • el6
  • el7
  • squeeze
  • bionic
  • freebsd (freebsd)
  • ….

Currently I use a borg-wrapper-script which uses a built borg-binary and this approach works on (almost) all platforms. I can prebuild the binary and put it on the system without any problem and I can back up all my machines using borg.

This approach is what I would like for borgmatic.

If you point me in the correct direction on how to optimally build that binary I might be able to work something out for CI.

(again, this is not an enterprise-need, but an identified shortcoming of Borgmatic for me to be able to use it on all platforms borg support)

This request is not for an Enterprise-need, but it's based on experiences on both open-source and enterprise. It's the sheer variety of distro's and older packages, obsolete python's, ... that makes using borgmatic difficult. Imagine a mix of: - el6 - el7 - squeeze - bionic - freebsd (freebsd) - .... Currently I use a borg-wrapper-script which uses a built borg-binary and this approach works on (almost) all platforms. I can prebuild the binary and put it on the system without any problem and I can back up all my machines using borg. This approach is what I would like for borgmatic. If you point me in the correct direction on how to optimally build that binary I might be able to work something out for CI. (again, this is not an enterprise-need, but an identified shortcoming of Borgmatic for me to be able to use it on all platforms borg support)
witten commented 6 months ago
Owner

Thanks for clarifying! One thing I’m still a little bit unclear on though is the gap you see for something like borgmatic-binary to meet this need.

Earlier, you said:

The borgmatic-binary would be perfect if it was implemented into borgmatic just as borg does for releasing cross-platform binaries. That way I am able to quickly run borg on a variety of platform/cpus and deploy at scale.

So is the problem just the number of platforms that borgmatic-binary has binary releases for? It looks like right now, there are only releases for x86_64 Linux. Sounds like you ideally would like FreeBSD releases as well. Are there other platforms or architectures? And are there other concerns with using binary releases from a third-party project?

It appears that borgmatic-binary includes build instructions for any platform supporting Borg even if there aren’t pre-built binaries. So if you have your own CI, that may get your started building borgmatic for your target architectures/platforms.

I’m asking about all this because I want to understand your platform requirements here. borgmatic’s existing CI server is x86_64 Linux, so it would require some investment (both in time and hosting cost) to support other platforms like: FreeBSD, non-x86_64, etc.

Thanks for clarifying! One thing I'm still a little bit unclear on though is the gap you see for something like [borgmatic-binary](https://github.com/cmarquardt/borgmatic-binary) to meet this need. Earlier, you said: > The borgmatic-binary would be perfect if it was implemented into borgmatic just as borg does for releasing cross-platform binaries. That way I am able to quickly run borg on a variety of platform/cpus and deploy at scale. So is the problem just the number of platforms that borgmatic-binary has binary releases for? It looks like right now, there are only [releases for x86_64 Linux](https://github.com/cmarquardt/borgmatic-binary/releases). Sounds like you ideally would like FreeBSD releases as well. Are there other platforms or architectures? And are there other concerns with using binary releases from a third-party project? It appears that borgmatic-binary includes build [instructions](https://github.com/cmarquardt/borgmatic-binary#building-manually) for any platform supporting Borg even if there aren't pre-built binaries. So if you have your own CI, that may get your started building borgmatic for your target architectures/platforms. I'm asking about all this because I want to understand your platform requirements here. borgmatic's existing CI server is x86_64 Linux, so it would require some investment (both in time and hosting cost) to support other platforms like: FreeBSD, non-x86_64, etc.

Hi,

sorry for joining the discussion so late - and thanks to @witten for making me aware of it.

I started borgmatic-binary for the reasons @dverheslt mentions. Similar to the borg binaries, they are also built with PyInstaller. They thus should have the same advantages (and occasional issues).

Technically, I’m creating release files via Travis (GitHub’s CI) whenever there’s a new tag; that pushes a Linux release to GitHub. I usually do a manual build for OS-X as well and upload it, but I don’t have other OSs like FreeBSD available. And as you’ve found out, there’s no automatism in generating new releases based on borgmatic - sorry…

Building such an automatism isn’t overly difficult. It would be even easier if @witten’s CI would trigger them directly. I’m more than happy to contribute the scripts and PyInstaller configuration if Dan is willing to take them. That said - the binaries do require additional effort to maintain (and test) them. For example, I’m currently running into trouble when putting them on Linux machines with different Linux variants just because they have different versions of openSSL. I’m sure that can be solved by configuring PyInstaller correctly, but solving the problem generically may cost a lot of effort (I have little experience with that particular tool). Supporting more platforms, as Dan mentions, means even more work. I believe borg does it with virtual machines and vagrant, but that kind of infrastructure needs to be built and maintained in the first place.

In practice, I would agree with Dan that building the binaries on a development machine and then deploying them to the rest of your machines is by far the easiest solution.

Hi, sorry for joining the discussion so late - and thanks to @witten for making me aware of it. I started borgmatic-binary for the reasons @dverheslt mentions. Similar to the [borg binaries](https://borgbackup.readthedocs.io/en/stable/installation.html#pyinstaller-binary), they are also built with PyInstaller. They thus should have the same advantages (and occasional issues). Technically, I'm creating release files via Travis (GitHub's CI) whenever there's a new tag; that pushes a Linux release to GitHub. I usually do a manual build for OS-X as well and upload it, but I don't have other OSs like FreeBSD available. And as you've found out, there's no automatism in generating new releases based on borgmatic - sorry... Building such an automatism isn't overly difficult. It would be even easier if @witten's CI would trigger them directly. I'm more than happy to contribute the scripts and PyInstaller configuration if Dan is willing to take them. That said - the binaries do require additional effort to maintain (and test) them. For example, I'm currently running into trouble when putting them on Linux machines with different Linux variants just because they have different versions of openSSL. I'm sure that can be solved by configuring PyInstaller correctly, but solving the problem generically may cost a lot of effort (I have little experience with that particular tool). Supporting more platforms, as Dan mentions, means even more work. I believe `borg` does it with virtual machines and `vagrant`, but that kind of infrastructure needs to be built and maintained in the first place. In practice, I would agree with Dan that building the binaries on a development machine and then deploying them to the rest of your machines is by far the easiest solution.
witten commented 6 months ago
Owner

FWIW, I’d be happy to look into triggering an external build (like borgmatic-binary) from borgmatic’s CI. An alternative though might be for something external to poll for new borgmatic releases and use that information to trigger binary builds. Example:

$ curl --silent -X GET "https://projects.torsion.org/api/v1/repos/witten/borgmatic/releases" -H  "accept: application/json" | jq .[0]

{
  "id": 78,
  "tag_name": "1.3.4",
  "target_commitish": "",
  "name": "borgmatic 1.3.4",
  ...
}

But before jumping to solutions, it would be good to understand what problems (if any) we’re solving for!

FWIW, I'd be happy to look into triggering an external build (like borgmatic-binary) from borgmatic's CI. An alternative though might be for something external to poll for new borgmatic releases and use that information to trigger binary builds. Example: ``` $ curl --silent -X GET "https://projects.torsion.org/api/v1/repos/witten/borgmatic/releases" -H "accept: application/json" | jq .[0] { "id": 78, "tag_name": "1.3.4", "target_commitish": "", "name": "borgmatic 1.3.4", ... } ``` But before jumping to solutions, it would be good to understand what problems (if any) we're solving for!
witten commented 2 months ago
Owner

Closing due to inactivity, but please feel free to file a new ticket with specific additions! Or re-open this one with more info.

Closing due to inactivity, but please feel free to file a new ticket with specific additions! Or re-open this one with more info.
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
Cancel
Save
There is no content yet.