Cron job config for virtual environments #752

New Issue

everydayrain · 2023-09-05T16:35:25Z

everydayrain commented

2023-09-05 16:35:25 +00:00

What I'm trying to do and why

Hi, I responded in thread #751 on Debian's switch to virtual environments. I'm following up on the cron issue, similar to issue #68: when I install borgmatic using venv (sorry did that before you changed the instructions), I think it installs a user/local copy that is unavailable globally. I'm new to python virtual environments, so forgive my mistakes here.

The error messages borgmatic gave me about being unable to read /etc/ files were therefore super clear, and I simply created .yaml files in my local .config directory to run it manually. When it comes to the cron syntax however, I tried the standard root syntax, and played around with trying my user to initiate the job, but nothing has worked.

Steps to reproduce

I went with venv because I wanted to learn how python virtual environments work "under the hood" (hence the headaches, pipx was probably the right recommendation). I tried the following both with and without sudo:

sudo apt install python3-venv

python3 -m venv .venv/SOME_PROJECT

source .venv/SOME_PROJECT/bin/activate

.venv/SOME_PROJECT/bin/pip install borgmatic

I have config files already set up from prior to my OS upgrade. Then, either in cron.d or in crontab -e I tried using:

0 3 * * * root PATH=$PATH:/usr/bin:/usr/local/bin /root/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1 >> /tmp/borg.log 2>&1

Actual behavior

Cron job simply doesn't run, the tmp error log reads:

/bin/sh: 1: root: not found

Expected behavior

No response

Other notes / implementation ideas

No response

borgmatic version

1.8.2

borgmatic installation method

pip install using venv

Borg version

1.2.4

Python version

3.11.4

Database version (if applicable)

No response

Operating system and version

PRETTY_NAME="Ubuntu 23.04" 2 │ NAME="Ubuntu" 3 │ VERSION_ID="23.04" 4 │ VERSION="23.04 (Lunar Lobster)" 5 │ VERSION_CODENAME=lunar 6 │ ID=ubuntu 7 │ ID_LIKE=debian 8 │ HOME_URL="https://www.ubuntu.com/" 9 │ SUPPORT_URL="https://help.ubuntu.com/" 10 │ BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 11 │ PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 12 │ UBUNTU_CODENAME=lunar 13 │ LOGO=ubuntu-logo

### What I'm trying to do and why Hi, I responded in [thread #751](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/751) on Debian's switch to virtual environments. I'm following up on the cron issue, similar to [issue #68](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/68): when I install borgmatic using venv (sorry did that before you changed the instructions), I think it installs a user/local copy that is unavailable globally. I'm new to python virtual environments, so forgive my mistakes here. The error messages borgmatic gave me about being unable to read /etc/ files were therefore super clear, and I simply created .yaml files in my local .config directory to run it manually. When it comes to the cron syntax however, I tried the standard root syntax, and played around with trying my user to initiate the job, but nothing has worked. ### Steps to reproduce I went with venv because I wanted to learn how python virtual environments work "under the hood" (hence the headaches, pipx was probably the right recommendation). I tried the following both with and without sudo: ``` sudo apt install python3-venv ``` ``` python3 -m venv .venv/SOME_PROJECT ``` ``` source .venv/SOME_PROJECT/bin/activate ``` ``` .venv/SOME_PROJECT/bin/pip install borgmatic ``` I have config files already set up from prior to my OS upgrade. Then, either in `cron.d` or in `crontab -e` I tried using: ``` 0 3 * * * root PATH=$PATH:/usr/bin:/usr/local/bin /root/.local/bin/borgmatic --verbosity -1 --syslog-verbosity 1 >> /tmp/borg.log 2>&1 ``` ### Actual behavior Cron job simply doesn't run, the tmp error log reads: ``` /bin/sh: 1: root: not found ``` ### Expected behavior _No response_ ### Other notes / implementation ideas _No response_ ### borgmatic version 1.8.2 ### borgmatic installation method pip install using venv ### Borg version 1.2.4 ### Python version 3.11.4 ### Database version (if applicable) _No response_ ### Operating system and version PRETTY_NAME="Ubuntu 23.04" 2 │ NAME="Ubuntu" 3 │ VERSION_ID="23.04" 4 │ VERSION="23.04 (Lunar Lobster)" 5 │ VERSION_CODENAME=lunar 6 │ ID=ubuntu 7 │ ID_LIKE=debian 8 │ HOME_URL="https://www.ubuntu.com/" 9 │ SUPPORT_URL="https://help.ubuntu.com/" 10 │ BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 11 │ PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 12 │ UBUNTU_CODENAME=lunar 13 │ LOGO=ubuntu-logo

witten commented

2023-09-05 17:45:28 +00:00

Fortunately, I think this should be a pretty easy fix! User-specific crontabs don't include the user, because they run as the user they're installed as. So if you're using crontab -e to create/edit your crontab, then simply omit root from that crontab entry.

I certainly don't expect you to have used pipx prior to me having documented it, but if and when you feel sufficiently comfortable with learning about venvs, you may consider uninstalling borgmatic and reinstalling with pipx. It just streamlines a bunch of the process including upgrades!

Also, unrelated, but you may want to do --verbosity -2 instead of -1 since you're logging to syslog. That'll prevent duplicate errors going to your syslog logs.

Fortunately, I think this should be a pretty easy fix! User-specific crontabs don't include the user, because they run as the user they're installed as. So if you're using `crontab -e` to create/edit your crontab, then simply omit `root` from that crontab entry. I certainly don't expect you to have used pipx prior to me having documented it, but if and when you feel sufficiently comfortable with learning about venvs, you may consider uninstalling borgmatic and reinstalling with pipx. It just streamlines a bunch of the process including upgrades! Also, unrelated, but you may want to do `--verbosity -2` instead of `-1` since you're logging to syslog. That'll prevent duplicate errors going to your syslog logs.

witten added the

question / support

label 2023-09-05 17:45:39 +00:00

everydayrain commented

2023-09-05 18:26:05 +00:00

Does pipx automatically install a symlink to /root/.local/bin? There was no symlink to my root user's .local directory on my machine. That might be where venv diverges? Thus, I couldn't just take out root from the cron job syntax.

I got it to work by creating a symlink to my user's .local/bin directory, and specifying that path in the user-specific crontab crontab -e:

* 3 * * * PATH=$PATH:/usr/bin:/usr/local/bin /home/MY_USER/.local/bin/borgmatic --verbosity -2 --syslog-verbosity 1 >> /tmp/borg.log 2>&1

Everything works fine for me now. Thanks for the pointer about the logs and the assistance overall!

Does pipx automatically install a symlink to `/root/.local/bin`? There was no symlink to my root user's `.local` directory on my machine. That might be where venv diverges? Thus, I couldn't just take out _root_ from the cron job syntax. I got it to work by creating a symlink to my user's `.local/bin` directory, and specifying that path in the user-specific crontab `crontab -e`: ``` * 3 * * * PATH=$PATH:/usr/bin:/usr/local/bin /home/MY_USER/.local/bin/borgmatic --verbosity -2 --syslog-verbosity 1 >> /tmp/borg.log 2>&1 ``` Everything works fine for me now. Thanks for the pointer about the logs and the assistance overall!

witten commented

2023-09-05 22:32:53 +00:00

Does pipx automatically install a symlink to /root/.local/bin? There was no symlink to my root user's .local directory on my machine. That might be where venv diverges?

No, but it will put ~/.local/bin in your user's system PATH if you go through its install steps (specifically pipx ensurepath). And this accomplishes the same end effect as the symlink: Being able to run borgmatic commands without giving a full path. Note that this only applies to the current user, so if you run it with sudo, it'll do it for the root user.

I got it to work by creating a symlink to my user's .local/bin directory, and specifying that path in the user-specific crontab crontab -e:

That totally sounds like it'll work, but you may want to consider the pipx ensurepath approach if you want to be a little more "on the rails." Up to you.

Glad everything's working for you though! Please file tickets for any other problems you encounter.

> Does pipx automatically install a symlink to /root/.local/bin? There was no symlink to my root user's .local directory on my machine. That might be where venv diverges? No, but it _will_ put `~/.local/bin` in your user's system `PATH` if you go through its install steps (specifically `pipx ensurepath`). And this accomplishes the same end effect as the symlink: Being able to run borgmatic commands without giving a full path. Note that this only applies to the current user, so if you run it with `sudo`, it'll do it for the root user. > I got it to work by creating a symlink to my user's .local/bin directory, and specifying that path in the user-specific crontab crontab -e: That totally sounds like it'll work, but you may want to consider the `pipx ensurepath` approach if you want to be a little more "on the rails." Up to you. Glad everything's working for you though! Please file tickets for any other problems you encounter.

witten closed this issue

2023-09-05 22:32:53 +00:00

witten referenced this issue

2023-09-06 17:23:58 +00:00

WIP: Added debug log entries for borg, borgmatic and os version #750

Sign in to join this conversation.