Backup failes due to healthcheck fail, possible proxy issue #512

New Issue

frapit · 2022-03-23T09:36:23Z

frapit commented

2022-03-23 09:36:23 +00:00

What I'm trying to do and why

My backup was running fine until it didn't so I decided to add some healthcheck service.

Steps to reproduce (if a bug)

I added healthchecks: https://hc-ping.com/e8d831ce-2c15-XXXXXXXX in the hooks section. I think it may be a proxy problem. All environment variables are set and I can ping the healthcheck service from command line but apparently borgmatic cannnot reach it. Are proxy settings used?

Actual behavior (if a bug)

Mar 22 04:09:52 mss borgmatic[113090]: CRITICAL HTTPSConnectionPool(host='hc-ping.com', port=443): Max retries exceeded with url: /e8d831ce...
Mar 22 04:09:52 mss borgmatic[113090]: CRITICAL /etc/borgmatic/config.yaml: Error running pre hook
Mar 22 04:09:52 mss borgmatic[113090]: INFO /etc/borgmatic/config.yaml: Pinging Healthchecks fail

Environment

borgmatic version: [version here]

1.5.1

borgmatic installation method: [e.g., Debian package, Docker container, etc.]

debian package

Borg version: [version here]

borg 1.1.15

Python version: [version here]

Python 3.8.10

operating system and version: [OS here]

Ubuntu 20.04

#### What I'm trying to do and why My backup was running fine until it didn't so I decided to add some healthcheck service. #### Steps to reproduce (if a bug) I added `healthchecks: https://hc-ping.com/e8d831ce-2c15-XXXXXXXX` in the hooks section. I think it may be a proxy problem. All environment variables are set and I can ping the healthcheck service from command line but apparently borgmatic cannnot reach it. Are proxy settings used? #### Actual behavior (if a bug) ``` Mar 22 04:09:52 mss borgmatic[113090]: CRITICAL HTTPSConnectionPool(host='hc-ping.com', port=443): Max retries exceeded with url: /e8d831ce... Mar 22 04:09:52 mss borgmatic[113090]: CRITICAL /etc/borgmatic/config.yaml: Error running pre hook Mar 22 04:09:52 mss borgmatic[113090]: INFO /etc/borgmatic/config.yaml: Pinging Healthchecks fail ``` #### Environment **borgmatic version:** [version here] 1.5.1 **borgmatic installation method:** [e.g., Debian package, Docker container, etc.] debian package **Borg version:** [version here] borg 1.1.15 **Python version:** [version here] Python 3.8.10 **operating system and version:** [OS here] Ubuntu 20.04

witten commented

2022-03-23 18:15:19 +00:00

Thank you for filing this! borgmatic's Healthchecks hook doesn't have any special treatment for proxies. Can you tell me about the proxy settings you're using?

frapit commented

2022-03-24 15:13:22 +00:00

Sure, I am using a very basic corporate proxy. They are present in the environment as

http_proxy=http://proxy-domain.my-company.com:3128
https_proxy=http://proxy-domain.my-company.com:3128

but the requests api used by borgmatic is not picking up the environment per default.

I already tested the workaround by replacing the respective parts in the ping_monitor function in healthchecks.py with:

proxies = {
      'http': 'http://proxy-domain.my-company.com:3128',
      'https': 'http://proxy-domain.my-company.com:3128',
    }

if not dry_run:
    logging.getLogger('urllib3').setLevel(logging.ERROR)
    requests.post(ping_url, data=payload.encode('utf-8'), proxies=proxies)

With this, the healthcheck works fine.

Of course this should be done depending on a presence of an environment variable or maybe using a configuration option.

Sure, I am using a very basic corporate proxy. They are present in the environment as ``` http_proxy=http://proxy-domain.my-company.com:3128 https_proxy=http://proxy-domain.my-company.com:3128 ``` but the `requests` api used by borgmatic is not picking up the environment per default. I already tested the workaround by replacing the respective parts in the `ping_monitor` function in `healthchecks.py` with: ``` proxies = { 'http': 'http://proxy-domain.my-company.com:3128', 'https': 'http://proxy-domain.my-company.com:3128', } if not dry_run: logging.getLogger('urllib3').setLevel(logging.ERROR) requests.post(ping_url, data=payload.encode('utf-8'), proxies=proxies) ``` With this, the healthcheck works fine. Of course this should be done depending on a presence of an environment variable or maybe using a configuration option.

Sdub76 referenced this issue

2022-05-09 18:06:13 +00:00

Send --stats output to healthchecks #524

witten commented

2022-05-22 16:35:51 +00:00

I'm not sure why the environment variables aren't working on your machine. In my local testing, requests does use the https_proxy environment variable when a proxies= parameter isn't passed in. And the requests proxies documentations seems to support that.

Out of curiosity, if you run borgmatic with the environment variable, then does it get picked up properly? E.g.:

https_proxy=http://proxy-domain.my-company.com:3128 borgmatic create ...

Thanks!

I'm not sure why the environment variables aren't working on your machine. In my local testing, `requests` *does* use the `https_proxy` environment variable when a `proxies=` parameter isn't passed in. And the [requests proxies documentations](https://2.python-requests.org/en/master/user/advanced/#id10) seems to support that. Out of curiosity, if you run borgmatic _with_ the environment variable, then does it get picked up properly? E.g.: ``` https_proxy=http://proxy-domain.my-company.com:3128 borgmatic create ... ``` Thanks!

witten added the

waiting for response

label 2022-05-24 21:19:46 +00:00

witten commented

2022-10-05 06:14:38 +00:00

Closing this for now, but feel free to follow up. We can always reopen it. Thanks!

witten closed this issue

2022-10-05 06:14:38 +00:00

witten removed the

waiting for response

label 2022-10-05 06:14:46 +00:00

Sign in to join this conversation.