borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 57
Code Issues 71 Pull requests 3 Releases 162 Packages 1 Activity Actions 1

pg_dump_command: need detailed explanation #1020

New issue
Closed
opened 2025-03-08 20:34:06 +00:00 by gander · 4 comments
gander commented 2025-03-08 20:34:06 +00:00
Copy link

What I'm trying to do and why

Configure pg_dump_command to dump from the PostgreSQL container, since I can't update the PostgreSQL client.

Steps to reproduce

First attempt, exactly as in the documentation:

pg_dump_command: docker exec my_pg_container pg_dump

Second attempt, with attaching my configuration directory (I am using borgmatic.d)

pg_dump_command: docker run --rm -v /home/user/.config/borgmatic.d:/home/user/.config/borgmatic.d postgres:17-alpine pg_dump

Third attempt, with attaching the directory with the default configuration location:

pg_dump_command: docker run --rm -v /home/user/.config/borgmatic:/home/user/.config/borgmatic postgres:17-alpine pg_dump

And so on, I also tried with ~/.pgpass in host, in the container, still no success.

Actual behavior

An error occurred
Error running actions for repository
pg_dump: error: connection to server at "<REDACTED>" (<REDACTED>), port <REDACTED> failed: fe_sendauth: no password supplied
Command 'docker run --rm postgres:17-alpine pg_dump --no-password --clean --if-exists --host <REDACTED> --port <REDACTED> --username <REDACTED> --no-owner --format custom <REDACTED> > /run/user/<REDACTED>/./borgmatic/postgresql_databases/<REDACTED>/<REDACTED>' returned non-zero exit status 1.

Expected behavior

backup without error

Other notes / implementation ideas

I finally started debugging this configuration and found that all I had to do was do it this way to make it work:

pg_dump_command: docker run --rm -e PGPASSWORD postgres:17-alpine pg_dump

In my opinion, the documentation needs to explain in more detail how the password is passed to the tool (environment variable) and, for the docker example, add passing this environment variable through this option -e PGPASSWORD.

borgmatic version

1.9.13

borgmatic installation method

pipx install borgmatic

Borg version

1.4.0

Python version

3.11.4

Database version (if applicable)

17.3

Operating system and version

Ubuntu 23.04

### What I'm trying to do and why Configure `pg_dump_command` to dump from the PostgreSQL container, since I can't update the PostgreSQL client. ### Steps to reproduce First attempt, exactly as in the documentation: ``` pg_dump_command: docker exec my_pg_container pg_dump ``` Second attempt, with attaching my configuration directory (I am using borgmatic.d) ``` pg_dump_command: docker run --rm -v /home/user/.config/borgmatic.d:/home/user/.config/borgmatic.d postgres:17-alpine pg_dump ``` Third attempt, with attaching the directory with the default configuration location: ``` pg_dump_command: docker run --rm -v /home/user/.config/borgmatic:/home/user/.config/borgmatic postgres:17-alpine pg_dump ``` And so on, I also tried with `~/.pgpass` in `host`, in the `container`, still no success. ### Actual behavior ``` An error occurred Error running actions for repository pg_dump: error: connection to server at "<REDACTED>" (<REDACTED>), port <REDACTED> failed: fe_sendauth: no password supplied Command 'docker run --rm postgres:17-alpine pg_dump --no-password --clean --if-exists --host <REDACTED> --port <REDACTED> --username <REDACTED> --no-owner --format custom <REDACTED> > /run/user/<REDACTED>/./borgmatic/postgresql_databases/<REDACTED>/<REDACTED>' returned non-zero exit status 1. ``` ### Expected behavior backup without error ### Other notes / implementation ideas I finally started debugging this configuration and found that all I had to do was do it this way to make it work: ``` pg_dump_command: docker run --rm -e PGPASSWORD postgres:17-alpine pg_dump ``` In my opinion, the [documentation](https://torsion.org/borgmatic/docs/reference/configuration/) needs to explain in more detail how the password is passed to the tool (environment variable) and, for the docker example, add passing this environment variable through this option `-e PGPASSWORD`. ### borgmatic version 1.9.13 ### borgmatic installation method pipx install borgmatic ### Borg version 1.4.0 ### Python version 3.11.4 ### Database version (if applicable) 17.3 ### Operating system and version Ubuntu 23.04
witten commented 2025-03-09 17:17:20 +00:00
Owner
Copy link

Thanks for bringing this to my attention! I'm glad to hear you've got things working on your system now, but I'm not sure why it wasn't working to begin with. For instance, when I run a PostgreSQL container like this:

podman run -it --rm -p 5432:5432 -e POSTGRES_USER=test -e POSTGRES_DB=test -e POSTGRES_PASSWORD=pass --name postgresql docker.io/postgres:14.3

... and then exec into it like this:

podman exec -it postgresql bash
pg_dump --username test --format custom

... then I get the full database dump without being prompted for a password. Which is similar to the series of steps that borgmatic does when configured with pg_dump_command: docker exec postgresql pg_dump. My understanding is that the database password is not needed because exec-ing in bypasses that. From the docs:

Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container.

So I'm not sure what's going wrong on your system. Can I get a look at the error you get when docker exec ... is configured in borgmatic? And what about exec-ing in and running pg_dump manually? Also, are you setting POSTGRES_PASSWORD when creating your PostgreSQL container?

Thanks for bringing this to my attention! I'm glad to hear you've got things working on your system now, but I'm not sure why it wasn't working to begin with. For instance, when I run a PostgreSQL container like this: ```bash podman run -it --rm -p 5432:5432 -e POSTGRES_USER=test -e POSTGRES_DB=test -e POSTGRES_PASSWORD=pass --name postgresql docker.io/postgres:14.3 ``` ... and then exec into it like this: ```bash podman exec -it postgresql bash pg_dump --username test --format custom ``` ... then I get the full database dump without being prompted for a password. Which is similar to the series of steps that borgmatic does when configured with `pg_dump_command: docker exec postgresql pg_dump`. My understanding is that the database password is not needed because exec-ing in bypasses that. From [the docs](https://github.com/docker-library/docs/blob/master/postgres/README.md#postgres_password): > Note 1: The PostgreSQL image sets up trust authentication locally so you may notice a password is not required when connecting from localhost (inside the same container). However, a password will be required if connecting from a different host/container. So I'm not sure what's going wrong on your system. Can I get a look at the error you get when `docker exec ...` is configured in borgmatic? And what about exec-ing in and running `pg_dump` manually? Also, are you setting `POSTGRES_PASSWORD` when creating your PostgreSQL container?
gander commented 2025-03-09 21:44:38 +00:00
Author
Copy link

I don't have a postgres container locally. I connect to a database on another server. However, I need to use the postgres container with the pg_dump client to connect to that database.

I don't have a postgres container locally. I connect to a database on another server. However, I need to use the postgres container with the pg_dump client to connect to that database.
witten commented 2025-03-12 21:43:02 +00:00
Owner
Copy link

Ah okay that makes more sense! I'll see if I can work this use case into the documentation.

Ah okay that makes more sense! I'll see if I can work this use case into the documentation.
witten commented 2025-03-13 00:13:07 +00:00
Owner
Copy link

Okay, I attempted to document this use case here:

https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#containers

Those new docs should get deployed shortly.

Thanks for the suggestion!

Okay, I attempted to document this use case here: https://torsion.org/borgmatic/docs/how-to/backup-your-databases/#containers Those new docs should get deployed shortly. Thanks for the suggestion!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
parallel-borgmatic
command-hook-steps
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#1020
No description provided.