borgmatic hangs on postgres restore #430

New Issue

Ghost · 2021-07-08T16:13:45Z

Ghost commented

2021-07-08 16:13:45 +00:00

What I'm trying to do and why

Hello,

I'm trying to restore PostgreSQL database dumps made with borgmatic:

borgmatic -v2 --config file.yaml restore --archive archivename

This is an extract of the configuration file:

hooks:
    postgresql_databases:
        # Settings used by pg_dump and pg_restore
        - name: postgres_prod
          hostname: 127.0.0.1
          port: 3051
          username: postgres
          password: postgres
          format: tar
          options: "--verbose"

I have removed:

~/.borgmatic
~/.cache/borgmatic
the repository (and then recreated it with borg init -e repokey myrepo.borg)

just in case

Actual behavior (if a bug)

Borgmatic just hangs

output with -v 2:

Ensuring legacy configuration is upgraded
myrepo.borg: Restoring databases from archive archivename
myrepo.borg: Calling postgresql_databases hook function remove_database_dumps
myrepo.borg: Removing PostgreSQL database dumps
myrepo.borg: Calling postgresql_databases hook function make_database_dump_pattern
borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod >
myrepo.borg: Calling postgresql_databases hook function restore_database_dump
myrepo.borg: Restoring PostgreSQL database postgres_prod
pg_restore --no-password --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres < 3
using builtin fallback logging configuration
35 self tests completed in 0.08 seconds
Verified integrity of myrepo.borg/index.5
TAM-verified manifest
security: read previous location 'myrepo.borg'
security: read manifest timestamp '2021-07-08T15:30:30.203073'
security: determined newest manifest timestamp as 2021-07-08T15:30:30.203073
security: repository checks ok, allowing access
home/preview/.borgmatic/postgresql_databases/127.0.0.1/postgres_prod

traceback after CTRL+C:

^CTraceback (most recent call last):
  File "/home/preview/.local/bin/borgmatic", line 10, in <module>
    sys.exit(main())
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 804, in main
    summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments))
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 702, in collect_configuration_run_summary_logs
    results = list(run_configuration(config_filename, config, arguments))
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 134, in run_configuration
    repository_path=repository_path,
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 486, in run_actions
    extract_process,
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 60, in call_hooks
    for hook_name in hook_names
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 61, in <dictcomp>
    if hooks.get(hook_name)
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 41, in call_hook
    return getattr(module, function_name)(config, log_prefix, *args, **kwargs)
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/postgresql.py", line 177, in restore_database_dump
    borg_local_path=location_config.get('local_path', 'borg'),
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/execute.py", line 261, in execute_command_with_processes
    borg_local_path=borg_local_path,
  File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/execute.py", line 90, in log_outputs
    exit_code = process.poll() if output_buffers else process.wait()
  File "/usr/lib/python3.7/subprocess.py", line 966, in poll
    return self._internal_poll()
  File "/usr/lib/python3.7/subprocess.py", line 1561, in _internal_poll
    pid, sts = _waitpid(self.pid, _WNOHANG)
KeyboardInterrupt

output with strace: this repeats continuously.

wait4(12375, 0x7ffc30fcbddc, WNOHANG, NULL) = 0
select(7, [5 6], [], [], NULL)          = 1 (in [5])
read(5, "", 4096)                       = 0

more strace

wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
select(7, [5 6], [], [], NULL)          = 1 (in [6])
getpid()                                = 12610
write(1, "security: determined newest mani"..., 76security: determined newest manifest timestamp as 2021-07-08T15:30:30.203073) = 76
write(1, "\n", 1
)                       = 1
wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
select(7, [5 6], [], [], NULL)          = 1 (in [6])
read(6, "security: repository checks ok, "..., 4096) = 48
getpid()                                = 12610
write(1, "security: repository checks ok, "..., 47security: repository checks ok, allowing access) = 47
write(1, "\n", 1
)                       = 1
wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
select(7, [5 6], [], [], NULL)          = 1 (in [6])
read(6, "home/preview/.borgmatic/postgres"..., 4096) = 69
getpid()                                = 12610
write(1, "home/preview/.borgmatic/postgres"..., 68home/preview/.borgmatic/postgresql_databases/127.0.0.1/postgres_prod) = 68
write(1, "\n", 1
)                       = 1
wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0
select(7, [5 6], [], [], NULL^Cstrace: Process 12610 detached
 <detached ...>

output of ps aux | grep borg:

preview  12491  5.4  0.4  41696 29056 pts/0    S+   17:51   0:00 /usr/bin/python3 /home/preview/.local/bin/borgmatic -v2 --config file.yaml restore --archive archivename
preview  12492  9.5  0.6  51508 39644 pts/0    S+   17:51   0:00 /usr/bin/python3 /usr/bin/borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod
root     12503  0.0  0.0   6076   828 pts/1    S+   17:51   0:00 grep borg

Other notes / implementation ideas

it works by restoring manually with pg_restore, after running the extract command of borgmatic:

pg_restore --verbose --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres < dump.tar

Environment

borgmatic version: 1.5.15

borgmatic installation method: pip3

Borg version: 1.1.15

Python version: 3.7.3

Database version (if applicable): psql (PostgreSQL) 13.3 (Debian 13.3-1.pgdg100+1)

operating system and version: Debian GNU/Linux 10

#### What I'm trying to do and why Hello, I'm trying to restore PostgreSQL database dumps made with borgmatic: ```shell borgmatic -v2 --config file.yaml restore --archive archivename ``` This is an extract of the configuration file: ```yaml hooks: postgresql_databases: # Settings used by pg_dump and pg_restore - name: postgres_prod hostname: 127.0.0.1 port: 3051 username: postgres password: postgres format: tar options: "--verbose" ```` I have removed: - `~/.borgmatic` - `~/.cache/borgmatic` - the repository (and then recreated it with `borg init -e repokey myrepo.borg`) just in case #### Actual behavior (if a bug) Borgmatic just hangs output with `-v 2`: ``` Ensuring legacy configuration is upgraded myrepo.borg: Restoring databases from archive archivename myrepo.borg: Calling postgresql_databases hook function remove_database_dumps myrepo.borg: Removing PostgreSQL database dumps myrepo.borg: Calling postgresql_databases hook function make_database_dump_pattern borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod > myrepo.borg: Calling postgresql_databases hook function restore_database_dump myrepo.borg: Restoring PostgreSQL database postgres_prod pg_restore --no-password --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres < 3 using builtin fallback logging configuration 35 self tests completed in 0.08 seconds Verified integrity of myrepo.borg/index.5 TAM-verified manifest security: read previous location 'myrepo.borg' security: read manifest timestamp '2021-07-08T15:30:30.203073' security: determined newest manifest timestamp as 2021-07-08T15:30:30.203073 security: repository checks ok, allowing access home/preview/.borgmatic/postgresql_databases/127.0.0.1/postgres_prod ``` traceback after CTRL+C: ``` ^CTraceback (most recent call last): File "/home/preview/.local/bin/borgmatic", line 10, in <module> sys.exit(main()) File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 804, in main summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments)) File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 702, in collect_configuration_run_summary_logs results = list(run_configuration(config_filename, config, arguments)) File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 134, in run_configuration repository_path=repository_path, File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/commands/borgmatic.py", line 486, in run_actions extract_process, File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 60, in call_hooks for hook_name in hook_names File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 61, in <dictcomp> if hooks.get(hook_name) File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/dispatch.py", line 41, in call_hook return getattr(module, function_name)(config, log_prefix, *args, **kwargs) File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/hooks/postgresql.py", line 177, in restore_database_dump borg_local_path=location_config.get('local_path', 'borg'), File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/execute.py", line 261, in execute_command_with_processes borg_local_path=borg_local_path, File "/home/preview/.local/lib/python3.7/site-packages/borgmatic/execute.py", line 90, in log_outputs exit_code = process.poll() if output_buffers else process.wait() File "/usr/lib/python3.7/subprocess.py", line 966, in poll return self._internal_poll() File "/usr/lib/python3.7/subprocess.py", line 1561, in _internal_poll pid, sts = _waitpid(self.pid, _WNOHANG) KeyboardInterrupt ``` output with `strace`: this repeats continuously. ``` wait4(12375, 0x7ffc30fcbddc, WNOHANG, NULL) = 0 select(7, [5 6], [], [], NULL) = 1 (in [5]) read(5, "", 4096) = 0 ``` more strace ``` wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 select(7, [5 6], [], [], NULL) = 1 (in [6]) getpid() = 12610 write(1, "security: determined newest mani"..., 76security: determined newest manifest timestamp as 2021-07-08T15:30:30.203073) = 76 write(1, "\n", 1 ) = 1 wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 select(7, [5 6], [], [], NULL) = 1 (in [6]) read(6, "security: repository checks ok, "..., 4096) = 48 getpid() = 12610 write(1, "security: repository checks ok, "..., 47security: repository checks ok, allowing access) = 47 write(1, "\n", 1 ) = 1 wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 select(7, [5 6], [], [], NULL) = 1 (in [6]) read(6, "home/preview/.borgmatic/postgres"..., 4096) = 69 getpid() = 12610 write(1, "home/preview/.borgmatic/postgres"..., 68home/preview/.borgmatic/postgresql_databases/127.0.0.1/postgres_prod) = 68 write(1, "\n", 1 ) = 1 wait4(12611, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 wait4(12612, 0x7fffd3e6e34c, WNOHANG, NULL) = 0 select(7, [5 6], [], [], NULL^Cstrace: Process 12610 detached <detached ...> ``` output of `ps aux | grep borg`: ``` preview 12491 5.4 0.4 41696 29056 pts/0 S+ 17:51 0:00 /usr/bin/python3 /home/preview/.local/bin/borgmatic -v2 --config file.yaml restore --archive archivename preview 12492 9.5 0.6 51508 39644 pts/0 S+ 17:51 0:00 /usr/bin/python3 /usr/bin/borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod root 12503 0.0 0.0 6076 828 pts/1 S+ 17:51 0:00 grep borg ``` #### Other notes / implementation ideas it works by restoring manually with `pg_restore`, after running the extract command of borgmatic: ```shell pg_restore --verbose --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres < dump.tar ``` #### Environment **borgmatic version:** 1.5.15 **borgmatic installation method:** pip3 **Borg version:** 1.1.15 **Python version:** 3.7.3 **Database version (if applicable):** `psql (PostgreSQL) 13.3 (Debian 13.3-1.pgdg100+1)` **operating system and version:** Debian GNU/Linux 10

witten commented

2021-07-08 16:53:54 +00:00

Thank you for providing the detailed diagnostic information. Looking at the logs and your configuration, I'm not seeing any obvious problems.

I see that you already tried a manual pg_restore, but one other thing you can try is a direct pipe to simulate what borgmatic is doing—but cutting borgmatic out of the picture:

borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod | pg_restore --verbose --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres

If that works, then it's pretty likely a problem with the way borgmatic is executing processes. If it doesn't work, then it's a problem with Borg or pg_restore (or the flags being provided to them). Personally, my money is on a borgmatic issue. But probably good to check.

Thank you for providing the detailed diagnostic information. Looking at the logs and your configuration, I'm not seeing any obvious problems. I see that you already tried a manual `pg_restore`, but one other thing you can try is a direct pipe to simulate what borgmatic is doing—but cutting borgmatic out of the picture: ``` borg extract --debug --list --show-rc --stdout myrepo.borg::archivename sh:home/preview/.borgmatic/postgresql_databases/*/postgres_prod | pg_restore --verbose --if-exists --exit-on-error --clean --dbname postgres_prod --host 127.0.0.1 --port 3051 --username postgres ``` If that works, then it's pretty likely a problem with the way borgmatic is executing processes. If it doesn't work, then it's a problem with Borg or `pg_restore` (or the flags being provided to them). Personally, my money is on a borgmatic issue. But probably good to check.

Ghost commented

2021-07-09 09:06:54 +00:00

Ok, tried your command:

Local Exception
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4455, in main
    exit_code = archiver.run(args)
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4387, in run
    return set_ec(func(args))
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 154, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 167, in wrapper
    return method(self, args, repository=repository, manifest=manifest, key=key, archive=archive, **kwargs)
  File "/usr/lib/python3/dist-packages/borg/archiver.py", line 765, in do_extract
    stripped_components=strip_components, original_path=orig_path, pi=pi)
  File "/usr/lib/python3/dist-packages/borg/archive.py", line 559, in extract_item
    sys.stdout.buffer.write(data)
BrokenPipeError: [Errno 32] Broken pipe

but I guess it's normal since we are using a pipe. Anyway it gets to the end and everything seems to be working.
This time I tried with borg 1.1.9 since I am on another machine.

Ok, tried your command: ``` Local Exception Traceback (most recent call last): File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4455, in main exit_code = archiver.run(args) File "/usr/lib/python3/dist-packages/borg/archiver.py", line 4387, in run return set_ec(func(args)) File "/usr/lib/python3/dist-packages/borg/archiver.py", line 154, in wrapper return method(self, args, repository=repository, **kwargs) File "/usr/lib/python3/dist-packages/borg/archiver.py", line 167, in wrapper return method(self, args, repository=repository, manifest=manifest, key=key, archive=archive, **kwargs) File "/usr/lib/python3/dist-packages/borg/archiver.py", line 765, in do_extract stripped_components=strip_components, original_path=orig_path, pi=pi) File "/usr/lib/python3/dist-packages/borg/archive.py", line 559, in extract_item sys.stdout.buffer.write(data) BrokenPipeError: [Errno 32] Broken pipe ``` but I guess it's normal since we are using a pipe. Anyway it gets to the end and everything seems to be working. This time I tried with borg 1.1.9 since I am on another machine.

witten commented

2021-07-09 22:00:31 +00:00

I've tried to reproduce this with the exact same version of Debian Postgres, similar borgmatic options, and Borg 1.1.16. No reproduction here. The main difference I have on this machine is Python 3.9.5 instead of 3.7.3.

Are you by chance using a test database rather anything with proprietary data? If so, would it be feasible to include a database dump or even the Borg repository on this ticket? That might allow me to reproduce the problem if it's an issue triggered by the database rather than the environment.

Thanks for your patience.

I've tried to reproduce this with the exact same version of Debian Postgres, similar borgmatic options, and Borg 1.1.16. No reproduction here. The main difference I have on this machine is Python 3.9.5 instead of 3.7.3. Are you by chance using a test database rather anything with proprietary data? If so, would it be feasible to include a database dump or even the Borg repository on this ticket? That might allow me to reproduce the problem if it's an issue triggered by the database rather than the environment. Thanks for your patience.

Ghost commented

2021-07-09 22:43:37 +00:00

It's a test database based on this Django app that uses postgres. Both are running in Docker:

Just tried now: the same issue happens with django-futils even on a newly initialized database.

Just a few notes here:

borgmatic runs outside docker
since the postgres container is postgis/postgis:13-3.1 I installed postgres-client-13, which provides pgdump, etc... used by borgmatic, from the postgres apt repository (https://www.postgresql.org/download/linux/debian/)
this bug happens on 2 different debian 10 virtual machines, different databases and different apps

Maybe you can try installing debian stable on a vm and try everything from there

It's a test database based on this Django app that uses postgres. Both are running in Docker: - https://github.com/frnmst/django-futils - https://software.franco.net.eu.org/frnmst/django-futils - https://docs.franco.net.eu.org/django-futils/ Just tried now: the same issue happens with django-futils even on a newly initialized database. Just a few notes here: - borgmatic runs outside docker - since the postgres container is `postgis/postgis:13-3.1` I installed `postgres-client-13`, which provides pgdump, etc... used by borgmatic, from the postgres apt repository (https://www.postgresql.org/download/linux/debian/) - this bug happens on 2 different debian 10 virtual machines, different databases and different apps Maybe you can try installing debian stable on a vm and try everything from there

witten commented

2021-07-17 15:32:47 +00:00

While I was able to set up a VM with Debian 10, Postgis running in Docker, and the exact versions of borgmatic and borgbackup, I was not able to get django-futils initialized. Would it be possible to provide me with either a database dump of django-futils or a VM image where this problem occurs? Alternatively, a working Docker image of django-futils might help me run it so that I can reproduce the problem. Thanks.

Ghost commented

2021-07-17 16:09:51 +00:00

Thank you for your time. Unfortunately i cannot upload "big" stuff because of slow internet (it would take a day to upload just the docker image). Anyway, I attached a database dump, postgres_dev.tar.gz, obtained from borgmatic extract --archive ...

There is also the borg repository if you need it (password is password)

Thank you for your time. Unfortunately i cannot upload "big" stuff because of slow internet (it would take a day to upload just the docker image). Anyway, I attached a database dump, `postgres_dev.tar.gz`, obtained from `borgmatic extract --archive ...` There is also the borg repository if you need it (password is `password`)

dev.borg.tar.gz

61 KiB

postgres_dev.tar.gz

34 KiB

👍 1

witten commented

2021-07-18 01:07:57 +00:00

Finally got a repro! Thanks for all your help here. Now, to figure out why this hang is occurring..

Ghost commented

2021-07-18 07:44:48 +00:00

Great 👍

witten commented

2021-07-20 23:55:16 +00:00

Spent some more time digging into this. As far as I can tell, pg_restore is exiting "early" for some reason—before borg extract expects it to. That's why you're seeing Broken Pipe when running them directly with a shell pipe. When borgmatic is added to the picture, it appears that it's not passing on that unexpected pg_restore exit to Borg, hence the hang. I could change the borgmatic code to forcibly close the pipe when pg_restore exits, but that'd just result in a Broken Pipe from Borg when doing a borgmatic extract.

So I think in order to solve this, it may be necessary to determine the cause of pg_restore exiting before Borg expects it to.

Spent some more time digging into this. As far as I can tell, `pg_restore` is exiting "early" for some reason—before `borg extract` expects it to. That's why you're seeing `Broken Pipe` when running them directly with a shell pipe. When borgmatic is added to the picture, it appears that it's *not* passing on that unexpected `pg_restore` exit to Borg, hence the hang. I could change the borgmatic code to forcibly close the pipe when `pg_restore` exits, but that'd just result in a `Broken Pipe` from Borg when doing a `borgmatic extract`. So I think in order to solve this, it may be necessary to determine the cause of `pg_restore` exiting before Borg expects it to.

Ghost commented

2021-07-21 10:08:02 +00:00

I can't find anything on the borg and postgres issue trackers. Maybe trying with different postgres versions might pinpoint the problem.

witten commented

2021-07-25 23:31:53 +00:00

Okay, turns out this is an indeed an unfortunate interaction between Borg and pg_restore that borgmatic isn't currently doing anything to make better. I popped into #postgresql on Libera Chat IRC and got some help from RhodiumToad and ilmari. Apparently there are two types of restore data in a Postgres tar dump: A bunch of binary data files, and a restore.sql file at the end that pg_restore ignores but is there in case you want to restore manually / via some other mechanism.

The problem arises in that pg_restore unceremoneously closes the pipe as soon as it receives the binary restore data it needs—before the ignored restore.sql is fully streamed. This makes sense from pg_restore's perspective, but it means that any process streaming to pg_restore sees a broken pipe before the full dump streams across the pipe. Resulting in a hang in borgmatic (until there's a fix), or a mere error when piping directly from borg extract.

Now that I think I know what's going on, I'll see if I can come up with some sort of work-around in borgmatic.

Side note: You probably want to avoid the tar dump format if you can, as it's apparently pretty inefficient and uses extra temporary space on disk during the dumping. And it has this unfortunate interaction, as well. So the custom format is better, or directory if you need parallelism.

Okay, turns out this is an indeed an unfortunate interaction between Borg and `pg_restore` that borgmatic isn't currently doing anything to make better. I popped into `#postgresql` on Libera Chat IRC and got some help from `RhodiumToad` and `ilmari`. Apparently there are two types of restore data in a Postgres tar dump: A bunch of binary data files, and a `restore.sql` file at the end that `pg_restore` ignores but is there in case you want to restore manually / via some other mechanism. The problem arises in that `pg_restore` unceremoneously closes the pipe as soon as it receives the binary restore data it needs—*before* the ignored `restore.sql` is fully streamed. This makes sense from `pg_restore`'s perspective, but it means that any process streaming *to* `pg_restore` sees a broken pipe before the full dump streams across the pipe. Resulting in a hang in borgmatic (until there's a fix), or a mere error when piping directly from `borg extract`. Now that I think I know what's going on, I'll see if I can come up with some sort of work-around in borgmatic. Side note: You probably want to avoid the `tar` dump format if you can, as it's apparently pretty inefficient and uses extra temporary space on disk during the dumping. And it has this unfortunate interaction, as well. So the `custom` format is better, or `directory` if you need parallelism.

👍 1

witten added the

bug

label 2021-07-25 23:37:08 +00:00

witten referenced this issue from a commit

2021-07-26 05:30:45 +00:00

Fix hang when restoring a PostgreSQL "tar" format database dump (#430).

witten referenced this issue from a commit

2021-07-26 05:50:35 +00:00

Fix integration test to hopefully work on Alpine (#430).

witten referenced this issue from a commit

2021-07-26 16:42:23 +00:00