Database Dumps are not working #578

New issue

Closed

opened 2022-09-01 11:06:51 +00:00 by Bleala · 3 comments

Bleala commented 2022-09-01 11:06:51 +00:00

Copy link

Hello there!

I'm trying to do database dumps with borgmatic, but it is not working as expected.

This would be my database hook:

  mysql_databases:
    - name: database
      hostname: 192.xxx.xxx.xxx
      port: 3306
      username: borgmatic
      password: mypassword

This is the log (--verbosity 2) and as you can it stops at Processing files...

ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Creating archive
ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Calling mysql_databases hook function remove_database_dumps
ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Removing MySQL database dumps
ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Calling mysql_databases hook function dump_databases
ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Dumping MySQL databases
ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Dumping MySQL database mydatabase to /root/.borgmatic/mysql_databases/192.xxx.xxx.xxx/mydatabase
mysqldump --add-drop-database --host 192.xxx.xxx.xxx --port 3306 --protocol tcp --user borgmatic --databases mydatabase > /root/.borgmatic/mysql_databases/192.xxx.xxx.xxx/mydatabase
borg create --compression lz4 --one-file-system --read-special --remote-path /usr/local/bin/borg --stats --debug --show-rc ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo::databases-{now:%Y-%m-%dT%H:%M:%S.%f} /root/.borgmatic
using builtin fallback logging configuration
33 self tests completed in 0.20 seconds
SSH command line: ['ssh', '-i', '/root/.ssh/id_ed25519', 'borgmatic@192.xxx.xxx.xxx', '/usr/local/bin/borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: 33 self tests completed in 0.19 seconds
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/myborgrepo'
Remote: Resolved repository path to '/myborgrepo'
Remote: Verified integrity of /myborgrepo/index.5
TAM-verified manifest
security: read previous location 'ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo'
security: read manifest timestamp '2022-09-01T10:43:22.335281'
security: determined newest manifest timestamp as 2022-09-01T10:43:22.335281
security: repository checks ok, allowing access
Creating archive at "ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo::databases-2022-09-01T10:43:54.590869"
Verified integrity of /root/.cache/borg/3a40c4a197b4f5d95ba9cdaaa709f4879a27168dc8329d854d78b1c97f4c88c9/chunks
Reading files cache ...
security: read previous location 'ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo'
security: read manifest timestamp '2022-09-01T10:43:22.335281'
security: determined newest manifest timestamp as 2022-09-01T10:43:22.335281
security: repository checks ok, allowing access
Processing files ...

When i cancel the process (Ctrl+c) I'm getting the following error:

  File "/usr/local/bin/borgmatic", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 1037, in main
    summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments))
  File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 930, in collect_configuration_run_summary_logs
    results = list(run_configuration(config_filename, config, arguments))
  File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 117, in run_configuration
    yield from run_actions(
  File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 380, in run_actions
    json_output = borg_create.create_archive(
  File "/usr/local/lib/python3.10/site-packages/borgmatic/borg/create.py", line 322, in create_archive
    return execute_command_with_processes(
  File "/usr/local/lib/python3.10/site-packages/borgmatic/execute.py", line 275, in execute_command_with_processes
    captured_outputs = log_outputs(
  File "/usr/local/lib/python3.10/site-packages/borgmatic/execute.py", line 95, in log_outputs
    line = ready_buffer.readline().rstrip().decode()
KeyboardInterrupt

Same issue with Postgres databases!

Does anyone has an idea why this is not working?

Environment

borgmatic version: 1.7.1

borgmatic installation method: Official Docker Container

Borg version: 1.2.2

Python version: 3.10.6

Database version (if applicable): MariaDB 10.8.3

operating system and version: Docker Contanier

Hello there! I'm trying to do database dumps with borgmatic, but it is not working as expected. This would be my database hook: ``` mysql_databases: - name: database hostname: 192.xxx.xxx.xxx port: 3306 username: borgmatic password: mypassword ``` This is the log (--verbosity 2) and as you can it stops at `Processing files...` ``` ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Creating archive ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Calling mysql_databases hook function remove_database_dumps ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Removing MySQL database dumps ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Calling mysql_databases hook function dump_databases ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Dumping MySQL databases ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo: Dumping MySQL database mydatabase to /root/.borgmatic/mysql_databases/192.xxx.xxx.xxx/mydatabase mysqldump --add-drop-database --host 192.xxx.xxx.xxx --port 3306 --protocol tcp --user borgmatic --databases mydatabase > /root/.borgmatic/mysql_databases/192.xxx.xxx.xxx/mydatabase borg create --compression lz4 --one-file-system --read-special --remote-path /usr/local/bin/borg --stats --debug --show-rc ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo::databases-{now:%Y-%m-%dT%H:%M:%S.%f} /root/.borgmatic using builtin fallback logging configuration 33 self tests completed in 0.20 seconds SSH command line: ['ssh', '-i', '/root/.ssh/id_ed25519', 'borgmatic@192.xxx.xxx.xxx', '/usr/local/bin/borg', 'serve', '--debug'] Remote: using builtin fallback logging configuration Remote: 33 self tests completed in 0.19 seconds Remote: using builtin fallback logging configuration Remote: Initialized logging system for JSON-based protocol Remote: Resolving repository path b'/myborgrepo' Remote: Resolved repository path to '/myborgrepo' Remote: Verified integrity of /myborgrepo/index.5 TAM-verified manifest security: read previous location 'ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo' security: read manifest timestamp '2022-09-01T10:43:22.335281' security: determined newest manifest timestamp as 2022-09-01T10:43:22.335281 security: repository checks ok, allowing access Creating archive at "ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo::databases-2022-09-01T10:43:54.590869" Verified integrity of /root/.cache/borg/3a40c4a197b4f5d95ba9cdaaa709f4879a27168dc8329d854d78b1c97f4c88c9/chunks Reading files cache ... security: read previous location 'ssh://borgmatic@192.xxx.xxx.xxx/myborgrepo' security: read manifest timestamp '2022-09-01T10:43:22.335281' security: determined newest manifest timestamp as 2022-09-01T10:43:22.335281 security: repository checks ok, allowing access Processing files ... ``` When i cancel the process (Ctrl+c) I'm getting the following error: ``` File "/usr/local/bin/borgmatic", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 1037, in main summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments)) File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 930, in collect_configuration_run_summary_logs results = list(run_configuration(config_filename, config, arguments)) File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 117, in run_configuration yield from run_actions( File "/usr/local/lib/python3.10/site-packages/borgmatic/commands/borgmatic.py", line 380, in run_actions json_output = borg_create.create_archive( File "/usr/local/lib/python3.10/site-packages/borgmatic/borg/create.py", line 322, in create_archive return execute_command_with_processes( File "/usr/local/lib/python3.10/site-packages/borgmatic/execute.py", line 275, in execute_command_with_processes captured_outputs = log_outputs( File "/usr/local/lib/python3.10/site-packages/borgmatic/execute.py", line 95, in log_outputs line = ready_buffer.readline().rstrip().decode() KeyboardInterrupt ``` Same issue with Postgres databases! Does anyone has an idea why this is not working? #### Environment **borgmatic version:** 1.7.1 **borgmatic installation method:** Official Docker Container **Borg version:** 1.2.2 **Python version:** 3.10.6 **Database version (if applicable):** MariaDB 10.8.3 **operating system and version:** Docker Contanier

witten commented 2022-09-01 15:31:52 +00:00

Owner

Copy link

Thanks for taking the time to file this! It looks like Borg is hanging on reading special files (e.g., stale named pipes) due to the --read-special parameter, which borgmatic uses so database dumps can stream directly to Borg without hitting disk. Given that the only files you appear to be backing up here are database dumps, I'm guessing the problem is in the ~/.borgmatic directory where the dumps get streamed. So my recommendation is to delete the whole ~/.borgmatic directory and then try the backup again. Generally you shouldn't have to do that as borgmatic cleans up after itself, but there may be a few edge cases where this doesn't work properly.

Let me know what you find out!

Thanks for taking the time to file this! It *looks* like Borg is hanging on reading special files (e.g., stale named pipes) due to the `--read-special` parameter, which borgmatic uses so database dumps can stream directly to Borg without hitting disk. Given that the only files you appear to be backing up here are database dumps, I'm guessing the problem is in the `~/.borgmatic` directory where the dumps get streamed. So my recommendation is to delete the whole `~/.borgmatic` directory and then try the backup again. Generally you *shouldn't* have to do that as borgmatic cleans up after itself, but there may be a few [edge cases](https://projects.torsion.org/borgmatic-collective/borgmatic/issues/360) where this doesn't work properly. Let me know what you find out!

witten added the

question / support

label 2022-09-01 15:32:01 +00:00

Bleala commented 2022-09-03 19:09:20 +00:00

Author

Copy link

@witten Thanks that worked!

Yeah I was only backing up database dumps with this config.
Do I have to worry about this again, after deleting the ~/.borgmatic directory, or is there another way to control this behaviour?

@witten Thanks that worked! Yeah I was only backing up database dumps with this config. Do I have to worry about this again, after deleting the `~/.borgmatic` directory, or is there another way to control this behaviour?

Kritzefitz referenced this issue 2022-09-08 13:50:27 +00:00

borgmatic hangs when location.patterns includes /root/.borgmatic and database hooks are active #582

witten commented 2022-09-08 15:37:09 +00:00

Owner

Copy link

Glad to hear it worked! You generally shouldn't have to delete ~/.borgmatic, as borgmatic cleans up after itself except in a few edge cases. Those edge cases should be covered by #360.

Glad to hear it worked! You generally shouldn't have to delete `~/.borgmatic`, as borgmatic cleans up after itself except in a few edge cases. Those edge cases should be covered by #360.

witten closed this issue 2022-09-08 15:37:09 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

parallel-borgmatic

command-hook-steps

2.1.7

2.1.6

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.9.14

1.9.13

1.9.12

1.9.11

1.9.10

1.9.9

1.9.8

1.9.7

1.9.6

1.9.5

1.9.4

1.9.3

1.9.2

1.9.1

1.9.0

1.8.14

1.8.13

1.8.12

1.8.11

1.8.10

1.8.9

1.8.8

1.8.7

1.8.6

1.8.5

1.8.4

1.8.3

1.8.2

1.8.1

1.8.0

1.7.15

1.7.14

1.7.13

1.7.12

1.7.11

1.7.10

1.7.9

1.7.8

1.7.7

1.7.6

1.7.5

1.7.4

1.7.3

1.7.2

1.7.1

1.7.0

1.6.6

1.6.5

1.6.4

1.6.3

1.6.2

1.6.1

1.6.0

1.5.24

1.5.23

1.5.22

1.5.21

1.5.20

1.5.19

1.5.18

1.5.17

1.5.16

1.5.15

1.5.14

1.5.13

1.5.12

1.5.11

1.5.10

1.5.9

1.5.8

1.5.7

1.5.7.dev0

1.5.6

1.5.5

1.5.4

1.5.3

1.5.2

1.5.1

1.5.0

1.4.22

1.4.21

1.4.20

1.4.19

1.4.18

1.4.17

1.4.16

1.4.15

1.4.14

1.4.13

1.4.12

1.4.11

1.4.10

1.4.9

1.4.8

1.4.7

1.4.6

1.4.5

1.4.4

1.4.3

1.4.2

1.4.1

1.4.0

1.3.26

1.3.25

1.3.24

1.3.23

1.3.22

1.3.21

1.3.20

1.3.19

1.3.18

1.3.17

1.3.16

1.3.15

1.3.14

1.3.13

1.3.12

1.3.11

1.3.10

1.3.9

1.3.8

1.3.7

1.3.6

1.3.5

1.3.4

1.3.3

1.3.2

1.3.1

1.3.0

1.2.18

1.2.17

1.2.16

1.2.15

1.2.14

1.2.13

1.2.12

1.2.11

1.2.10

1.2.9

1.2.8

1.2.7

1.2.6

1.2.5

1.2.4

1.2.3

1.2.2

1.2.1

1.2.0

1.1.15

1.1.14

1.1.13

1.1.12

1.1.11

1.1.10

1.1.9

1.1.7

1.1.6

1.1.5

1.1.4

1.1.3

1.1.2

1.1.1

1.1.0

1.0.3

1.0.2

1.0.1

1.0.0

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

Labels

Clear labels   

blocked

  

breaking

  

bug

  

data loss

  

design finalized

  

good first issue

  

new feature area

  

question / support

  

security

  

waiting for response

No labels

blocked

breaking

bug

data loss

design finalized

good first issue

new feature area

question / support

security

waiting for response

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

borgmatic-collective/borgmatic#578

No description provided.