borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 54 Pull Requests 1 Actions Packages Releases 121 Activity

TypeError traceback with borgmatic create --json and database configured #563

New Issue
Closed
opened 2022-07-19 16:25:26 +00:00 by witten · 4 comments
witten commented 2022-07-19 16:25:26 +00:00
Owner
Copy Link

What I'm trying to do and why

Use the borgmatic create --json flag when a database happens to be configured for dumping, so I can use the JSON output from the backup in a downstream tool.

Steps to reproduce

  • Configure a database hook (MySQL, PostgreSQL, etc).
  • borgmatic create -c test.yaml -v 2 --json

Actual behavior

Ensuring legacy configuration is upgraded
test.yaml: No commands to run for pre-everything hook
borg --version --debug --show-rc
test.yaml: No commands to run for pre-backup hook
test.borg: Creating archive
test.borg: Calling postgresql_databases hook function remove_database_dumps
test.borg: Removing PostgreSQL database dumps
test.borg: Calling postgresql_databases hook function dump_databases
test.borg: Dumping PostgreSQL databases
test.borg: Dumping PostgreSQL database all to /root/.borgmatic/postgresql_databases/localhost/all
pg_dumpall --no-password --clean --if-exists --host localhost --username test > /root/.borgmatic/postgresql_databases/localhost/all
borg create --one-file-system --read-special --json test.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /root/.borgmatic /root/tmp
Traceback (most recent call last):
  File "/usr/bin/borgmatic", line 33, in <module>
    sys.exit(load_entry_point('borgmatic', 'console_scripts', 'borgmatic')())
  File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 920, in main
    summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments))
  File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 813, in collect_configuration_run_summary_logs
    results = list(run_configuration(config_filename, config, arguments))
  File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 114, in run_configuration
    yield from run_actions(
  File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 350, in run_actions
    json_output = borg_create.create_archive(
  File "/home/witten/code/borgmatic/borgmatic/borg/create.py", line 323, in create_archive
    return execute_command_with_processes(
  File "/home/witten/code/borgmatic/borgmatic/execute.py", line 262, in execute_command_with_processes
    log_outputs(
  File "/home/witten/code/borgmatic/borgmatic/execute.py", line 102, in log_outputs
    logger.log(output_log_level, line)
  File "/usr/lib/python3.10/logging/__init__.py", line 1543, in log
    raise TypeError("level must be an integer")
TypeError: level must be an integer

Expected behavior

No traceback. Logging is suppressed at the console and only JSON output is produced.

Other notes / implementation ideas

execute_command_with_processes() probably needs a special case for when output_log_level is None (which is set when --json is used) similar to execute_command()'s existing case.

Original PR: https://github.com/borgmatic-collective/borgmatic/pull/47

Environment

borgmatic version: 1.6.5 / master

borgmatic installation method: pip

Borg version: 1.2.1

Python version: 3.10.5

Database version (if applicable): PostgreSQL 14.4, but occurs on MySQL as well.

operating system and version: Manjaro stable

#### What I'm trying to do and why Use the `borgmatic create --json` flag when a database happens to be configured for dumping, so I can use the JSON output from the backup in a downstream tool. #### Steps to reproduce * Configure a database hook (MySQL, PostgreSQL, etc). * `borgmatic create -c test.yaml -v 2 --json` #### Actual behavior ```python Ensuring legacy configuration is upgraded test.yaml: No commands to run for pre-everything hook borg --version --debug --show-rc test.yaml: No commands to run for pre-backup hook test.borg: Creating archive test.borg: Calling postgresql_databases hook function remove_database_dumps test.borg: Removing PostgreSQL database dumps test.borg: Calling postgresql_databases hook function dump_databases test.borg: Dumping PostgreSQL databases test.borg: Dumping PostgreSQL database all to /root/.borgmatic/postgresql_databases/localhost/all pg_dumpall --no-password --clean --if-exists --host localhost --username test > /root/.borgmatic/postgresql_databases/localhost/all borg create --one-file-system --read-special --json test.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} /root/.borgmatic /root/tmp Traceback (most recent call last): File "/usr/bin/borgmatic", line 33, in <module> sys.exit(load_entry_point('borgmatic', 'console_scripts', 'borgmatic')()) File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 920, in main summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments)) File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 813, in collect_configuration_run_summary_logs results = list(run_configuration(config_filename, config, arguments)) File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 114, in run_configuration yield from run_actions( File "/home/witten/code/borgmatic/borgmatic/commands/borgmatic.py", line 350, in run_actions json_output = borg_create.create_archive( File "/home/witten/code/borgmatic/borgmatic/borg/create.py", line 323, in create_archive return execute_command_with_processes( File "/home/witten/code/borgmatic/borgmatic/execute.py", line 262, in execute_command_with_processes log_outputs( File "/home/witten/code/borgmatic/borgmatic/execute.py", line 102, in log_outputs logger.log(output_log_level, line) File "/usr/lib/python3.10/logging/__init__.py", line 1543, in log raise TypeError("level must be an integer") TypeError: level must be an integer ``` #### Expected behavior No traceback. Logging is suppressed at the console and only JSON output is produced. #### Other notes / implementation ideas `execute_command_with_processes()` probably needs a special case for when `output_log_level` is None (which is set when `--json` is used) similar to `execute_command()`'s existing case. Original PR: https://github.com/borgmatic-collective/borgmatic/pull/47 #### Environment **borgmatic version:** 1.6.5 / master **borgmatic installation method:** pip **Borg version:** 1.2.1 **Python version:** 3.10.5 **Database version (if applicable):** PostgreSQL 14.4, but occurs on MySQL as well. **operating system and version:** Manjaro stable
witten added the
bug
 label 2022-07-19 16:25:32 +00:00
witten commented 2022-07-19 17:30:45 +00:00
Author
Owner
Copy Link

This has been fixed in master. With the fix in place, you can get JSON output even when a database is configured for dumping. This will be part of the next release!

This has been fixed in master. With the fix in place, you can get JSON output even when a database is configured for dumping. This will be part of the next release!
deleb commented 2022-07-19 17:33:41 +00:00
Copy Link

Wow thanks you very much! What speed !

Wow thanks you very much! What speed !
👍 1
witten commented 2022-07-20 03:36:00 +00:00
Author
Owner
Copy Link

This has been released in borgmatic 1.6.6!

This has been released in borgmatic 1.6.6!
deleb commented 2022-07-20 07:18:45 +00:00
Copy Link

It works perfectly thank you again 👍

It works perfectly thank you again 👍
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#563
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?