borgmatic create --json errors when Borg includes non-JSON warnings in JSON output #847

New Issue

witten · 2024-04-05T18:51:57Z

witten commented

2024-04-05 18:51:57 +00:00

What I'm trying to do and why

I'm trying to run borgmatic create --json so that I can consume its output programmatically as part of prototyping another ticket. This generally works—except when I pass Borg a non-existent source directory, in which case Borg includes a non-JSON warning in its output which borgmatic then chokes on as it expects JSON. (borgmatic parses the JSON instead of just passing it through to stdout/logging because borgmatic provides the service of combining JSON output from multiple Borg runs.)

Steps to reproduce

Create a working borgmatic configuration and add a non-existent file path in source_directories.
Initialize a corresponding Borg repository.
Run borgmatic create -c yourconfig.yaml --verbosity 2 --json

Actual behavior

Relevant borgmatic output:

CRITICAL:None:backup: Error running actions for repository
CRITICAL:None:Expecting value: line 1 column 1 (char 0)

Under the hood, Borg outputs something like the following for borgmatic to choke on:

/non/existent/path: stat: [Errno 2] No such file or directory: '/non/existent/path'
{
    "archive": {
        "command_line": [
            "/usr/bin/borg",
            "create",
            "--keep-exclude-tags",
...
}

Expected behavior

Ideally Borg wouldn't output non-JSON output with --json on, but lacking that, maybe borgmatic could ignore the non-JSON portion.

Other notes / implementation ideas

One way to ignore the non-JSON portion would be scan for the first { and discard anything prior. May not be foolproof though.

borgmatic version

main

borgmatic installation method

pipx install --editable

Borg version

1.2.7

Python version

3.11.8

Database version (if applicable)

No response

Operating system and version

Manjaro stable

### What I'm trying to do and why I'm trying to run `borgmatic create --json` so that I can consume its output programmatically as part of prototyping another ticket. This generally works—except when I pass Borg a non-existent source directory, in which case Borg includes a non-JSON warning in its output which borgmatic then chokes on as it expects JSON. (borgmatic parses the JSON instead of just passing it through to stdout/logging because borgmatic provides the service of *combining* JSON output from multiple Borg runs.) ### Steps to reproduce 1. Create a working borgmatic configuration and add a non-existent file path in `source_directories`. 2. Initialize a corresponding Borg repository. 3. Run `borgmatic create -c yourconfig.yaml --verbosity 2 --json` ### Actual behavior Relevant borgmatic output: ``` CRITICAL:None:backup: Error running actions for repository CRITICAL:None:Expecting value: line 1 column 1 (char 0) ``` Under the hood, Borg outputs something like the following for borgmatic to choke on: ``` /non/existent/path: stat: [Errno 2] No such file or directory: '/non/existent/path' { "archive": { "command_line": [ "/usr/bin/borg", "create", "--keep-exclude-tags", ... } ``` ### Expected behavior Ideally Borg wouldn't output non-JSON output with `--json` on, but lacking that, maybe borgmatic could ignore the non-JSON portion. ### Other notes / implementation ideas One way to ignore the non-JSON portion would be scan for the first `{` and discard anything prior. May not be foolproof though. ### borgmatic version main ### borgmatic installation method pipx install --editable ### Borg version 1.2.7 ### Python version 3.11.8 ### Database version (if applicable) _No response_ ### Operating system and version Manjaro stable

witten added the

bug

label 2024-04-05 18:51:57 +00:00

witten referenced this issue from a commit

2024-04-05 19:24:03 +00:00

Fix "--json" error when Borg includes non-JSON warnings in JSON output (#847).

witten commented

2024-04-05 19:24:52 +00:00

I went with the solution of scanning for the first { to start parsing JSON, but only if it's at the start of a line. Should be a little more robust that way.

I went with the solution of scanning for the first `{` to start parsing JSON, but only if it's at the start of a line. Should be a little more robust that way.

witten closed this issue

2024-04-05 19:24:52 +00:00

witten commented

2024-04-16 17:45:24 +00:00

Released in 1.8.10!

Sign in to join this conversation.