Glob-Expansion (of source directories) does not work on "working directory" #609

New Issue

simon-77 · 2022-10-29T10:18:24Z

simon-77 commented

2022-10-29 10:18:24 +00:00

Hi, it's me again :)

What I'm trying to do and why

I try to use relative paths during backup via working_directory field.

Issue: using relative-path globs in source_directories field are not resolved relative to working_directory

Steps to reproduce

location:
    working_directory: /path/to/work-dir
    source_directories:
        - "*"

Actual behavior

running $ sudo borgmatic create --verbosity 2 --dry-run lists the create command with * expanded to a list of all files/folders in current directory (where this command is executed)

Partial output:

borg create --dry-run <path-to-repo>::{hostname}-user-{now} /root/.borgmatic <a list of files/folders in current directory> --debug --show-rc

Additionally I am curious why /root/.borgmatic is always included in the backup ?

Expected behavior

a list of files in <path/to/work-dir>
not having /root/.borgmatic in the backup included - although, if it has a reason that this directory is always included, I am fine with this fact.

Other notes / implementation ideas

Notes about the location.source_directories field:

I expected - * to work, but this throws an YAML-parser error: did not find expected alphabetic or numeric character

Additionally I tried - ./* which gives the same result as - "*" - resolving relative to current directory instead of working directory.

Environment

borgmatic version: 1.4.4

borgmatic installation method: openSUSE package manger

Borg version: 1.2.2

Python version: 3.10.7

operating system and version:
Operating System: openSUSE Tumbleweed 20221026
Kernel Version: 6.0.3-1-default (64-bit)

Hi, it's me again :) #### What I'm trying to do and why I try to use relative paths during backup via `working_directory` field. Issue: using relative-path globs in `source_directories` field are not resolved relative to `working_directory` #### Steps to reproduce ``` YAML location: working_directory: /path/to/work-dir source_directories: - "*" ``` #### Actual behavior running `$ sudo borgmatic create --verbosity 2 --dry-run` lists the create command with `*` expanded to a list of all files/folders in current directory (where this command is executed) Partial output: ``` borg create --dry-run <path-to-repo>::{hostname}-user-{now} /root/.borgmatic <a list of files/folders in current directory> --debug --show-rc ``` Additionally I am curious why `/root/.borgmatic` is always included in the backup ? #### Expected behavior 1) a list of files in <path/to/work-dir> 2) not having /root/.borgmatic in the backup included - although, if it has a reason that this directory is always included, I am fine with this fact. #### Other notes / implementation ideas Notes about the `location.source_directories` field: I expected `- *` to work, but this throws an YAML-parser error: `did not find expected alphabetic or numeric character` Additionally I tried `- ./*` which gives the same result as `- "*"` - resolving relative to current directory instead of working directory. #### Environment borgmatic version: 1.4.4 borgmatic installation method: openSUSE package manger Borg version: 1.2.2 Python version: 3.10.7 operating system and version: Operating System: openSUSE Tumbleweed 20221026 Kernel Version: 6.0.3-1-default (64-bit)

simon-77 commented

2022-10-29 10:26:17 +00:00

Additional Note:

The Level-2 output of borgmatic create has no info that the relative working_directory path is used.

I gues borgmatic simply does a cd bevore. Maybe include this in Level-2 output.

Additional Note: The Level-2 output of `borgmatic create` has no info that the relative `working_directory` path is used. I gues borgmatic simply does a `cd` bevore. Maybe include this in Level-2 output.

simon-77 commented

2022-10-29 14:11:49 +00:00

I now have a work-around:

location:
    working_directory: /path/to/work-dir
    source_directories:
        - .

includes all directories and files of /path/to/work-dir in a relative manner.

My initial goal was to only include directories (and files) that do not start with a dot. For this I found the following glob pattern to work: [!.]*
This is how I came up with the simplified example in the original post.

I just noticed: for not having .* directories/files in the repo I could also simply use the exclude patterns.

I now have a work-around: ``` YAML location: working_directory: /path/to/work-dir source_directories: - . ``` includes all directories and files of `/path/to/work-dir` in a relative manner. My initial goal was to only include directories (and files) that do not start with a dot. For this I found the following glob pattern to work: `[!.]*` This is how I came up with the simplified example in the original post. I just noticed: for not having `.*` directories/files in the repo I could also simply use the exclude patterns.

witten commented

2023-03-27 03:18:39 +00:00

I apologize for the delay in getting to this. Supporting globs relative to the configured working directory seems like a totally reasonable feature!

I expected - * to work, but this throws an YAML-parser error: did not find expected alphabetic or numeric character

Yeah, this is a YAML limitation. You need to quote the asterisk, as you discovered.

The Level-2 output of borgmatic create has no info that the relative working_directory path is used.

Agreed.. This would be a good thing to log!

Additionally I am curious why /root/.borgmatic is always included in the backup ?

The short answer is that ~/.borgmatic is included to support backing up database dumps. More info is here: https://torsion.org/borgmatic/docs/how-to/backup-your-databases/

I apologize for the delay in getting to this. Supporting globs relative to the configured working directory seems like a totally reasonable feature! > I expected - * to work, but this throws an YAML-parser error: did not find expected alphabetic or numeric character Yeah, this is a YAML limitation. You need to quote the asterisk, as you discovered. > The Level-2 output of borgmatic create has no info that the relative working_directory path is used. Agreed.. This would be a good thing to log! > Additionally I am curious why /root/.borgmatic is always included in the backup ? The short answer is that `~/.borgmatic` is included to support backing up database dumps. More info is here: https://torsion.org/borgmatic/docs/how-to/backup-your-databases/

witten added the

good first issue

label 2023-03-27 03:18:46 +00:00

Sign in to join this conversation.