[Bug] source_directories_must_exist does not expand globs and tildes #682

New issue

Closed

opened 2023-04-20 00:10:16 +00:00 by holocronweaver · 2 comments

holocronweaver commented

2023-04-20 00:10:16 +00:00

Contributor

What I'm trying to do and why

I am trying to add source_directories_must_exist: true to my existing Borgmatic config so I can make use of this feature. However I any directories with globs or tildes are not being expanded and always fail the existence check.

Steps to reproduce (if a bug)

Add source directories with globs and tildes while enabling source_directories_must_exist.

Example config file:

location:
    source_directories:
        - ~/downloads
        - /home/me/.bash*

    repositories:
        - /home/me/myrepo

    # If true, then source directories must exist, otherwise an
    # error is raised. Defaults to false.
    source_directories_must_exist: true

Removing source_directories_must_exist resolves the issue.

Actual behavior (if a bug)

borgmatic -c backup.yaml
...
summary:
backup.yaml: An error occurred
/home/me/myrepo: Error running actions for repository
Source directories do not exist: ~/downloads, /home/me/.bash*

Expected behavior (if a bug)

A successful backup of source directories containing globs and tildes, as was the case before adding source_directories_must_exist.

Other notes / implementation ideas

Seems no expansion is being done during the existence check.

def check_all_source_directories_exist(source_directories):
    '''
    Given a sequence of source directories, check that they all exist. If any do not, raise an
    exception.
    '''
    missing_directories = [
        source_directory
        for source_directory in source_directories
        if not os.path.exists(source_directory)
    ]
    if missing_directories:
        raise ValueError(f"Source directories do not exist: {', '.join(missing_directories)}")

so should be a simple matter to fix:

if not os.path.exists(expand_directory(source_directory))

Happy to open a pull request to fix and add a unit test.

Environment

borgmatic version: 1.7.12

Use sudo borgmatic --version or sudo pip show borgmatic | grep ^Version

borgmatic installation method: pip

Borg version: 1.2.1

Use sudo borg --version

Python version: 3.10.6

Use python3 --version

Database version (if applicable): n/a

Use psql --version or mysql --version on client and server.

operating system and version: Ubuntu 22.04.2

#### What I'm trying to do and why I am trying to add `source_directories_must_exist: true` to my existing Borgmatic config so I can make use of this feature. However I any directories with globs or tildes are not being expanded and always fail the existence check. #### Steps to reproduce (if a bug) Add source directories with globs and tildes while enabling `source_directories_must_exist`. Example config file: ``` location: source_directories: - ~/downloads - /home/me/.bash* repositories: - /home/me/myrepo # If true, then source directories must exist, otherwise an # error is raised. Defaults to false. source_directories_must_exist: true ``` Removing `source_directories_must_exist` resolves the issue. #### Actual behavior (if a bug) ``` borgmatic -c backup.yaml ... summary: backup.yaml: An error occurred /home/me/myrepo: Error running actions for repository Source directories do not exist: ~/downloads, /home/me/.bash* ``` #### Expected behavior (if a bug) A successful backup of source directories containing globs and tildes, as was the case before adding `source_directories_must_exist`. #### Other notes / implementation ideas Seems [no expansion is being done during the existence check](https://github.com/borgmatic-collective/borgmatic/blob/9f5769f87be3e75a3b86ee1a6f31a6754ec2ed4e/borgmatic/borg/create.py#L317). ``` def check_all_source_directories_exist(source_directories): ''' Given a sequence of source directories, check that they all exist. If any do not, raise an exception. ''' missing_directories = [ source_directory for source_directory in source_directories if not os.path.exists(source_directory) ] if missing_directories: raise ValueError(f"Source directories do not exist: {', '.join(missing_directories)}") ``` so should be a simple matter to fix: ``` if not os.path.exists(expand_directory(source_directory)) ``` Happy to open a pull request to fix and add a unit test. #### Environment **borgmatic version:** 1.7.12 Use `sudo borgmatic --version` or `sudo pip show borgmatic | grep ^Version` **borgmatic installation method:** pip **Borg version:** 1.2.1 Use `sudo borg --version` **Python version:** 3.10.6 Use `python3 --version` **Database version (if applicable):** n/a Use `psql --version` or `mysql --version` on client and server. **operating system and version:** Ubuntu 22.04.2

holocronweaver referenced this issue from a pull request that will close it,

2023-04-20 01:56:58 +00:00

Expand source directories when checking for existence. #683