borgmatic-collective/borgmatic
9
71
Fork 49
Code Issues 55 Pull Requests 4 Actions 1 Packages Releases 157 Activity

Patterns broken with v1.9.7 #979

New Issue
Closed
opened 2025-01-25 10:05:09 +00:00 by flortsch · 2 comments
flortsch commented 2025-01-25 10:05:09 +00:00
Copy Link

What I'm trying to do and why

Starting with 1.9.7, borgmatic seems to append 'sh:' to my pattern entries. This results in "No such file or directory" errors when running borg.

Steps to reproduce

patterns: [
  'R /etc',
  'R /home',
  'R /root',
  'R /var',
  '- /**/.cache',
  '- /**/[cC]ache',
  '- /**/+([a-zA-Z]) [cC]ache',
  '- /**/*.tmp',
  '- /**/tmp',
  '- /**/temp',
  '- /**/node_modules',
  '- /**/.npm',
  '- /**/pnpm/store',
  '- /**/.gradle',
  '+ /**/containers/storage/volumes',
  '- /**/containers',
  '- /var/lib/borg',
]

repositories:
  - path: /mnt/seagate-backup/borg/lafl-laptop.borg

encryption_passphrase: "XXXXXXXXXXXXXX"

keep_daily: 7
keep_weekly: 4
keep_monthly: 12
keep_yearly: 3

before_actions:
  - mount /dev/disk/by-id/usb-Seagate_Portable_NACMRD8B-0:0-part1 /mnt/seagate-backup || exit 75

Actual behavior

/etc/borgmatic.d/lafl-laptop.yaml: No commands to run for pre-everything hook
BORG_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc
/etc/borgmatic.d/lafl-laptop.yaml: Borg 1.4.0
/mnt/seagate-backup/borg/lafl-laptop.borg: Running actions for repository
/etc/borgmatic.d/lafl-laptop.yaml: Running command for pre-actions hook
mount /dev/disk/by-id/usb-Seagate_Portable_NACMRD8B-0:0-part1 /mnt/seagate-backup || exit 75
/etc/borgmatic.d/lafl-laptop.yaml: No commands to run for pre-backup hook
/mnt/seagate-backup/borg/lafl-laptop.borg: Creating archive
/mnt/seagate-backup/borg/lafl-laptop.borg: Using runtime directory /tmp/borgmatic-y3ewt8e8/borgmatic
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling bootstrap hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling btrfs hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling lvm hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling mariadb hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Removing MariaDB data source dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling mongodb hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Removing MongoDB data source dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling mysql hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Removing MySQL data source dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling postgresql hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Removing PostgreSQL data source dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling sqlite hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Removing SQLite data source dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling zfs hook function remove_data_source_dumps
/mnt/seagate-backup/borg/lafl-laptop.borg: Calling bootstrap hook function dump_data_sources
/mnt/seagate-backup/borg/lafl-laptop.borg: Writing patterns to /tmp/borgmatic-y3ewt8e8/borgmatic/tmpd8ps5op0:
R sh:/etc
R sh:/home
R sh:/root
R sh:/var
- sh:/**/.cache
- sh:/**/[cC]ache
- sh:/**/+([a-zA-Z]) [cC]ache
- sh:/**/*.tmp
- sh:/**/tmp
- sh:/**/temp
- sh:/**/node_modules
- sh:/**/.npm
- sh:/**/pnpm/store
- sh:/**/.gradle
+ sh:/**/containers/storage/volumes
- sh:/**/containers
- sh:/var/lib/borg
R /tmp/borgmatic-y3ewt8e8/./borgmatic/bootstrap
BORG_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-y3ewt8e8/borgmatic/tmpd8ps5op0 --debug --show-rc --progress /mnt/seagate-backup/borg/lafl-laptop.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} > 
using builtin fallback logging configuration
33 self tests completed in 0.05 seconds
Verified integrity of /mnt/seagate-backup/borg/lafl-laptop.borg/index.4243
TAM-verified manifest
security: read previous location '/mnt/seagate-backup/borg/lafl-laptop.borg'
security: read manifest timestamp '2025-01-25T09:44:28.077485'
security: determined newest manifest timestamp as 2025-01-25T09:44:28.077485
security: repository checks ok, allowing access
Creating archive at "/mnt/seagate-backup/borg/lafl-laptop.borg::lafl-laptop-2025-01-25T10:52:40.240662"
Verified integrity of /root/.cache/borg/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848/chunks
Reading files cache ...
Verified integrity of /root/.cache/borg/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848/files
security: read previous location '/mnt/seagate-backup/borg/lafl-laptop.borg'
security: read manifest timestamp '2025-01-25T09:44:28.077485'
security: determined newest manifest timestamp as 2025-01-25T09:44:28.077485
security: repository checks ok, allowing access
Processing files ...
sh:/etc: stat: [Errno 2] No such file or directory: 'sh:/etc'
sh:/home: stat: [Errno 2] No such file or directory: 'sh:/home'
sh:/root: stat: [Errno 2] No such file or directory: 'sh:/root'
sh:/var: stat: [Errno 2] No such file or directory: 'sh:/var'
sh:/etc: stat: [Errno 2] No such file or directory: 'sh:/etc'
sh:/home: stat: [Errno 2] No such file or directory: 'sh:/home'
sh:/root: stat: [Errno 2] No such file or directory: 'sh:/root'
sh:/var: stat: [Errno 2] No such file or directory: 'sh:/var'
Cleaned up 0 uncommitted segment files (== everything after segment 4243).                                                                                                                                                                                                                                                     
Verified integrity of /mnt/seagate-backup/borg/lafl-laptop.borg/hints.4243
check_free_space: required bytes 1035723608, free bytes 3791046074368
security: saving state for dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848 to /root/.config/borg/security/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848
security: current location   /mnt/seagate-backup/borg/lafl-laptop.borg
security: key type           3
security: manifest timestamp 2025-01-25T09:52:45.530544
rcs: [107]                                                                                                                                                                                                                                                                                                                     
terminating with warning status, rc 107

Expected behavior

Borgmatic should not append 'sh:' to the pattern entries.

Other notes / implementation ideas

No response

borgmatic version

1.9.7

borgmatic installation method

pacman

Borg version

1.4.0

Python version

3.13.1

Database version (if applicable)

No response

Operating system and version

Arch Linux

### What I'm trying to do and why Starting with 1.9.7, borgmatic seems to append 'sh:' to my pattern entries. This results in "No such file or directory" errors when running borg. ### Steps to reproduce ``` patterns: [ 'R /etc', 'R /home', 'R /root', 'R /var', '- /**/.cache', '- /**/[cC]ache', '- /**/+([a-zA-Z]) [cC]ache', '- /**/*.tmp', '- /**/tmp', '- /**/temp', '- /**/node_modules', '- /**/.npm', '- /**/pnpm/store', '- /**/.gradle', '+ /**/containers/storage/volumes', '- /**/containers', '- /var/lib/borg', ] repositories: - path: /mnt/seagate-backup/borg/lafl-laptop.borg encryption_passphrase: "XXXXXXXXXXXXXX" keep_daily: 7 keep_weekly: 4 keep_monthly: 12 keep_yearly: 3 before_actions: - mount /dev/disk/by-id/usb-Seagate_Portable_NACMRD8B-0:0-part1 /mnt/seagate-backup || exit 75 ``` ### Actual behavior ``` /etc/borgmatic.d/lafl-laptop.yaml: No commands to run for pre-everything hook BORG_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc /etc/borgmatic.d/lafl-laptop.yaml: Borg 1.4.0 /mnt/seagate-backup/borg/lafl-laptop.borg: Running actions for repository /etc/borgmatic.d/lafl-laptop.yaml: Running command for pre-actions hook mount /dev/disk/by-id/usb-Seagate_Portable_NACMRD8B-0:0-part1 /mnt/seagate-backup || exit 75 /etc/borgmatic.d/lafl-laptop.yaml: No commands to run for pre-backup hook /mnt/seagate-backup/borg/lafl-laptop.borg: Creating archive /mnt/seagate-backup/borg/lafl-laptop.borg: Using runtime directory /tmp/borgmatic-y3ewt8e8/borgmatic /mnt/seagate-backup/borg/lafl-laptop.borg: Calling bootstrap hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap /mnt/seagate-backup/borg/lafl-laptop.borg: Calling btrfs hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling lvm hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling mariadb hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Removing MariaDB data source dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling mongodb hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Removing MongoDB data source dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling mysql hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Removing MySQL data source dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling postgresql hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Removing PostgreSQL data source dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling sqlite hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Removing SQLite data source dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling zfs hook function remove_data_source_dumps /mnt/seagate-backup/borg/lafl-laptop.borg: Calling bootstrap hook function dump_data_sources /mnt/seagate-backup/borg/lafl-laptop.borg: Writing patterns to /tmp/borgmatic-y3ewt8e8/borgmatic/tmpd8ps5op0: R sh:/etc R sh:/home R sh:/root R sh:/var - sh:/**/.cache - sh:/**/[cC]ache - sh:/**/+([a-zA-Z]) [cC]ache - sh:/**/*.tmp - sh:/**/tmp - sh:/**/temp - sh:/**/node_modules - sh:/**/.npm - sh:/**/pnpm/store - sh:/**/.gradle + sh:/**/containers/storage/volumes - sh:/**/containers - sh:/var/lib/borg R /tmp/borgmatic-y3ewt8e8/./borgmatic/bootstrap BORG_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-y3ewt8e8/borgmatic/tmpd8ps5op0 --debug --show-rc --progress /mnt/seagate-backup/borg/lafl-laptop.borg::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f} > using builtin fallback logging configuration 33 self tests completed in 0.05 seconds Verified integrity of /mnt/seagate-backup/borg/lafl-laptop.borg/index.4243 TAM-verified manifest security: read previous location '/mnt/seagate-backup/borg/lafl-laptop.borg' security: read manifest timestamp '2025-01-25T09:44:28.077485' security: determined newest manifest timestamp as 2025-01-25T09:44:28.077485 security: repository checks ok, allowing access Creating archive at "/mnt/seagate-backup/borg/lafl-laptop.borg::lafl-laptop-2025-01-25T10:52:40.240662" Verified integrity of /root/.cache/borg/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848/chunks Reading files cache ... Verified integrity of /root/.cache/borg/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848/files security: read previous location '/mnt/seagate-backup/borg/lafl-laptop.borg' security: read manifest timestamp '2025-01-25T09:44:28.077485' security: determined newest manifest timestamp as 2025-01-25T09:44:28.077485 security: repository checks ok, allowing access Processing files ... sh:/etc: stat: [Errno 2] No such file or directory: 'sh:/etc' sh:/home: stat: [Errno 2] No such file or directory: 'sh:/home' sh:/root: stat: [Errno 2] No such file or directory: 'sh:/root' sh:/var: stat: [Errno 2] No such file or directory: 'sh:/var' sh:/etc: stat: [Errno 2] No such file or directory: 'sh:/etc' sh:/home: stat: [Errno 2] No such file or directory: 'sh:/home' sh:/root: stat: [Errno 2] No such file or directory: 'sh:/root' sh:/var: stat: [Errno 2] No such file or directory: 'sh:/var' Cleaned up 0 uncommitted segment files (== everything after segment 4243). Verified integrity of /mnt/seagate-backup/borg/lafl-laptop.borg/hints.4243 check_free_space: required bytes 1035723608, free bytes 3791046074368 security: saving state for dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848 to /root/.config/borg/security/dce146f0346d888b34cb5f6ca114816e90dc19103a97753c64ba0e91f3d4e848 security: current location /mnt/seagate-backup/borg/lafl-laptop.borg security: key type 3 security: manifest timestamp 2025-01-25T09:52:45.530544 rcs: [107] terminating with warning status, rc 107 ``` ### Expected behavior Borgmatic should not append 'sh:' to the pattern entries. ### Other notes / implementation ideas _No response_ ### borgmatic version 1.9.7 ### borgmatic installation method pacman ### Borg version 1.4.0 ### Python version 3.13.1 ### Database version (if applicable) _No response_ ### Operating system and version Arch Linux
witten commented 2025-01-25 15:47:57 +00:00
Owner
Copy Link

Ugh, thank you for filing this. I'm pretty sure I know what happened here and what the fix is.

Ugh, thank you for filing this. I'm pretty sure I know what happened here and what the fix is.
witten commented 2025-01-25 16:16:46 +00:00
Owner
Copy Link

This is fixed and released in borgmatic 1.9.8. Thanks again!

This is fixed and released in borgmatic 1.9.8. Thanks again!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
witten
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#979
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?