borgmatic-collective/borgmatic
borgmatic-collective/borgmatic
Watch
9
74
Fork
You've already forked borgmatic
 57
Code Issues 71 Pull requests 3 Releases 162 Packages 1 Activity Actions 1

Listing archive contents replaces invalid UTF-8 bytes with "?" (question mark) #1282

New issue
Open
opened 2026-03-09 22:45:15 +00:00 by hop · 1 comment
hop commented 2026-03-09 22:45:15 +00:00
Copy link

What I'm trying to do and why

I'm trying to check, whether files containing non UTF-8 byte sequences in the name are backed up correctly.

Steps to reproduce

/tmp/scratch$ LC_CTYPE=de_AT.utf8
/tmp/scratch$ cat > config.yaml <<EOF
source_directories:
    - /tmp/scratch/source
repositories:
    - path: /tmp/scratch/repo
keep_within: 1H
EOF

/tmp/scratch$ mkdir source
/tmp/scratch$ touch source/$'\326'sterreich
/tmp/scratch$ borgmatic -c config.yaml repo-create -e none
/tmp/scratch$ borgmatic -c config.yaml
/tmp/scratch$ mkdir dest
/tmp/scratch$ borgmatic -c config.yaml extract --path tmp/ --archive latest --destination dest --strip-components 3
/tmp/scratch$ ls dest/
''$'\326''sterreich'
/tmp/scratch$ borgmatic -c config.yaml list --archive latest | tail -n1 | xxd
00000000: 1b5b 3335 6d2d 7277 2d72 2d2d 722d 2d20  .[35m-rw-r--r--
00000010: 686f 7020 2020 2068 6f70 2020 2020 2020  hop    hop
00000020: 2020 2020 2030 2053 6174 2c20 3230 3236       0 Sat, 2026
00000030: 2d30 332d 3037 2032 303a 3331 3a34 3220  -03-07 20:31:42
00000040: 746d 702f 7363 7261 7463 682f 736f 7572  tmp/scratch/sour
00000050: 6365 2f3f 7374 6572 7265 6963 681b 5b30  ce/?sterreich.[0
00000060: 6d0a
hop@pvsvie01a2:/tmp/scratch$ LC_CTYPE=de_AT.iso8859-1 borgmatic -c config.yaml list --archive latest | tail -n1 | xxd
00000000: 1b5b 3335 6d2d 7277 2d72 2d2d 722d 2d20  .[35m-rw-r--r--
00000010: 686f 7020 2020 2068 6f70 2020 2020 2020  hop    hop
00000020: 2020 2020 2030 2053 756e 2c20 3230 3236       0 Sun, 2026
00000030: 2d30 332d 3038 2032 323a 3233 3a30 3620  -03-08 22:23:06
00000040: 746d 702f 7363 7261 7463 682f 736f 7572  tmp/scratch/sour
00000050: 6365 2f3f 7374 6572 7265 6963 681b 5b30  ce/?sterreich.[0
00000060: 6d0a

Actual behavior

Invalid bytes are rendered as ? (ASCII \x3f).

Expected behavior

Invalid bytes should be rendered as is or as they would be extracted, especially when running in a single byte encoding locale.

Other notes / implementation ideas

No response

borgmatic version

2.1.3

borgmatic installation method

No response

Borg version

borg 1.4.3

Python version

Python 3.14.3

Database version (if applicable)

No response

Operating system and version

Linux / ext4 filesystem

### What I'm trying to do and why I'm trying to check, whether files containing non UTF-8 byte sequences in the name are backed up correctly. ### Steps to reproduce ``` /tmp/scratch$ LC_CTYPE=de_AT.utf8 /tmp/scratch$ cat > config.yaml <<EOF source_directories: - /tmp/scratch/source repositories: - path: /tmp/scratch/repo keep_within: 1H EOF /tmp/scratch$ mkdir source /tmp/scratch$ touch source/$'\326'sterreich /tmp/scratch$ borgmatic -c config.yaml repo-create -e none /tmp/scratch$ borgmatic -c config.yaml /tmp/scratch$ mkdir dest /tmp/scratch$ borgmatic -c config.yaml extract --path tmp/ --archive latest --destination dest --strip-components 3 /tmp/scratch$ ls dest/ ''$'\326''sterreich' /tmp/scratch$ borgmatic -c config.yaml list --archive latest | tail -n1 | xxd 00000000: 1b5b 3335 6d2d 7277 2d72 2d2d 722d 2d20 .[35m-rw-r--r-- 00000010: 686f 7020 2020 2068 6f70 2020 2020 2020 hop hop 00000020: 2020 2020 2030 2053 6174 2c20 3230 3236 0 Sat, 2026 00000030: 2d30 332d 3037 2032 303a 3331 3a34 3220 -03-07 20:31:42 00000040: 746d 702f 7363 7261 7463 682f 736f 7572 tmp/scratch/sour 00000050: 6365 2f3f 7374 6572 7265 6963 681b 5b30 ce/?sterreich.[0 00000060: 6d0a hop@pvsvie01a2:/tmp/scratch$ LC_CTYPE=de_AT.iso8859-1 borgmatic -c config.yaml list --archive latest | tail -n1 | xxd 00000000: 1b5b 3335 6d2d 7277 2d72 2d2d 722d 2d20 .[35m-rw-r--r-- 00000010: 686f 7020 2020 2068 6f70 2020 2020 2020 hop hop 00000020: 2020 2020 2030 2053 756e 2c20 3230 3236 0 Sun, 2026 00000030: 2d30 332d 3038 2032 323a 3233 3a30 3620 -03-08 22:23:06 00000040: 746d 702f 7363 7261 7463 682f 736f 7572 tmp/scratch/sour 00000050: 6365 2f3f 7374 6572 7265 6963 681b 5b30 ce/?sterreich.[0 00000060: 6d0a ``` ### Actual behavior Invalid bytes are rendered as `?` (ASCII `\x3f`). ### Expected behavior Invalid bytes should be rendered as is or as they would be extracted, especially when running in a single byte encoding locale. ### Other notes / implementation ideas _No response_ ### borgmatic version 2.1.3 ### borgmatic installation method _No response_ ### Borg version borg 1.4.3 ### Python version Python 3.14.3 ### Database version (if applicable) _No response_ ### Operating system and version Linux / ext4 filesystem
witten commented 2026-03-10 01:47:23 +00:00
Owner
Copy link

I'm pretty sure this is a Borg limitation. Example:

$  borg list repo::flux-2026-03-09T18:38:14.130398 | tail -n1 | xxd
00000000: 2d72 772d 722d 2d72 2d2d 2077 6974 7465  -rw-r--r-- witte
00000010: 6e20 7769 7474 656e 2020 2020 2020 2020  n witten        
00000020: 3020 4d6f 6e2c 2032 3032 362d 3033 2d30  0 Mon, 2026-03-0
00000030: 3920 3138 3a33 363a 3431 2068 6f6d 652f  9 18:36:41 home/
00000040: 7769 7474 656e 2f44 6f77 6e6c 6f61 6473  witten/Downloads
00000050: 2f74 6d70 2f73 6f75 7263 652f 3f73 7465  /tmp/source/?ste
00000060: 7272 6569 6368 0a                        rreich.
$ LC_CTYPE=de_AT.iso8859-1 borg list repo::flux-2026-03-09T18:38:14.130398 | tail -n1 | xxd
00000000: 2d72 772d 722d 2d72 2d2d 2077 6974 7465  -rw-r--r-- witte
00000010: 6e20 7769 7474 656e 2020 2020 2020 2020  n witten        
00000020: 3020 4d6f 6e2c 2032 3032 362d 3033 2d30  0 Mon, 2026-03-0
00000030: 3920 3138 3a33 363a 3431 2068 6f6d 652f  9 18:36:41 home/
00000040: 7769 7474 656e 2f44 6f77 6e6c 6f61 6473  witten/Downloads
00000050: 2f74 6d70 2f73 6f75 7263 652f 3f73 7465  /tmp/source/?ste
00000060: 7272 6569 6368 0a                        rreich.

Let me know if I'm missing something.

I'm pretty sure this is a Borg limitation. Example: ``` $  borg list repo::flux-2026-03-09T18:38:14.130398 | tail -n1 | xxd 00000000: 2d72 772d 722d 2d72 2d2d 2077 6974 7465 -rw-r--r-- witte 00000010: 6e20 7769 7474 656e 2020 2020 2020 2020 n witten 00000020: 3020 4d6f 6e2c 2032 3032 362d 3033 2d30 0 Mon, 2026-03-0 00000030: 3920 3138 3a33 363a 3431 2068 6f6d 652f 9 18:36:41 home/ 00000040: 7769 7474 656e 2f44 6f77 6e6c 6f61 6473 witten/Downloads 00000050: 2f74 6d70 2f73 6f75 7263 652f 3f73 7465 /tmp/source/?ste 00000060: 7272 6569 6368 0a rreich. $ LC_CTYPE=de_AT.iso8859-1 borg list repo::flux-2026-03-09T18:38:14.130398 | tail -n1 | xxd 00000000: 2d72 772d 722d 2d72 2d2d 2077 6974 7465 -rw-r--r-- witte 00000010: 6e20 7769 7474 656e 2020 2020 2020 2020 n witten 00000020: 3020 4d6f 6e2c 2032 3032 362d 3033 2d30 0 Mon, 2026-03-0 00000030: 3920 3138 3a33 363a 3431 2068 6f6d 652f 9 18:36:41 home/ 00000040: 7769 7474 656e 2f44 6f77 6e6c 6f61 6473 witten/Downloads 00000050: 2f74 6d70 2f73 6f75 7263 652f 3f73 7465 /tmp/source/?ste 00000060: 7272 6569 6368 0a rreich. ``` Let me know if I'm missing something.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
parallel-borgmatic
command-hook-steps
2.1.7
2.1.6
2.1.5
2.1.4
2.1.3
2.1.2
2.1.1
2.1.0
2.0.13
2.0.12
2.0.11
2.0.10
2.0.9
2.0.8
2.0.7
2.0.6
2.0.5
2.0.4
2.0.3
2.0.2
2.0.1
2.0.0
1.9.14
1.9.13
1.9.12
1.9.11
1.9.10
1.9.9
1.9.8
1.9.7
1.9.6
1.9.5
1.9.4
1.9.3
1.9.2
1.9.1
1.9.0
1.8.14
1.8.13
1.8.12
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
blocked

   
breaking

   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No labels
blocked
breaking
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
borgmatic-collective/borgmatic#1282
No description provided.