Borgmatic crashing with a "'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data"

Belphemur commented

2026-01-30 20:27:38 +00:00

What I'm trying to do and why

Up until this morning, I had no issue with the same config.
Basically backing up some postgres server and then some of my personal files.

Steps to reproduce

Backup a 17 postgres database of Immich. It fails now.

Actual behavior

f8a31960e2fc:/etc/borgmatic.d# borgmatic --verbosity 2
/etc/borgmatic.d/config.yaml: Calling cronitor hook function initialize_monitor
/etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor
/etc/borgmatic.d/config.yaml: Pinging Cronitor start
/etc/borgmatic.d/config.yaml: Using Cronitor ping URL https://cronitor.link/p/2cb32490afef4606b9bd3050d3a2fc5c/unraid-backup/run
/etc/borgmatic.d/config.yaml: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg --version --log-json --debug --show-rc
/etc/borgmatic.d/config.yaml: Borg 1.4.3
backupserver: Running actions for repository
backupserver: Creating archive
backupserver: Using runtime directory /tmp/borgmatic-h985p9to/borgmatic
backupserver: Calling bootstrap hook function remove_data_source_dumps
backupserver: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
backupserver: Calling btrfs hook function remove_data_source_dumps
backupserver: Calling lvm hook function remove_data_source_dumps
backupserver: Calling mariadb hook function remove_data_source_dumps
backupserver: Removing MariaDB data source dumps
backupserver: Calling mongodb hook function remove_data_source_dumps
backupserver: Removing MongoDB data source dumps
backupserver: Calling mysql hook function remove_data_source_dumps
backupserver: Removing MySQL data source dumps
backupserver: Calling postgresql hook function remove_data_source_dumps
backupserver: Removing PostgreSQL data source dumps
backupserver: Calling sqlite hook function remove_data_source_dumps
backupserver: Removing SQLite data source dumps
backupserver: Calling zfs hook function remove_data_source_dumps
backupserver: Calling bootstrap hook function dump_data_sources
backupserver: Calling postgresql hook function dump_data_sources
backupserver: Dumping PostgreSQL databases
backupserver: Querying for "all" PostgreSQL databases to dump
backupserver: PGPASSWORD=*** psql --list --no-password --no-psqlrc --csv --tuples-only --host postgresql15 --username postgres
backupserver: Dumping PostgreSQL database "firefly" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/firefly
backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host postgresql15 --username postgres --format custom firefly > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/firefly
backupserver: Dumping PostgreSQL database "postgres" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/postgres
backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host postgresql15 --username postgres --format custom postgres > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/postgres
backupserver: Querying for "all" PostgreSQL databases to dump
backupserver: PGPASSWORD=*** psql --list --no-password --no-psqlrc --csv --tuples-only --host immich_postgres --username postgres
backupserver: Dumping PostgreSQL database "immich" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/immich
backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host immich_postgres --username postgres --format custom immich > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/immich
backupserver: Dumping PostgreSQL database "postgres" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/postgres
backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host immich_postgres --username postgres --format custom postgres > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/postgres
backupserver: Writing patterns to /tmp/borgmatic-h985p9to/borgmatic/tmpdje7ombb:
R /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases
+ /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases
R /etc/borgmatic.d/config.yaml
+ /etc/borgmatic.d/config.yaml
R /tmp/borgmatic-h985p9to/./borgmatic/bootstrap
+ /tmp/borgmatic-h985p9to/./borgmatic/bootstrap
R /boot
R /mnt/user/appdata
R /mnt/user/Private
R /mnt/temporary/Minio
R /etc/borgmatic.d
R /mnt/user/Main/Audiobooks/
! fm:/mnt/user/appdata/postgres*
! fm:*pgdata*
! fm:*mysqldata*
! fm:/mnt/user/appdata/EmbyServerBeta/cache/
! fm:/mnt/user/appdata/netdata/cache/
backupserver: Checking file paths Borg plans to include
backupserver: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-h985p9to/borgmatic/tmpdje7ombb --read-special --remote-path borg14 ssh://rsync/./backup/unraid::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}.checkpoint --dry-run --list
backupserver: Calling bootstrap hook function remove_data_source_dumps
backupserver: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
backupserver: Removing bootstrap manifest at /tmp/borgmatic-h985p9to/borgmatic/bootstrap/manifest.json
backupserver: Calling btrfs hook function remove_data_source_dumps
backupserver: Calling lvm hook function remove_data_source_dumps
backupserver: Calling mariadb hook function remove_data_source_dumps
backupserver: Removing MariaDB data source dumps
backupserver: Calling mongodb hook function remove_data_source_dumps
backupserver: Removing MongoDB data source dumps
backupserver: Calling mysql hook function remove_data_source_dumps
backupserver: Removing MySQL data source dumps
backupserver: Calling postgresql hook function remove_data_source_dumps
backupserver: Removing PostgreSQL data source dumps
backupserver: Calling sqlite hook function remove_data_source_dumps
backupserver: Removing SQLite data source dumps
backupserver: Calling zfs hook function remove_data_source_dumps
backupserver: Error running actions for repository
backupserver: 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data
/etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor
/etc/borgmatic.d/config.yaml: Ignoring unsupported monitoring state log in Cronitor hook
/etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor
/etc/borgmatic.d/config.yaml: Pinging Cronitor fail
/etc/borgmatic.d/config.yaml: Using Cronitor ping URL https://cronitor.link/p/2cb32490afef4606b9bd3050d3a2fc5c/unraid-backup/fail
/etc/borgmatic.d/config.yaml: Calling cronitor hook function destroy_monitor
/etc/borgmatic.d/config.yaml: Error running configuration
/etc/borgmatic.d/config.yaml: An error occurred

summary:
/etc/borgmatic.d/config.yaml: Loading configuration file
An error occurred
Error running actions for repository
'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data
Error running configuration

Need some help? https://torsion.org/borgmatic/#issues

Expected behavior

No response

Other notes / implementation ideas

No response

borgmatic version

2.1.1

borgmatic installation method

Docker image

Borg version

borg 1.4.3

Python version

Python 3.14.2

Database version (if applicable)

psql (PostgreSQL) 17.7

Operating system and version

NAME="Alpine Linux" ID=alpine VERSION_ID=3.22.3 PRETTY_NAME="Alpine Linux v3.22" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

### What I'm trying to do and why Up until this morning, I had no issue with the same config. Basically backing up some postgres server and then some of my personal files. ### Steps to reproduce Backup a 17 postgres database of Immich. It fails now. ### Actual behavior ``` f8a31960e2fc:/etc/borgmatic.d# borgmatic --verbosity 2 /etc/borgmatic.d/config.yaml: Calling cronitor hook function initialize_monitor /etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor /etc/borgmatic.d/config.yaml: Pinging Cronitor start /etc/borgmatic.d/config.yaml: Using Cronitor ping URL https://cronitor.link/p/2cb32490afef4606b9bd3050d3a2fc5c/unraid-backup/run /etc/borgmatic.d/config.yaml: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg --version --log-json --debug --show-rc /etc/borgmatic.d/config.yaml: Borg 1.4.3 backupserver: Running actions for repository backupserver: Creating archive backupserver: Using runtime directory /tmp/borgmatic-h985p9to/borgmatic backupserver: Calling bootstrap hook function remove_data_source_dumps backupserver: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap backupserver: Calling btrfs hook function remove_data_source_dumps backupserver: Calling lvm hook function remove_data_source_dumps backupserver: Calling mariadb hook function remove_data_source_dumps backupserver: Removing MariaDB data source dumps backupserver: Calling mongodb hook function remove_data_source_dumps backupserver: Removing MongoDB data source dumps backupserver: Calling mysql hook function remove_data_source_dumps backupserver: Removing MySQL data source dumps backupserver: Calling postgresql hook function remove_data_source_dumps backupserver: Removing PostgreSQL data source dumps backupserver: Calling sqlite hook function remove_data_source_dumps backupserver: Removing SQLite data source dumps backupserver: Calling zfs hook function remove_data_source_dumps backupserver: Calling bootstrap hook function dump_data_sources backupserver: Calling postgresql hook function dump_data_sources backupserver: Dumping PostgreSQL databases backupserver: Querying for "all" PostgreSQL databases to dump backupserver: PGPASSWORD=*** psql --list --no-password --no-psqlrc --csv --tuples-only --host postgresql15 --username postgres backupserver: Dumping PostgreSQL database "firefly" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/firefly backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host postgresql15 --username postgres --format custom firefly > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/firefly backupserver: Dumping PostgreSQL database "postgres" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/postgres backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host postgresql15 --username postgres --format custom postgres > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/postgresql15/postgres backupserver: Querying for "all" PostgreSQL databases to dump backupserver: PGPASSWORD=*** psql --list --no-password --no-psqlrc --csv --tuples-only --host immich_postgres --username postgres backupserver: Dumping PostgreSQL database "immich" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/immich backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host immich_postgres --username postgres --format custom immich > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/immich backupserver: Dumping PostgreSQL database "postgres" to /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/postgres backupserver: PGPASSWORD=*** pg_dump --no-password --clean --if-exists --host immich_postgres --username postgres --format custom postgres > /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases/immich_postgres/postgres backupserver: Writing patterns to /tmp/borgmatic-h985p9to/borgmatic/tmpdje7ombb: R /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases + /tmp/borgmatic-h985p9to/./borgmatic/postgresql_databases R /etc/borgmatic.d/config.yaml + /etc/borgmatic.d/config.yaml R /tmp/borgmatic-h985p9to/./borgmatic/bootstrap + /tmp/borgmatic-h985p9to/./borgmatic/bootstrap R /boot R /mnt/user/appdata R /mnt/user/Private R /mnt/temporary/Minio R /etc/borgmatic.d R /mnt/user/Main/Audiobooks/ ! fm:/mnt/user/appdata/postgres* ! fm:*pgdata* ! fm:*mysqldata* ! fm:/mnt/user/appdata/EmbyServerBeta/cache/ ! fm:/mnt/user/appdata/netdata/cache/ backupserver: Checking file paths Borg plans to include backupserver: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-h985p9to/borgmatic/tmpdje7ombb --read-special --remote-path borg14 ssh://rsync/./backup/unraid::{hostname}-{now:%Y-%m-%dT%H:%M:%S.%f}.checkpoint --dry-run --list backupserver: Calling bootstrap hook function remove_data_source_dumps backupserver: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap backupserver: Removing bootstrap manifest at /tmp/borgmatic-h985p9to/borgmatic/bootstrap/manifest.json backupserver: Calling btrfs hook function remove_data_source_dumps backupserver: Calling lvm hook function remove_data_source_dumps backupserver: Calling mariadb hook function remove_data_source_dumps backupserver: Removing MariaDB data source dumps backupserver: Calling mongodb hook function remove_data_source_dumps backupserver: Removing MongoDB data source dumps backupserver: Calling mysql hook function remove_data_source_dumps backupserver: Removing MySQL data source dumps backupserver: Calling postgresql hook function remove_data_source_dumps backupserver: Removing PostgreSQL data source dumps backupserver: Calling sqlite hook function remove_data_source_dumps backupserver: Removing SQLite data source dumps backupserver: Calling zfs hook function remove_data_source_dumps backupserver: Error running actions for repository backupserver: 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data /etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor /etc/borgmatic.d/config.yaml: Ignoring unsupported monitoring state log in Cronitor hook /etc/borgmatic.d/config.yaml: Calling cronitor hook function ping_monitor /etc/borgmatic.d/config.yaml: Pinging Cronitor fail /etc/borgmatic.d/config.yaml: Using Cronitor ping URL https://cronitor.link/p/2cb32490afef4606b9bd3050d3a2fc5c/unraid-backup/fail /etc/borgmatic.d/config.yaml: Calling cronitor hook function destroy_monitor /etc/borgmatic.d/config.yaml: Error running configuration /etc/borgmatic.d/config.yaml: An error occurred summary: /etc/borgmatic.d/config.yaml: Loading configuration file An error occurred Error running actions for repository 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data Error running configuration Need some help? https://torsion.org/borgmatic/#issues ``` ### Expected behavior _No response_ ### Other notes / implementation ideas _No response_ ### borgmatic version 2.1.1 ### borgmatic installation method Docker image ### Borg version borg 1.4.3 ### Python version Python 3.14.2 ### Database version (if applicable) psql (PostgreSQL) 17.7 ### Operating system and version NAME="Alpine Linux" ID=alpine VERSION_ID=3.22.3 PRETTY_NAME="Alpine Linux v3.22" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

witten commented

2026-01-31 23:38:44 +00:00

Owner

Thanks for filing this. Some thoughts:

I assume this worked for you with a previous version of borgmatic, and an upgrade broke it? Do you know what the last version was that worked? For instance, was it 2.0.13 or 2.1.0? That will help me narrow down the cause.
What does your borgmatic PostgreSQL configuration look like? Feel free to redact names as necessary.
What's your system locale? Running the locale command might show you.
If you comment out the PostgreSQL configuration, does the problem still occur?
If you temporarily change your source_directories or patterns to greatly limit the source paths (e.g. just backup /boot only or whatever), does the problem still occur?

My guess is what's happening is that when borgmatic is doing a Borg dry run to figure out what files it plans to backup, it's encountering a filename that's not in an encoding it expects.

Thanks for filing this. Some thoughts: * I assume this worked for you with a previous version of borgmatic, and an upgrade broke it? Do you know what the last version was that worked? For instance, was it 2.0.13 or 2.1.0? That will help me narrow down the cause. * What does your borgmatic PostgreSQL configuration look like? Feel free to redact names as necessary. * What's your system locale? Running the `locale` command might show you. * If you comment out the PostgreSQL configuration, does the problem still occur? * If you temporarily change your `source_directories` or `patterns` to greatly limit the source paths (e.g. just backup `/boot` only or whatever), does the problem still occur? My guess is what's happening is that when borgmatic is doing a Borg dry run to figure out what files it plans to backup, it's encountering a filename that's not in an encoding it expects.

maxhamon commented

2026-02-01 11:19:30 +00:00

Contributor

Same errors here since Borgmatic 2.1.2, it does not appear at every run, and not for every of my database (only 2 of them) :

➜ borgmatic -c /etc/borgmatic.d/redacted.yaml --verbosity 2 create
/etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function initialize_monitor
/etc/borgmatic.d/redacted.yaml: Calling loki hook function initialize_monitor
/etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Pinging Healthchecks start
/etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/start
/etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Calling file hook function load_credential
/etc/borgmatic.d/redacted.yaml: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg --version --log-json --debug --show-rc
/etc/borgmatic.d/redacted.yaml: Borg 1.4.3
redacted: Running actions for repository
redacted: Running before create action command hook
redacted: echo "`date` - Starting a backup on ssh://redacted@redacted.repo.borgbase.com/./repo"
Sun Feb  1 12:09:32 PM CET 2026 - Starting a backup on ssh://redacted@redacted.repo.borgbase.com/./repo
redacted: Creating archive
redacted: Using runtime directory /tmp/borgmatic-_lgvw9w5/borgmatic
redacted: Calling bootstrap hook function remove_data_source_dumps
redacted: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
redacted: Calling btrfs hook function remove_data_source_dumps
redacted: Calling lvm hook function remove_data_source_dumps
redacted: Calling mariadb hook function remove_data_source_dumps
redacted: Removing MariaDB data source dumps
redacted: Calling mongodb hook function remove_data_source_dumps
redacted: Removing MongoDB data source dumps
redacted: Calling mysql hook function remove_data_source_dumps
redacted: Removing MySQL data source dumps
redacted: Calling postgresql hook function remove_data_source_dumps
redacted: Removing PostgreSQL data source dumps
redacted: Calling sqlite hook function remove_data_source_dumps
redacted: Removing SQLite data source dumps
redacted: Calling zfs hook function remove_data_source_dumps
redacted: Calling bootstrap hook function dump_data_sources
redacted: Calling postgresql hook function dump_data_sources
redacted: Dumping PostgreSQL databases
redacted: Dumping PostgreSQL database "all" to /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases/localhost/all
redacted: docker compose --project-directory=/srv/redacted exec -i db pg_dumpall --no-password --clean --if-exists --username redacted > /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases/localhost/all
redacted: Writing patterns to /tmp/borgmatic-_lgvw9w5/borgmatic/tmppg7bnam7:
R /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases
+ /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases
R /etc/borgmatic/common.yml
+ /etc/borgmatic/common.yml
R /etc/borgmatic.d/redacted.yaml
+ /etc/borgmatic.d/redacted.yaml
R /tmp/borgmatic-_lgvw9w5/./borgmatic/bootstrap
+ /tmp/borgmatic-_lgvw9w5/./borgmatic/bootstrap
R /srv/redacted
! fm:**/tmp
! fm:**/temp
! fm:**/valkey
! fm:**/[Cc]ache
redacted: Checking file paths Borg plans to include
redacted: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-_lgvw9w5/borgmatic/tmppg7bnam7 --exclude-caches --exclude-if-present .nobackup --compression auto,zstd --read-special ssh://redacted@redacted.repo.borgbase.com/./repo::redacted-{now}.checkpoint --dry-run --list
redacted: Calling bootstrap hook function remove_data_source_dumps
redacted: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap
redacted: Removing bootstrap manifest at /tmp/borgmatic-_lgvw9w5/borgmatic/bootstrap/manifest.json
redacted: Calling btrfs hook function remove_data_source_dumps
redacted: Calling lvm hook function remove_data_source_dumps
redacted: Calling mariadb hook function remove_data_source_dumps
redacted: Removing MariaDB data source dumps
redacted: Calling mongodb hook function remove_data_source_dumps
redacted: Removing MongoDB data source dumps
redacted: Calling mysql hook function remove_data_source_dumps
redacted: Removing MySQL data source dumps
redacted: Calling postgresql hook function remove_data_source_dumps
redacted: Removing PostgreSQL data source dumps
redacted: Calling sqlite hook function remove_data_source_dumps
redacted: Removing SQLite data source dumps
redacted: Calling zfs hook function remove_data_source_dumps
redacted: Running after create action command hook
redacted: echo "`date` - Finished a backup on ssh://redacted@redacted.repo.borgbase.com/./repo"
Sun Feb  1 12:09:32 PM CET 2026 - Finished a backup on ssh://redacted@redacted.repo.borgbase.com/./repo
redacted: Error running actions for repository
redacted: 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data
/etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Pinging Healthchecks log
/etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/log
/etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Pinging Healthchecks fail
/etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/fail
/etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor
/etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function destroy_monitor
/etc/borgmatic.d/redacted.yaml: Calling loki hook function destroy_monitor
/etc/borgmatic.d/redacted.yaml: Error running configuration
/etc/borgmatic.d/redacted.yaml: An error occurred

summary:
/etc/borgmatic.d/redacted.yaml: Loading configuration file
An error occurred
Error running actions for repository
'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data
Error running configuration

Need some help? https://torsion.org/borgmatic/#issues

My config :

➜ cat /etc/borgmatic/common.yml 
#
# This file is managed by Ansible. Manual changes are likely to be overwritten !
#

compression: auto,zstd
syslog_verbosity: 1
statistics: true
exclude_caches: true
exclude_if_present:
  - '.nobackup'

# Retention
keep_hourly: 12
keep_daily: 7
keep_weekly: 4
keep_monthly: 12
keep_yearly: 2

# Checks
checks:
  - name: 'archives'
    frequency: '1 day'
  - name: 'repository'
    frequency: '1 week'
check_last: 3

borg_exit_codes:
  - code: 102
    treat_as: error
  - code: 105
    treat_as: error
  - code: 106
    treat_as: error

commands:
  - before: action
    when:
      - create
    run:
      - echo "`date` - Starting a backup on {repository}"
  - before: action
    when:
      - prune
    run:
      - echo "`date` - Starting pruning on {repository}"
  - before: action
    when:
      - check
    run:
      - echo "`date` - Starting checks on {repository}"
  - after: action
    when:
      - create
    run:
      - echo "`date` - Finished a backup on {repository}"
  - after: action
    when:
      - prune
    run:
      - echo "`date` - Finished pruning on {repository}"
  - after: action
    when:
      - check
    run:
      - echo "`date` - Finished checks on {repository}"

loki:
  url: 'https://logs/loki/api/v1/push'
  send_logs: true
  labels:
    job: logs/borgmatic
    instance: __hostname
    
➜ cat /etc/borgmatic.d/redacted.yaml 
#
# This file is managed by Ansible. Manual changes are likely to be overwritten !
#

<<: !include /etc/borgmatic/common.yml

source_directories:
  - '/srv/redacted'
exclude_patterns:
  - '**/tmp'
  - '**/temp'
  - '**/valkey'
  - '**/[Cc]ache'

repositories:
  - path: 'ssh://redacted@redacted.repo.borgbase.com/./repo'
    label: 'redacted'
encryption_passphrase: '{credential file /etc/borgmatic/credentials/redacted}'

archive_name_format: 'redacted-{now}'

read_special: true

postgresql_databases:
  - name: all
    username: paperless
    pg_dump_command: docker compose --project-directory=/srv/redacted exec -i db pg_dumpall
    psql_command: docker compose --project-directory=/srv/redacted exec -i db psql


healthchecks:
  ping_url: 'https://cron.hamon.me/ping/redacted/redacted'

loki:
  labels:
    service: redacted

I'm using :
Borgmatic 2.1.1
Borg 1.4.3
Python 3.13.5
Debian 13 - using en_US.UTF-8
Docker version 29.2.0, build 0b9d198

My database in my compose if you need

  db:
    image: docker.io/postgres:18.1
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      PGPASSWORD: ${POSTGRES_PASSWORD}
    networks:
      - default
    volumes:
      - ./db/data:/var/lib/postgresql

Same errors here since Borgmatic 2.1.2, it does not appear at every run, and not for every of my database (only 2 of them) : ``` ➜ borgmatic -c /etc/borgmatic.d/redacted.yaml --verbosity 2 create /etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function initialize_monitor /etc/borgmatic.d/redacted.yaml: Calling loki hook function initialize_monitor /etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Pinging Healthchecks start /etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/start /etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Calling file hook function load_credential /etc/borgmatic.d/redacted.yaml: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg --version --log-json --debug --show-rc /etc/borgmatic.d/redacted.yaml: Borg 1.4.3 redacted: Running actions for repository redacted: Running before create action command hook redacted: echo "`date` - Starting a backup on ssh://redacted@redacted.repo.borgbase.com/./repo" Sun Feb 1 12:09:32 PM CET 2026 - Starting a backup on ssh://redacted@redacted.repo.borgbase.com/./repo redacted: Creating archive redacted: Using runtime directory /tmp/borgmatic-_lgvw9w5/borgmatic redacted: Calling bootstrap hook function remove_data_source_dumps redacted: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap redacted: Calling btrfs hook function remove_data_source_dumps redacted: Calling lvm hook function remove_data_source_dumps redacted: Calling mariadb hook function remove_data_source_dumps redacted: Removing MariaDB data source dumps redacted: Calling mongodb hook function remove_data_source_dumps redacted: Removing MongoDB data source dumps redacted: Calling mysql hook function remove_data_source_dumps redacted: Removing MySQL data source dumps redacted: Calling postgresql hook function remove_data_source_dumps redacted: Removing PostgreSQL data source dumps redacted: Calling sqlite hook function remove_data_source_dumps redacted: Removing SQLite data source dumps redacted: Calling zfs hook function remove_data_source_dumps redacted: Calling bootstrap hook function dump_data_sources redacted: Calling postgresql hook function dump_data_sources redacted: Dumping PostgreSQL databases redacted: Dumping PostgreSQL database "all" to /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases/localhost/all redacted: docker compose --project-directory=/srv/redacted exec -i db pg_dumpall --no-password --clean --if-exists --username redacted > /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases/localhost/all redacted: Writing patterns to /tmp/borgmatic-_lgvw9w5/borgmatic/tmppg7bnam7: R /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases + /tmp/borgmatic-_lgvw9w5/./borgmatic/postgresql_databases R /etc/borgmatic/common.yml + /etc/borgmatic/common.yml R /etc/borgmatic.d/redacted.yaml + /etc/borgmatic.d/redacted.yaml R /tmp/borgmatic-_lgvw9w5/./borgmatic/bootstrap + /tmp/borgmatic-_lgvw9w5/./borgmatic/bootstrap R /srv/redacted ! fm:**/tmp ! fm:**/temp ! fm:**/valkey ! fm:**/[Cc]ache redacted: Checking file paths Borg plans to include redacted: BORG_PASSPHRASE_FD=*** BORG_DEBUG_PASSPHRASE=*** BORG_DISPLAY_PASSPHRASE=*** BORG_RELOCATED_REPO_ACCESS_IS_OK=*** BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=*** BORG_USE_CHUNKS_ARCHIVE=*** BORG_EXIT_CODES=*** borg create --patterns-from /tmp/borgmatic-_lgvw9w5/borgmatic/tmppg7bnam7 --exclude-caches --exclude-if-present .nobackup --compression auto,zstd --read-special ssh://redacted@redacted.repo.borgbase.com/./repo::redacted-{now}.checkpoint --dry-run --list redacted: Calling bootstrap hook function remove_data_source_dumps redacted: Looking for bootstrap manifest files to remove in /tmp/borgmatic-*/borgmatic/bootstrap redacted: Removing bootstrap manifest at /tmp/borgmatic-_lgvw9w5/borgmatic/bootstrap/manifest.json redacted: Calling btrfs hook function remove_data_source_dumps redacted: Calling lvm hook function remove_data_source_dumps redacted: Calling mariadb hook function remove_data_source_dumps redacted: Removing MariaDB data source dumps redacted: Calling mongodb hook function remove_data_source_dumps redacted: Removing MongoDB data source dumps redacted: Calling mysql hook function remove_data_source_dumps redacted: Removing MySQL data source dumps redacted: Calling postgresql hook function remove_data_source_dumps redacted: Removing PostgreSQL data source dumps redacted: Calling sqlite hook function remove_data_source_dumps redacted: Removing SQLite data source dumps redacted: Calling zfs hook function remove_data_source_dumps redacted: Running after create action command hook redacted: echo "`date` - Finished a backup on ssh://redacted@redacted.repo.borgbase.com/./repo" Sun Feb 1 12:09:32 PM CET 2026 - Finished a backup on ssh://redacted@redacted.repo.borgbase.com/./repo redacted: Error running actions for repository redacted: 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data /etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Pinging Healthchecks log /etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/log /etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Pinging Healthchecks fail /etc/borgmatic.d/redacted.yaml: Using Healthchecks ping URL https://cron.hamon.me/ping/redacted/redacted-sauvegarde-borg/fail /etc/borgmatic.d/redacted.yaml: Calling loki hook function ping_monitor /etc/borgmatic.d/redacted.yaml: Calling healthchecks hook function destroy_monitor /etc/borgmatic.d/redacted.yaml: Calling loki hook function destroy_monitor /etc/borgmatic.d/redacted.yaml: Error running configuration /etc/borgmatic.d/redacted.yaml: An error occurred summary: /etc/borgmatic.d/redacted.yaml: Loading configuration file An error occurred Error running actions for repository 'utf-8' codec can't decode byte 0xc3 in position 4095: unexpected end of data Error running configuration Need some help? https://torsion.org/borgmatic/#issues ``` My config : ``` ➜ cat /etc/borgmatic/common.yml # # This file is managed by Ansible. Manual changes are likely to be overwritten ! # compression: auto,zstd syslog_verbosity: 1 statistics: true exclude_caches: true exclude_if_present: - '.nobackup' # Retention keep_hourly: 12 keep_daily: 7 keep_weekly: 4 keep_monthly: 12 keep_yearly: 2 # Checks checks: - name: 'archives' frequency: '1 day' - name: 'repository' frequency: '1 week' check_last: 3 borg_exit_codes: - code: 102 treat_as: error - code: 105 treat_as: error - code: 106 treat_as: error commands: - before: action when: - create run: - echo "`date` - Starting a backup on {repository}" - before: action when: - prune run: - echo "`date` - Starting pruning on {repository}" - before: action when: - check run: - echo "`date` - Starting checks on {repository}" - after: action when: - create run: - echo "`date` - Finished a backup on {repository}" - after: action when: - prune run: - echo "`date` - Finished pruning on {repository}" - after: action when: - check run: - echo "`date` - Finished checks on {repository}" loki: url: 'https://logs/loki/api/v1/push' send_logs: true labels: job: logs/borgmatic instance: __hostname ➜ cat /etc/borgmatic.d/redacted.yaml # # This file is managed by Ansible. Manual changes are likely to be overwritten ! # <<: !include /etc/borgmatic/common.yml source_directories: - '/srv/redacted' exclude_patterns: - '**/tmp' - '**/temp' - '**/valkey' - '**/[Cc]ache' repositories: - path: 'ssh://redacted@redacted.repo.borgbase.com/./repo' label: 'redacted' encryption_passphrase: '{credential file /etc/borgmatic/credentials/redacted}' archive_name_format: 'redacted-{now}' read_special: true postgresql_databases: - name: all username: paperless pg_dump_command: docker compose --project-directory=/srv/redacted exec -i db pg_dumpall psql_command: docker compose --project-directory=/srv/redacted exec -i db psql healthchecks: ping_url: 'https://cron.hamon.me/ping/redacted/redacted' loki: labels: service: redacted ``` I'm using : Borgmatic 2.1.1 Borg 1.4.3 Python 3.13.5 Debian 13 - using en_US.UTF-8 Docker version 29.2.0, build 0b9d198 My database in my compose if you need ``` db: image: docker.io/postgres:18.1 environment: POSTGRES_DB: ${POSTGRES_DB} POSTGRES_USER: ${POSTGRES_USER} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} PGPASSWORD: ${POSTGRES_PASSWORD} networks: - default volumes: - ./db/data:/var/lib/postgresql ```

witten commented

2026-02-01 19:09:13 +00:00

Owner

@maxhamon Thanks for weighing in and including this information about your setup. So it sounds like this started happening in 2.1.2 and didn't occur in 2.1.1? If so, that definitely helps narrow it down. And your system locale is UTF-8 rather than something more exotic, so the locale probably isn't causing this.

What about these other two possibilities?

If you comment out the PostgreSQL configuration entirely, does the problem still occur?
If you temporarily change your source_directories to include just a subset of your files (e.g. just backup /srv/redacted/one_small_sub_directory only instead of all of /srv/redacted), does the problem still occur?

Thank you!

@maxhamon Thanks for weighing in and including this information about your setup. So it sounds like this started happening in 2.1.2 and didn't occur in 2.1.1? If so, that definitely helps narrow it down. And your system locale is UTF-8 rather than something more exotic, so the locale probably isn't causing this. What about these other two possibilities? * If you comment out the PostgreSQL configuration entirely, does the problem still occur? * If you temporarily change your `source_directories` to include just a subset of your files (e.g. just backup `/srv/redacted/one_small_sub_directory` only instead of all of `/srv/redacted`), does the problem still occur? Thank you!

slarti commented

2026-02-01 20:44:26 +00:00

Hi, I am also getting the error in 2.1.1 but not in 2.1.0. Once in a configuration to dump a PostgreSQL database and another one in an SQLite database but not all PostgreSQL or SQLite database are affected. I am using the ghcr.io docker images and pinned it to 2.1.0 at the moment.

witten commented

2026-02-01 21:23:02 +00:00

Owner

Hi @slarti, that's helpful to know this started for you in 2.1.1. A few questions:

Can I get a look at your SQLite and PostgreSQL borgmatic configuration, redacted as necessary?
Does the error show up in the same relative place in borgmatic's logs for you? In other words, after Checking file paths Borg plans to include, the logged Borg dry run command, and all the various remove_data_source_dumps logs?
If you comment out the SQLite and PostgreSQL borgmatic configuration, does the problem still occur?
If you temporarily change your source_directories or patterns to greatly limit the source paths (e.g. just backup one small directory), does the problem still occur?

Hi @slarti, that's helpful to know this started for you in 2.1.1. A few questions: * Can I get a look at your SQLite and PostgreSQL borgmatic configuration, redacted as necessary? * Does the error show up in the same relative place in borgmatic's logs for you? In other words, after `Checking file paths Borg plans to include`, the logged Borg dry run command, and all the various `remove_data_source_dumps` logs? * If you comment out the SQLite and PostgreSQL borgmatic configuration, does the problem still occur? * If you temporarily change your `source_directories` or `patterns` to greatly limit the source paths (e.g. just backup one small directory), does the problem still occur?

witten commented

2026-02-01 21:51:55 +00:00

Owner

I have a theory about what could be going wrong here: Starting in borgmatic 2.1.1, borgmatic reads output from executed commands (like Borg, PostgreSQL, and SQLite) in up to 4096-byte chunks. It unicode-decodes that data (as UTF-8) and breaks it into lines for consumption elsewhere in borgmatic. This change was made in #1242. The problem, I'm guessing, is that if a multi-byte unicode character happens to straddle that 4096-byte boundary, borgmatic will attempt to decode just the first byte of that character—and fail with the error you're seeing. Note the position 4095 (4096 - 1) mentioned in the error.

In terms of a fix, the right thing to do is probably to hold off any unicode decoding until a full line is received. That way, we won't be at risk of accidentally splitting a multi-byte unicode character and trying to decode just one part.

Note that this is just a theory at this point, and I don't have a local repro. Getting answers to some of the questions above would help me pin it down.

EDIT: I've managed to write a failing unit test that produces the same error y'all are getting. Now the trick will just be fixing the code to make that test pass.

I have a theory about what could be going wrong here: Starting in borgmatic 2.1.1, borgmatic reads output from executed commands (like Borg, PostgreSQL, and SQLite) in up to 4096-byte chunks. It unicode-decodes that data (as UTF-8) and breaks it into lines for consumption elsewhere in borgmatic. This change was made in #1242. The problem, I'm guessing, is that if a multi-byte unicode character happens to straddle that 4096-byte boundary, borgmatic will attempt to decode just the first byte of that character—and fail with the error you're seeing. Note the position 4095 (4096 - 1) mentioned in the error. In terms of a fix, the right thing to do is probably to hold off any unicode decoding until a full line is received. That way, we won't be at risk of accidentally splitting a multi-byte unicode character and trying to decode just one part. Note that this is just a theory at this point, and I don't have a local repro. Getting answers to some of the questions above would help me pin it down. EDIT: I've managed to write a failing unit test that produces the same error y'all are getting. Now the trick will just be fixing the code to make that test pass.

witten referenced this issue from a commit

2026-02-02 04:15:41 +00:00

Fix a "codec can't decode byte" error when running commands that output multi-byte unicode characters (#1258).

witten commented

2026-02-02 04:19:11 +00:00

Owner

Okay, I believe I have a fix for this in main, and it will be part of the next release. If anyone would like to test it, you can download execute.py with the fix and use it to replace your local copy (located at borgmatic/execute.py in your borgmatic source directory, wherever that is on your system) and report back! Otherwise, you can wait for the next release. Thanks.

Okay, I believe I have a fix for this in main, and it will be part of the next release. If anyone would like to test it, you can [download execute.py with the fix](https://projects.torsion.org/borgmatic-collective/borgmatic/raw/commit/a27dc95c87f59ac29a9d73f0f819abdfa1dca4c7/borgmatic/execute.py) and use it to replace your local copy (located at `borgmatic/execute.py` in your borgmatic source directory, wherever that is on your system) and report back! Otherwise, you can wait for the next release. Thanks.

witten closed this issue

2026-02-02 04:19:11 +00:00

maxhamon commented

2026-02-02 12:52:30 +00:00

Contributor

I haven’t had any errors since patching execute.py a few hours ago, so it looks like this fixes the issue for me.

I haven’t had any errors since patching `execute.py` a few hours ago, so it looks like this fixes the issue **for me**.

🎉 1

witten commented

2026-02-02 18:55:04 +00:00

Owner

Awesome, I appreciate you testing it out and reporting back! Glad to hear it's working for you.