returned non-zero exit status 2 #667

New Issue

kilonde · 2023-04-05T19:11:26Z

kilonde commented

2023-04-05 19:11:26 +00:00

What I'm trying to do and why

run remote backup that has previously worked and it fails

Steps to reproduce (if a bug)

borgmatic prune create -v 1 --files --stats > /mnt/logs/backup-"date +"%F+%T"".log

location:
    source_directories:
        - /boot
        - /mnt/user/appdata
        - /mnt/user/nextcloud
        - /mnt/user/data
        - /mnt/user/photos
    repositories:
        - ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo
    one_file_system: true
    files_cache: mtime,size
    patterns:
        - '- [Tt]rash'
        - '- [Cc]ache'
    exclude_patterns:
        - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Crash Reports
        - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Logs
        - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Media
        - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Cache
        - /mnt/user/appdata/radarr/MediaCover
        - /mnt/user/appdata/sonarr/MediaCover
        - '.DS_Store'
        - '*.log'
        - '*.log.*'
        - '*.tmp'
        - '.Recycle.Bin'                                      
    exclude_if_present:
        - .nobackup
        - .NOBACKUP

storage:
    encryption_passphrase: "PASSPHRASE"
    compression: lz4
    ssh_command: ssh -i /root/.ssh/Pi
    archive_name_format: 'backup-{now}'
    relocated_repo_access_is_ok: true

retention:
    keep_hourly: 2
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 12
    keep_yearly: 10
    prefix: 'backup-'

consistency:
    checks:
        - repository
        - archives
    prefix: 'backup-'

hooks:
    before_backup:
        - echo "Starting a backup."
    after_backup:
        - echo "External drive unmounted, finished a backup."
    on_error:
        - echo "Error during prune/create/check."
    mysql_databases:
        - name: all
          hostname: 192.168.1.94
          username: USERNAME          
          password: PASSWORD
          options: "--skip-comments"          
    healthchecks: https://hc-ping.com/8e20df00-1a06-4de3-9b9b-6c0574aaa03b


#borgmatic create -c "/etc/borgmatic.d/test.yaml" -v 2 --files --stats

Actual behavior (if a bug)

Include (sanitized) --verbosity 2 output if applicable.

ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo: Error running actions for repository
Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --stats --debug --show-rc ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo' returned non-zero exit status 2.
Error during prune/create/check.
/etc/borgmatic.d/portable2TB.yaml: Error running configuration file

summary:
/etc/borgmatic.d/portable2TB.yaml: Error running configuration file
ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo: Error running actions for repository
...
    res = f(**args)
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 1122, in get
    self.index = self.open_index(self.get_transaction_id())
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 401, in get_transaction_id
    self.check_transaction()
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 382, in check_transaction
    segments_transaction_id = self.io.get_segments_transaction_id()
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 1276, in get_segments_transaction_id
    if self.is_committed_segment(segment):
  File "/usr/lib/python3/dist-packages/borg/repository.py", line 1311, in is_committed_segment
    if fd.read(self.header_fmt.size) != self.COMMIT:
OSError: [Errno 5] Input/output error
Borg server: Platform: Linux dietIoT 6.1.19-v7+ #1637 SMP Tue Mar 14 11:04:52 GMT 2023 armv7l
Borg server: Linux: Unknown Linux
Borg server: Borg: 1.1.16  Python: CPython 3.9.2 msgpack: 0.5.6.+borg1
Borg server: PID: 10733  CWD: /root
Borg server: sys.argv: ['/usr/bin/borg', 'serve', '--debug']
Borg server: SSH_ORIGINAL_COMMAND: None
Platform: Linux 53f1bb57cd98 5.19.17-Unraid #2 SMP PREEMPT_DYNAMIC Wed Nov 2 11:54:15 PDT 2022 x86_64
Linux: Unknown Linux
Borg: 1.2.3  Python: CPython 3.11.2 msgpack: 1.0.4 fuse: llfuse 1.4.2 [pyfuse3,llfuse]
PID: 173  CWD: /
sys.argv: ['/usr/local/bin/borg', 'prune', '--keep-hourly', '2', '--keep-daily', '7', '--keep-weekly', '4', '--keep-monthly', '12', '--keep-yearly', '10', '--glob-archives', 'backup-*', '--stats', '--debug', '--show-rc', 'ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo']
SSH_ORIGINAL_COMMAND: None
terminating with error status, rc 2
Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --stats --debug --show-rc ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo' returned non-zero exit status 2.

Need some help? https://torsion.org/borgmatic/#issues

Expected behavior (if a bug)

Other notes / implementation ideas

Environment

borgmatic version: 1.7.9, server. none on the remote

Use sudo borgmatic --version or sudo pip show borgmatic | grep ^Version

borgmatic installation method: Docker container

Borg version: 1.2.3, server. 1.1.16 on the remote.

Use sudo borg --version

Python version: 3.11.2, server. 3.9.2 remote

Use python3 --version

Database version (if applicable): [version here]

Use psql --version or mysql --version on client and server.

operating system and version: Unraid and dietpi 8.15.2

Not really sure where to start to troubleshoot, so any help is appreciated.

When I type sudo apt-get install --only-upgrade borgbackup on the remote server I get this message
borgbackup is already the newest version (1.1.16-3)

#### What I'm trying to do and why run remote backup that has previously worked and it fails #### Steps to reproduce (if a bug) borgmatic prune create -v 1 --files --stats > /mnt/logs/backup-"`date +"%F+%T"`".log ``` location: source_directories: - /boot - /mnt/user/appdata - /mnt/user/nextcloud - /mnt/user/data - /mnt/user/photos repositories: - ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo one_file_system: true files_cache: mtime,size patterns: - '- [Tt]rash' - '- [Cc]ache' exclude_patterns: - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Crash Reports - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Logs - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Media - /mnt/user/appdata/Plex-Media-Server/Library/Application Support/Plex Media Server/Cache - /mnt/user/appdata/radarr/MediaCover - /mnt/user/appdata/sonarr/MediaCover - '.DS_Store' - '*.log' - '*.log.*' - '*.tmp' - '.Recycle.Bin' exclude_if_present: - .nobackup - .NOBACKUP storage: encryption_passphrase: "PASSPHRASE" compression: lz4 ssh_command: ssh -i /root/.ssh/Pi archive_name_format: 'backup-{now}' relocated_repo_access_is_ok: true retention: keep_hourly: 2 keep_daily: 7 keep_weekly: 4 keep_monthly: 12 keep_yearly: 10 prefix: 'backup-' consistency: checks: - repository - archives prefix: 'backup-' hooks: before_backup: - echo "Starting a backup." after_backup: - echo "External drive unmounted, finished a backup." on_error: - echo "Error during prune/create/check." mysql_databases: - name: all hostname: 192.168.1.94 username: USERNAME password: PASSWORD options: "--skip-comments" healthchecks: https://hc-ping.com/8e20df00-1a06-4de3-9b9b-6c0574aaa03b #borgmatic create -c "/etc/borgmatic.d/test.yaml" -v 2 --files --stats ``` #### Actual behavior (if a bug) Include (sanitized) `--verbosity 2` output if applicable. ``` ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo: Error running actions for repository Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --stats --debug --show-rc ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo' returned non-zero exit status 2. Error during prune/create/check. /etc/borgmatic.d/portable2TB.yaml: Error running configuration file summary: /etc/borgmatic.d/portable2TB.yaml: Error running configuration file ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo: Error running actions for repository ... res = f(**args) File "/usr/lib/python3/dist-packages/borg/repository.py", line 1122, in get self.index = self.open_index(self.get_transaction_id()) File "/usr/lib/python3/dist-packages/borg/repository.py", line 401, in get_transaction_id self.check_transaction() File "/usr/lib/python3/dist-packages/borg/repository.py", line 382, in check_transaction segments_transaction_id = self.io.get_segments_transaction_id() File "/usr/lib/python3/dist-packages/borg/repository.py", line 1276, in get_segments_transaction_id if self.is_committed_segment(segment): File "/usr/lib/python3/dist-packages/borg/repository.py", line 1311, in is_committed_segment if fd.read(self.header_fmt.size) != self.COMMIT: OSError: [Errno 5] Input/output error Borg server: Platform: Linux dietIoT 6.1.19-v7+ #1637 SMP Tue Mar 14 11:04:52 GMT 2023 armv7l Borg server: Linux: Unknown Linux Borg server: Borg: 1.1.16 Python: CPython 3.9.2 msgpack: 0.5.6.+borg1 Borg server: PID: 10733 CWD: /root Borg server: sys.argv: ['/usr/bin/borg', 'serve', '--debug'] Borg server: SSH_ORIGINAL_COMMAND: None Platform: Linux 53f1bb57cd98 5.19.17-Unraid #2 SMP PREEMPT_DYNAMIC Wed Nov 2 11:54:15 PDT 2022 x86_64 Linux: Unknown Linux Borg: 1.2.3 Python: CPython 3.11.2 msgpack: 1.0.4 fuse: llfuse 1.4.2 [pyfuse3,llfuse] PID: 173 CWD: / sys.argv: ['/usr/local/bin/borg', 'prune', '--keep-hourly', '2', '--keep-daily', '7', '--keep-weekly', '4', '--keep-monthly', '12', '--keep-yearly', '10', '--glob-archives', 'backup-*', '--stats', '--debug', '--show-rc', 'ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo'] SSH_ORIGINAL_COMMAND: None terminating with error status, rc 2 Command 'borg prune --keep-hourly 2 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --keep-yearly 10 --glob-archives backup-* --stats --debug --show-rc ssh://root@10.253.0.6/mnt/40F6E558F6E54EAA/borg_backup/repo' returned non-zero exit status 2. Need some help? https://torsion.org/borgmatic/#issues ``` #### Expected behavior (if a bug) #### Other notes / implementation ideas #### Environment **borgmatic version:** 1.7.9, server. none on the remote Use `sudo borgmatic --version` or `sudo pip show borgmatic | grep ^Version` **borgmatic installation method:** Docker container **Borg version:** 1.2.3, server. 1.1.16 on the remote. Use `sudo borg --version` **Python version:** 3.11.2, server. 3.9.2 remote Use `python3 --version` **Database version (if applicable):** [version here] Use `psql --version` or `mysql --version` on client and server. **operating system and version:** Unraid and dietpi 8.15.2 Not really sure where to start to troubleshoot, so any help is appreciated. When I type sudo apt-get install --only-upgrade borgbackup on the remote server I get this message borgbackup is already the newest version (1.1.16-3)

witten commented

2023-04-05 20:26:29 +00:00

Thanks for the detailed ticket. This error message is coming from Borg, and I believe indicates an error with Borg reading/writing the repository. I think the place to start is a borgmatic check. Then, if necessary, you can move to borgmatic check --repair. You might also want to do some hardware checks on either your client machine or the machine with the Borg repository or both. Hope this helps!

Side note: borgmatic has a --log-file option that you may be interested in.

Thanks for the detailed ticket. This error message is coming from Borg, and I believe indicates an error with Borg reading/writing the repository. I think the place to start is a `borgmatic check`. Then, if necessary, you can move to [`borgmatic check --repair`](https://borgbackup.readthedocs.io/en/stable/usage/check.html#description). You might also want to do some hardware checks on either your client machine or the machine with the Borg repository or both. Hope this helps! Side note: borgmatic has a `--log-file` option that you may be interested in.

witten added the

question / support

label 2023-04-05 20:26:39 +00:00

witten commented

2023-05-09 22:55:50 +00:00

I'm closing this for now, but feel free to reopen with any followup comments.

witten closed this issue

2023-05-09 22:55:50 +00:00

Sign in to join this conversation.