borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
14
58
Fork 42
Code Issues 58 Pull Requests 3 Actions Packages Releases 122 Activity

MySQL 'all' database not added to backup. #299

New Issue
Closed
opened 2020-03-19 13:50:27 +00:00 by NoxInmortus · 6 comments
NoxInmortus commented 2020-03-19 13:50:27 +00:00
Copy Link

Hello, I'm trying to setup borgmatic with a 'all' mysql_database dump.

Conf :

location:
    source_directories:
        - /var/www
    one_file_system: false
    files_cache: ctime,size,inode
    repositories:
        - /var/backups/borg
    exclude_from:
        - /etc/borgmatic/excludes
    exclude_caches: true
    exclude_if_present: .nobackup
    borgmatic_source_directory: /tmp/borgmatic

storage:
    encryption_passphrase: sOm37h1nG
    compression: lz4
    remote_rate_limit: 5000
    umask: 0077
    lock_wait: 5
    archive_name_format: '{hostname}-{now}'
    relocated_repo_access_is_ok: true

retention:
    keep_within: 2d
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 1
    keep_yearly: 1
    prefix: '{hostname}-'

consistency:
    checks:
        - repository
        - archives
        - extract
        - data

    check_last: 1
    prefix: '{hostname}-'

hooks:
    before_backup:
        - echo "`date` - Starting backup."

    after_backup:
        - echo "`date` - Finished backup."

    mysql_databases:
        - name: all
          options: --single-transaction --quick
          password: mypwd
          username: myuser

check:

root@lemp01:[/etc/borgmatic]: validate-borgmatic-config -c config.yaml 
All given configuration files are valid: config.yaml

Cron command used:

borgmatic --log-file /var/log/borg.log --log-file-verbosity 2 -c /etc/borgmatic/config.yaml

Relevant logs :

root@lemp01:[/var/log]: grep -i "mysql" borg.log 
[2020-03-19 11:42:04,963] DEBUG: /etc/borgmatic/config.yaml: Calling mysql_databases hook function dump_databases
[2020-03-19 11:42:04,964] INFO: /etc/borgmatic/config.yaml: Dumping MySQL databases
[2020-03-19 11:42:04,965] DEBUG: /etc/borgmatic/config.yaml: Dumping MySQL database all to /tmp/borgmatic/mysql_databases/localhost/all
[2020-03-19 11:42:04,966] DEBUG: mysqldump --add-drop-database --user backups --single-transaction --quick --all-databases > /tmp/borgmatic/mysql_databases/localhost/all
[2020-03-19 11:44:31,265] DEBUG: /etc/borgmatic/config.yaml: Calling mysql_databases hook function remove_database_dumps
[2020-03-19 11:44:31,269] INFO: /etc/borgmatic/config.yaml: Removing MySQL database dumps
[2020-03-19 11:44:31,269] DEBUG: /etc/borgmatic/config.yaml: Removing MySQL database dump all from /tmp/borgmatic/mysql_databases/localhost/all

But then, when I try to extract :

root@lemp01:[/var/log]: borgmatic extract --archive latest --path tmp/borgmatic/mysql_databases/*/*
/var/backups/borg: Error running actions for repository
Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T11:42:10 tmp/borgmatic/mysql_databases/*/*' returned non-zero exit status 1.
/etc/borgmatic/config.yaml: Error running configuration file

summary:
/etc/borgmatic/config.yaml: Error running configuration file
/var/backups/borg: Error running actions for repository
Include pattern 'tmp/borgmatic/mysql_databases/*/*' never matched.
Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T11:42:10 tmp/borgmatic/mysql_databases/*/*' returned non-zero exit status 1.

root@lemp01:[/var/log]: borg extract --umask 77 --lock-wait 5 --debug --list --show-rc /var/backups/borg::lemp01-2020-03-19T11:42:10 sh:tmp/borgmatic/mysql_databases/localhost/all
using builtin fallback logging configuration
35 self tests completed in 0.42 seconds
Verified integrity of /var/backups/borg/index.20
Enter passphrase for key /var/backups/borg: 
TAM-verified manifest
security: read previous location '/var/backups/borg'
security: read manifest timestamp '2020-03-19T10:43:29.871876'
security: determined newest manifest timestamp as 2020-03-19T10:43:29.871876
security: repository checks ok, allowing access
Include pattern 'tmp/borgmatic/mysql_databases/localhost/all' never matched.
terminating with warning status, rc 1

I dont see any dump file/tmp/mysql file with borg --list either. But I dont understand why, I get the /tmp/borgmatic/mysql_databases/localhost/all created fine.

Environnement:

Debian 10
borgmatic --version : 1.5.1
borg --version : borg 1.1.9

Debian 10 lemp01 4.19.53-mainline-rev1 #1 SMP Wed Jun 19 23:37:38 UTC 2019 aarch64 GNU/Linux

Python 3.7.3
mysql  Ver 15.1 Distrib 10.3.22-MariaDB, for debian-linux-gnu (aarch64) using readline 5.2

Any clue ?

Hello, I'm trying to setup borgmatic with a 'all' mysql_database dump. Conf : ``` location: source_directories: - /var/www one_file_system: false files_cache: ctime,size,inode repositories: - /var/backups/borg exclude_from: - /etc/borgmatic/excludes exclude_caches: true exclude_if_present: .nobackup borgmatic_source_directory: /tmp/borgmatic storage: encryption_passphrase: sOm37h1nG compression: lz4 remote_rate_limit: 5000 umask: 0077 lock_wait: 5 archive_name_format: '{hostname}-{now}' relocated_repo_access_is_ok: true retention: keep_within: 2d keep_daily: 7 keep_weekly: 4 keep_monthly: 1 keep_yearly: 1 prefix: '{hostname}-' consistency: checks: - repository - archives - extract - data check_last: 1 prefix: '{hostname}-' hooks: before_backup: - echo "`date` - Starting backup." after_backup: - echo "`date` - Finished backup." mysql_databases: - name: all options: --single-transaction --quick password: mypwd username: myuser ``` check: ``` root@lemp01:[/etc/borgmatic]: validate-borgmatic-config -c config.yaml All given configuration files are valid: config.yaml ``` Cron command used: ``` borgmatic --log-file /var/log/borg.log --log-file-verbosity 2 -c /etc/borgmatic/config.yaml ``` Relevant logs : ``` root@lemp01:[/var/log]: grep -i "mysql" borg.log [2020-03-19 11:42:04,963] DEBUG: /etc/borgmatic/config.yaml: Calling mysql_databases hook function dump_databases [2020-03-19 11:42:04,964] INFO: /etc/borgmatic/config.yaml: Dumping MySQL databases [2020-03-19 11:42:04,965] DEBUG: /etc/borgmatic/config.yaml: Dumping MySQL database all to /tmp/borgmatic/mysql_databases/localhost/all [2020-03-19 11:42:04,966] DEBUG: mysqldump --add-drop-database --user backups --single-transaction --quick --all-databases > /tmp/borgmatic/mysql_databases/localhost/all [2020-03-19 11:44:31,265] DEBUG: /etc/borgmatic/config.yaml: Calling mysql_databases hook function remove_database_dumps [2020-03-19 11:44:31,269] INFO: /etc/borgmatic/config.yaml: Removing MySQL database dumps [2020-03-19 11:44:31,269] DEBUG: /etc/borgmatic/config.yaml: Removing MySQL database dump all from /tmp/borgmatic/mysql_databases/localhost/all ``` But then, when I try to extract : ``` root@lemp01:[/var/log]: borgmatic extract --archive latest --path tmp/borgmatic/mysql_databases/*/* /var/backups/borg: Error running actions for repository Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T11:42:10 tmp/borgmatic/mysql_databases/*/*' returned non-zero exit status 1. /etc/borgmatic/config.yaml: Error running configuration file summary: /etc/borgmatic/config.yaml: Error running configuration file /var/backups/borg: Error running actions for repository Include pattern 'tmp/borgmatic/mysql_databases/*/*' never matched. Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T11:42:10 tmp/borgmatic/mysql_databases/*/*' returned non-zero exit status 1. root@lemp01:[/var/log]: borg extract --umask 77 --lock-wait 5 --debug --list --show-rc /var/backups/borg::lemp01-2020-03-19T11:42:10 sh:tmp/borgmatic/mysql_databases/localhost/all using builtin fallback logging configuration 35 self tests completed in 0.42 seconds Verified integrity of /var/backups/borg/index.20 Enter passphrase for key /var/backups/borg: TAM-verified manifest security: read previous location '/var/backups/borg' security: read manifest timestamp '2020-03-19T10:43:29.871876' security: determined newest manifest timestamp as 2020-03-19T10:43:29.871876 security: repository checks ok, allowing access Include pattern 'tmp/borgmatic/mysql_databases/localhost/all' never matched. terminating with warning status, rc 1 ``` I dont see any dump file/tmp/mysql file with `borg --list` either. But I dont understand why, I get the `/tmp/borgmatic/mysql_databases/localhost/all` created fine. Environnement: ``` Debian 10 borgmatic --version : 1.5.1 borg --version : borg 1.1.9 Debian 10 lemp01 4.19.53-mainline-rev1 #1 SMP Wed Jun 19 23:37:38 UTC 2019 aarch64 GNU/Linux Python 3.7.3 mysql Ver 15.1 Distrib 10.3.22-MariaDB, for debian-linux-gnu (aarch64) using readline 5.2 ``` Any clue ?
witten commented 2020-03-19 16:16:10 +00:00
Owner
Copy Link

Thanks for filing this and providing the detailed logs. Based on what you've included here, it sounds like something is going wrong with the the source directory paths that borgmatic passes borg create. So one thing that would be helpful is to get a look at the borg create invocation that results from your borgmatic command:

borgmatic --log-file /var/log/borg.log --log-file-verbosity 2 -c /etc/borgmatic/config.yaml

That would give me an idea if the source directory paths are incorrect, and whether it has anything to do with the custom borgmatic_source_directory. Thanks!

Thanks for filing this and providing the detailed logs. Based on what you've included here, it sounds like something is going wrong with the the source directory paths that borgmatic passes `borg create`. So one thing that would be helpful is to get a look at the `borg create` invocation that results from your borgmatic command: > `borgmatic --log-file /var/log/borg.log --log-file-verbosity 2 -c /etc/borgmatic/config.yaml` That would give me an idea if the source directory paths are incorrect, and whether it has anything to do with the custom `borgmatic_source_directory`. Thanks!
witten added the
bug
 label 2020-03-19 16:16:38 +00:00
NoxInmortus commented 2020-03-19 16:39:24 +00:00
Author
Copy Link

Hello @witten, thanks for your time.

I restarted a new job and there is the log (could not upload it here) : https://framadrop.org/r/qBMe4eXNeQ#jS7+bZ4PYCnLFJibO+fQpS9t0oV7Ba6uLls1mMVD0/c=

I have deleted some fqdn names, you may see weird paths after /var/www

I also noticed something, it seems to see the mysql_databases folder in some way (I looked for /tmp/borgmatic/* and it returns /tmp/borgmatic/mysql_databases) :

root@lemp01:[~]: borgmatic extract --archive latest --path /tmp/borgmatic/*
/var/backups/borg: Error running actions for repository
Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T16:06:23 /tmp/borgmatic/mysql_databases' returned non-zero exit status 1.
/etc/borgmatic/config.yaml: Error running configuration file

summary:
/etc/borgmatic/config.yaml: Error running configuration file
/var/backups/borg: Error running actions for repository
Include pattern '/tmp/borgmatic/mysql_databases' never matched.
Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T16:06:23 /tmp/borgmatic/mysql_databases' returned non-zero exit status 1.
Hello @witten, thanks for your time. I restarted a new job and there is the log (could not upload it here) : https://framadrop.org/r/qBMe4eXNeQ#jS7+bZ4PYCnLFJibO+fQpS9t0oV7Ba6uLls1mMVD0/c= I have deleted some fqdn names, you may see weird paths after /var/www I also noticed something, it seems to see the `mysql_databases` folder in some way (I looked for `/tmp/borgmatic/*` and it returns `/tmp/borgmatic/mysql_databases`) : ``` root@lemp01:[~]: borgmatic extract --archive latest --path /tmp/borgmatic/* /var/backups/borg: Error running actions for repository Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T16:06:23 /tmp/borgmatic/mysql_databases' returned non-zero exit status 1. /etc/borgmatic/config.yaml: Error running configuration file summary: /etc/borgmatic/config.yaml: Error running configuration file /var/backups/borg: Error running actions for repository Include pattern '/tmp/borgmatic/mysql_databases' never matched. Command 'borg extract --umask 77 --lock-wait 5 /var/backups/borg::lemp01-2020-03-19T16:06:23 /tmp/borgmatic/mysql_databases' returned non-zero exit status 1. ```
witten commented 2020-03-20 05:09:39 +00:00
Owner
Copy Link

Well, looking at the log you attached, I don't see anything wrong with the borg create command:

[2020-03-19 17:27:38,706] DEBUG: borg create --exclude-from /etc/borgmatic/excludes --exclude-caches --exclude-if-present .nobackup --compression lz4 --remote-ratelimit 5000 --files-cache ctime,size,inode --umask 77 --lock-wait 5 --debug --show-rc /var/backups/borg::{hostname}-{now} /var/www /tmp/borgmatic

Which is a little mystifying, because with that invocation, I'd expect it to bundle up and include all the contents of /tmp/borgmatic including /tmp/borgmatic/mysql_databases.

One additional thing you can try is running the same borgmatic invocation with the new --files flag included. That should give you a full accounting of absolutely every file that's included in the archive as it's added. I'd be interested to know: Is anything from /tmp/borgmatic showing up in that output?

I also noticed something, it seems to see the mysql_databases folder in some way (I looked for /tmp/borgmatic/* and it returns /tmp/borgmatic/mysql_databases) :

root@lemp01:[~]: borgmatic extract --archive latest --path /tmp/borgmatic/*

I think what's going on here is that your shell is expanding /tmp/borgmatic/* to /tmp/borgmatic/mysql_databases and passing that to borgmatic, which is in turn passing it to Borg, which is complaining that it doesn't match anything in the archive. So it's only seeing the folder as present on your host.. not in the archive. Because it's not present in the archive!

Well, looking at the log you attached, I don't see anything wrong with the `borg create` command: > `[2020-03-19 17:27:38,706] DEBUG: borg create --exclude-from /etc/borgmatic/excludes --exclude-caches --exclude-if-present .nobackup --compression lz4 --remote-ratelimit 5000 --files-cache ctime,size,inode --umask 77 --lock-wait 5 --debug --show-rc /var/backups/borg::{hostname}-{now} /var/www /tmp/borgmatic` Which is a little mystifying, because with that invocation, I'd expect it to bundle up and include all the contents of `/tmp/borgmatic` including `/tmp/borgmatic/mysql_databases`. One additional thing you can try is running the same `borgmatic` invocation with the new `--files` flag included. That should give you a full accounting of absolutely every file that's included in the archive as it's added. I'd be interested to know: Is anything from `/tmp/borgmatic` showing up in that output? > I also noticed something, it seems to see the `mysql_databases` folder in some way (I looked for `/tmp/borgmatic/*` and it returns `/tmp/borgmatic/mysql_databases`) : > `root@lemp01:[~]: borgmatic extract --archive latest --path /tmp/borgmatic/*` I think what's going on here is that your shell is expanding `/tmp/borgmatic/*` to `/tmp/borgmatic/mysql_databases` and passing that to borgmatic, which is in turn passing it to Borg, which is complaining that it doesn't match anything in the archive. So it's only seeing the folder as present on your host.. not in the archive. Because it's not present in the archive!
NoxInmortus commented 2020-03-20 10:18:01 +00:00
Author
Copy Link

@witten thanks for your insight, it seems indeed the shell was expanding (for /tmp/borgmatic/mysql_databases).

I do not understand what you said about the same borgmatic invocation with new --files flag included, I did not succeded to find any --files option with borgmatic or borg.

But in the log I previously provided, we can see every file added to the archive (with the INFO log level), and I did not see any /tmp/borgmatic included... :(

@witten thanks for your insight, it seems indeed the shell was expanding (for `/tmp/borgmatic/mysql_databases`). I do not understand what you said about `the same borgmatic invocation with new --files flag included`, I did not succeded to find any `--files` option with borgmatic or borg. But in the log I previously provided, we can see every file added to the archive (with the INFO log level), and I did not see any `/tmp/borgmatic` included... :(
NoxInmortus commented 2020-03-21 12:02:15 +00:00
Author
Copy Link

@witten My sincere appologies for your lost time...

I wake up this morning almost shouting "what a dumpfuck!!".

I had /tmp in my exclude file...

Removed it and I was good... :(

@witten My sincere appologies for your lost time... I wake up this morning almost shouting "what a dumpfuck!!". I had /tmp in my `exclude` file... Removed it and I was good... :(
witten commented 2020-03-21 20:40:28 +00:00
Owner
Copy Link

No worries! I'm glad you figured it out, and that it was only a /tmp exclude.

No worries! I'm glad you figured it out, and that it was only a `/tmp` exclude.
witten added
question / support
 and removed
bug
 labels 2020-03-21 20:41:04 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
spot-check
1.8.11
1.8.10
1.8.9
1.8.8
1.8.7
1.8.6
1.8.5
1.8.4
1.8.3
1.8.2
1.8.1
1.8.0
1.7.15
1.7.14
1.7.13
1.7.12
1.7.11
1.7.10
1.7.9
1.7.8
1.7.7
1.7.6
1.7.5
1.7.4
1.7.3
1.7.2
1.7.1
1.7.0
1.6.6
1.6.5
1.6.4
1.6.3
1.6.2
1.6.1
1.6.0
1.5.24
1.5.23
1.5.22
1.5.21
1.5.20
1.5.19
1.5.18
1.5.17
1.5.16
1.5.15
1.5.14
1.5.13
1.5.12
1.5.11
1.5.10
1.5.9
1.5.8
1.5.7
1.5.7.dev0
1.5.6
1.5.5
1.5.4
1.5.3
1.5.2
1.5.1
1.5.0
1.4.22
1.4.21
1.4.20
1.4.19
1.4.18
1.4.17
1.4.16
1.4.15
1.4.14
1.4.13
1.4.12
1.4.11
1.4.10
1.4.9
1.4.8
1.4.7
1.4.6
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.26
1.3.25
1.3.24
1.3.23
1.3.22
1.3.21
1.3.20
1.3.19
1.3.18
1.3.17
1.3.16
1.3.15
1.3.14
1.3.13
1.3.12
1.3.11
1.3.10
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.18
1.2.17
1.2.16
1.2.15
1.2.14
1.2.13
1.2.12
1.2.11
1.2.10
1.2.9
1.2.8
1.2.7
1.2.6
1.2.5
1.2.4
1.2.3
1.2.2
1.2.1
1.2.0
1.1.15
1.1.14
1.1.13
1.1.12
1.1.11
1.1.10
1.1.9
1.1.7
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
1.1.1
1.1.0
1.0.3
1.0.2
1.0.1
1.0.0
0.1.8
0.1.7
0.1.6
0.1.5
0.1.4
0.1.3
0.1.2
0.1.1
0.1.0
0.0.7
0.0.6
0.0.5
0.0.4
0.0.3
0.0.2
Labels
Clear labels   
bug

   
data loss

   
design finalized

   
good first issue

   
new feature area

   
question / support

   
security

   
waiting for response
No Label
bug
data loss
design finalized
good first issue
new feature area
question / support
security
waiting for response
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: borgmatic-collective/borgmatic#299
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?