borgmatic-collective/borgmatic
borgmatic-collective
/
borgmatic
15
57
Fork 40
Code Issues 54 Pull Requests 1 Actions Packages Releases 121 Activity

WIP: Add locking of borgmatic config file #254

Closed
drewkett wants to merge 9 commits from (deleted):configlock into main
pull from: (deleted):configlock
merge into: borgmatic-collective:main
Conversation 5 Commits 9 Files Changed 2 +9 -17
2 changed files with 9 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 4fb6a49b33 - Show all commits

24
borgmatic/commands/borgmatic.py
View File

@ -48,6 +48,14 @@ def run_configuration(config_filename, config, arguments):
)
global_arguments = arguments['global']
if location.get("lock_client",False):
lock_f = open(config_filename)
try:
fcntl.flock(lock_f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
witten commented 2019-12-03 21:22:38 +00:00
Outdated
Review
Copy Link

Forgive me if this is a duplicate comment.. Is there a reason this is non-blocking? Seems like we'd want it to synchronously lock to be absolutely sure that no other borgmatic is running.

Forgive me if this is a duplicate comment.. Is there a reason this is non-blocking? Seems like we'd want it to synchronously lock to be absolutely sure that no other borgmatic is running.
drewkett commented 2019-12-04 03:29:22 +00:00
Outdated
Review
Copy Link

Non blocking here indicates that we don’t want to block the borgmatic and wait for a lock to be freed if the file is currently locked. I think you can do blocking with a timeout but it doesn’t seem that beneficial to me since borgmatic is typically a long running operation so the lock is unlikely to be free in a short amount of time. I think borg uses a timeout so I guess we could replicate that behavior.

Non blocking here indicates that we don’t want to block the borgmatic and wait for a lock to be freed if the file is currently locked. I think you can do blocking with a timeout but it doesn’t seem that beneficial to me since borgmatic is typically a long running operation so the lock is unlikely to be free in a short amount of time. I think borg uses a timeout so I guess we could replicate that behavior.
witten commented 2019-12-04 05:27:24 +00:00
Outdated
Review
Copy Link

Oh, gotcha. I originally misunderstood what LOCK_NB did. I see now that you're using it here to error immediately if a lock is in place, which makes sense. Carry on!

Oh, gotcha. I originally misunderstood what `LOCK_NB` did. I see now that you're using it here to error immediately if a lock is in place, which makes sense. Carry on!
except IOError:
logger.critical("Failed to acquire lock for {}".format(config_filename))
sys.exit(1)
witten commented 2019-12-03 21:26:29 +00:00
Outdated
Review
Copy Link

Given that this is in run_configuration() now, you can just:

except IOError as error:
    yield from make_error_log_records(
        '{}: Failed to acquire lock'.format(config_filename), error
    )
    return

... and then this would go through the rest of the borgmatic logging/error/summary machinery.

Another option would be this:

except IOError as error:
    encountered_error = error
    yield from make_error_log_records(
        '{}: Failed to acquire lock'.format(config_filename), error
    )

Which would fall through and trigger the on_error hook, if that's what you want to do in this case.

Given that this is in `run_configuration()` now, you can just: ``` except IOError as error: yield from make_error_log_records( '{}: Failed to acquire lock'.format(config_filename), error ) return ``` ... and then this would go through the rest of the borgmatic logging/error/summary machinery. Another option would be this: ``` except IOError as error: encountered_error = error yield from make_error_log_records( '{}: Failed to acquire lock'.format(config_filename), error ) ``` Which would fall through and trigger the `on_error` hook, if that's what you want to do in this case.
witten commented 2019-12-09 18:00:58 +00:00
Outdated
Review
Copy Link

Make sure you run tests with tox, as the code formatter will probably complain about this. To have it reformat things for you: tox -e black is all you need to do.

Make sure you run tests with `tox`, as the code formatter will probably complain about this. To have it reformat things for you: `tox -e black` is all you need to do.
local_path = location.get('local_path', 'borg')
remote_path = location.get('remote_path')
borg_environment.initialize(storage)
@ -567,25 +575,9 @@ def main(): # pragma: no cover
logger.debug('Ensuring legacy configuration is upgraded')
convert.guard_configuration_upgraded(LEGACY_CONFIG_PATH, config_filenames)
locks = []
for config_filename,config in configs.items():
lock_config = config.get("location",dict()).get("lock_config",False)
if lock_config:
f = open(config_filename)
try:
fcntl.flock(f.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
logger.critical("Failed to acquire lock for {}".format(config_filename))
sys.exit(0)
locks.append(f)
summary_logs = parse_logs + list(collect_configuration_run_summary_logs(configs, arguments))
summary_logs_max_level = max(log.levelno for log in summary_logs)
# this removes the reference to the open files which python will garbage collect the file objects
# and close them
locks = None
for message in ('', 'summary:'):
log_record(
levelno=summary_logs_max_level,

2
borgmatic/config/schema.yaml
View File

@ -32,7 +32,7 @@ map:
type: bool
desc: Stay in same file system (do not cross mount points). Defaults to false.
example: true
lock_config:
lock_client:
type: bool
desc: Lock config when running borgmatic to prevent multiple instances from running simultaneously
witten commented 2019-12-09 18:04:46 +00:00
Outdated
Review
Copy Link

You could add: ". Defaults to false." here. See other descriptions for an example.

You could add: ". Defaults to false." here. See other descriptions for an example.
example: true
witten commented 2019-12-03 01:01:31 +00:00
Review
Copy Link

The fact that it's locking the configuration file might just be an implementation detail as far as the user is concerned. What about calling this something like lock_client, lock_borgmatic, etc? I don't feel strongly about this in any case.

The fact that it's locking the configuration file might just be an implementation detail as far as the user is concerned. What about calling this something like `lock_client`, `lock_borgmatic`, etc? I don't feel strongly about this in any case.
drewkett commented 2019-12-03 06:52:39 +00:00
Review
Copy Link

Those sound better. I’ll fix it when I get the chance

Those sound better. I’ll fix it when I get the chance
drewkett commented 2019-12-03 19:28:14 +00:00
Review
Copy Link

I renamed it to lock_client and left it as a bool for now. I didn't see immediately the best way to do it with the config (and validation), but if you want to keep options open switching this to something like lock: "config" so that then you could add a lock: "everything option later would be an option.

I renamed it to `lock_client` and left it as a bool for now. I didn't see immediately the best way to do it with the config (and validation), but if you want to keep options open switching this to something like `lock: "config"` so that then you could add a `lock: "everything` option later would be an option.
drewkett commented 2019-12-03 19:29:27 +00:00
Review
Copy Link

I also wasn't sure if the example line is supposed to be the default value or not.

I also wasn't sure if the example line is supposed to be the default value or not.
witten commented 2019-12-03 21:19:54 +00:00
Review
Copy Link

A bool is fine for now, as far as I'm concerned.

example: is not usually the default value. In fact it's often the opposite of the default value in that it's demonstrating how you would override the default if you are so inclined. So example: true sounds right to me here, given that I'm assuming the default is false (don't lock).

A bool is fine for now, as far as I'm concerned. `example:` is not usually the default value. In fact it's often the *opposite* of the default value in that it's demonstrating how you would override the default if you are so inclined. So `example: true` sounds right to me here, given that I'm assuming the default is `false` (don't lock).