Dr. Stefan Schimanski e98c8e7685 Fix deadlocks and race conditions in mesos master election notifier ...

- n.node used the n.lock as underlaying locker. The service loop initially
  locked it, the Notify function tried to lock it before calling n.node.Signal,
  leading to a dead-lock.
- the go routine calling ChangeMaster was not synchronized with the Notify
  method. The former was triggering change events that the later never saw
  when the former's startup was faster that of Notify. Hence, not even a single
  event was noticed and not even a single start/stop call of the slow service
  was triggered.

This patch replaces the n.node condition object with a simple channel n.changed.
The service loop watches it.

Updating the notified private variables is still protected with n.lock against
races, but independently of the n.changed channel. Hence, the deadlock is gone.

Moreover, the startup of the Notify loop is synchronized with the go routine which
changes the master. Hence, the Notify loop will see the master changes.

Fixes #10776

2015-07-15 21:45:53 +02:00

2.5 KiB

Raw Blame History

View Raw