Impossibility of Distributed Consensus with One Faulty Process

Suvash Thapaliya - Papers We Love, Gothenburg

27th April, 2017

Also called as the FLP impossibility result.

 

Published in April 1985 by Fischer, Lynch and Patterson.

 

Eventually won the Dijkstra Prize in 2001.

(given to outstanding papers on the principles of distributed computing)

Proved the impossibility of consensus using asynchronous communication.

settled a dispute that had been ongoing in distributed systems discussions.

The FLP result shows that in an asynchronous setting, where even if only one processor might crash, there is no distributed algorithm that solves the consensus problem.

By contrast, solutions were/are known for the synchronous case.

Some definitions

Processors : Voters/Processes trying to agree on a value.

 

Consensus: comes in various flavors of 'strength'.

(solution to a stronger problem will solve weaker problems at the same time.)

 

A strong form of consensus is as follows:

Given a set of processors, each with an initial value:

- All non-faulty* processes eventually decide on a value. (Termination)
- All processes that decide do so on the same value. (Agreement)
- The value that has been decided must have been proposed by some process. (Validity)

*Faulty nodes are not expected of any specific behaviour.

FLP result concerns a 'weaker' form of  consensus :

- for termination it is enough only that some non-faulty process decides. 

 

The motivation being that, if there's no solution to a weaker problem, there's def. no solution to an even stronger one.

 

Hence, the application of the proof being more widely applicable.

Asynchronous processors

 

- no bounds on the amount of time a processor might take to complete its work and then respond with a message. 

 

- processes do not have access to synchronised clocks. (time-outs based algos. not usable)

 

- impossible to tell whether the process has died or is running very slowly.

System Model

- Messaging system is reliable. (correct + exactly once)

System Model

The communication links between processors are assumed to be reliable. It was well known that given arbitrarily unreliable links no solution for consensus could be found even in a synchronous model; therefore by assuming reliable links and yet still proving consensus impossible in an asynchronous system the FLP result is made stronger.

- Processors fail via the Fail-stop model (simply stop working)

(no byzantine failures etc.)

System Model

System Model

if a given problem is proven unsolvable in a very restricted environment where only a few things can go wrong, it will certainly be unsolvable in an environment where yet more things can go wrong.

System Model

Processors are modelled as automata with infinite states that communicate via messages, and have a message buffer to receive messages.

 

In one atomic step, a process can

- attempt to receive a message

- perform local computation based on the rec. message

- send an arbitrary(but finite) set of messages to other processes.

System Model : Some definitions

For the sake of clarity, and introducing some more vocabulary.

- Message is a pair (p, m) : p -> destination process, m -> message contents.

- Configuration : Internal state of all the processors in the system.

- Step : Moves the system from one config. to another.

- Event : e = (p, m) which was received/processed to move to the next step.

- Schedule : Finite or Infinite sequence of events that can be applied to a starting configuration

- Run : Sequence of steps taken to realise the schedule is called a run

System Model : Some definitions

Non-faulty processes take infinitely many steps in a run (presumably eventually just receiving Null/Theta once the algorithm has finished its work) – otherwise a process is considered faulty.

An admissible run is one where at most one process is faulty (capturing the failure requirements of the system model) and every message is eventually delivered (this means that every processor eventually gets chosen to receive infinitely many times).

We say that a run is deciding provided that some process eventually decides according to the properties of consensus.

....and that a consensus protocol is totally correct if every admissible run is a deciding run.

The idea behind the Proof

The authors present their main theorem which is that no totally correct consensus algorithm exists.

 

The idea behind it is to show that there is some admissible run – i.e. one with only one processor failure and eventual delivery of every message – that is not a deciding run – i.e. in which no processor eventually decides.

 

This is enough to prove the general impossibility of consensus – it’s enough to show that there’s just one initial configuration in which a given protocol will not work because starting in that configuration can never be ruled out, and the result is a protocol which runs for ever (because no processor decides).

The Proof

will not walk through the proof. (as it's quite involved and doesn't flow from my head)

 

Instead, will try to share my intuition about it.

The Proof

the basic idea is to show circumstances under which the protocol remains forever indecisive.

 

2 steps presented as two separate lemmas

- first lemma demonstrate the existence of an initial condition.

- second lemma exploits the result from the first one.

- they are then tied together to demonstrate the above mentioned idea.

The Proof : First Lemma

this is the Lemma 2 in the paper.

The point of the first lemma is to show that there is some initial configuration in which the decision is not predetermined, but in fact arrived as a result of the sequence of steps taken and the occurrence of any failure.

For example, say that this initial configuration was two processors whose initial value was ‘0’ and one whose value was ‘1’.

The authors show that what is decided from this configuration depends on the order in which messages are received and whether any of the processors fail, not just what the initial values of the processors are.

This is due to the inherent non-determinism of the asynchronous system model.

We call the configurations that may lead to either decision value bivalent, and configurations that will only result in one value 0-valent or 1-valent.

The Proof : Second Lemma

this is the Lemma 3 in the paper.

The statement of the lemma says, informally, that if a protocol starts from a bivalent configuration and there is a message e that is applicable to that configuration, then the set of configurations reachable through any sequence of messages where e is applied last contains a bivalent configuration.

 

More intuitively, the lemma proves that there is a path from a bivalent (read ‘undecided’) configuration to another bivalent configuration, done by delaying a message long enough which raises the possibility that an infinite loop will ensue and the protocol will remain forever undecided.

 

The Proof : Bringing it together

- As a result of first lemma, we start with a bivalent configuration.

- Based on the second lemma, we can see that if we start with a bivalent configuration, it is provable that there is path to end up in a bivalent configuration.

 

This can go on forever, which causes the algorithm to wait indefinitely, hence being 'impossible' to arrive to a consensus.

Why does all this matter ?

This impossibility result is important because it highlights that assuming the asynchronous system model leads to a tradeoff.

 

Algorithms that solve the consensus problem must either give up safety or liveness when the guarantees regarding bounds on message delivery do not hold.

 

particularly relevant to people who design algorithms, because it imposes a hard constraint on the problems that we know are solvable in the asynchronous system model.

 

Why does this mean practically ?

Because, we now know the conditions that can cause a distributed system to end up in a non decisive loop, that possibility can be made extremely small, by working to reduce the premises that can cause it.

 

In practice, most real distributed consensus systems aren't always truly async.

 

Having said that, it's good to keep this is mind when one sees long running livelocks during consensus. (because the network is not unreliable all the time.)

 

- more interesting than CAP or Byzantine problems, because it proves impossibility even for relaxed conditions (weaker consensus).

Subsequent work

This paper spawned a huge amount of subsequent work in the theoretical distributed systems community with different "refined" system models.

 

Some of them being

- Randomized algorithms

- Failure detectors

- Partial Synchrony

....

Conclusion

These results do not show that such problems cannot be "solved" in practice` rather, thy point up the need for more refined models of distributed computing that better reflect realistic assumptions about processor and communication timings, and for less stringent requirements on the solution to such problems.