An Incomplete History of Concurrency - Leslie Lamport

Leslie Lamport explains the history of concurrency from 1965 1977 viewed from 2014
and explained with 40 years of hindsight. Concurrency has gone by many names such as parallel
computing, concurrent programing, multiprogramming, or distributed computing. However,
Lamport will simply call it concurrency

The field of concurrency began with Edsger Dijkstra and his paper Solution of a Problem
in Concurrent Control (1965). In the paper he introduced the mutual exclusion problem in which
N processes execute a portion of code called the critical section. The processes must synchronize
by reading and writing shared memory such that two critical sections arent executed at the same
time in order to prevent data corruption and if a process is waiting to execute a section then it
will. The computer can also pause outside code in the critical section and when synchronizing
code. Dijkstra made no real time assumption, just that each non halted processes will continue to
take steps. This is because at the time computers only had one processor. Multiprocessing at this
time was simply one processor shared among processes. When one process is executed all other
processes did nothing.

In 1972 Leslie Lamport learned about the mutual exclusion problem. Lamport thought
the problem wasnt that hard so he created an algorithm and sent it to the ACM. The ACM told
him the algorithm was wrong which in turned made Lamport determined to solve the problem
and taught him the importance of rigorous proofs for algorithms.

The bakery algorithm was created by Leslie Lamport. The idea is similar to a bakery in
which a process waiting to execute its critical section takes a numbered ticket and the process
with the lowest ticket number executes its code. The algorithm, instead of a tradition bakery shop
of taking a number ticket, each process created its own ticket by reading the ticket of all the other
processes and creating a ticket number one larger than the largest ticket. This is because a
process that creates tickets for the other process would be a non-halt able process. A problem
with the algorithm is that the ticket numbers can grow without bounds to the point in which the
ticket number must be stored in multiple memory words and multiple memory words cannot be
written or read atomically (instantly). However, when writing the proof, he discovered that only
the write must be correct, the read can be arbitrary. This meant that the algorithm implements
mutual exclusion without any lower level mutual exclusion, something which was considered
impossible. Proof means mathematical reasoning and requires a mathematical of the algorithm.
In Djikstras implicit model he states that an execution is represented as a sequence of states.
Lamport calls this the standard model as is assuming atomic actions. If Lamport followed this
model than he would not have discovered that non atomic reads and writes can work.

Lamport then describes the two arrow model in which X arrow Y means is true if X ends
before Y begins and X dash arrow Y means the is true if X begins before Y ends. Using these,
the assumptions made about the algorithm are that operation of a single process are ordered by a
solid arrow and for reads and writes they are either: the read begins before the write ends or the
write ends before the read begins. In the bakery algorithm the memory words area written by a
single process or that one write is preceded by the next write. If a read precedes a write and
doesnt overlap, then it is reading the correct value.

The fundamental problem of interprocess communication is to ensure that an operation is

preceded by another operation. This is done in some multicore processors by providing a
memory barrier. Process a is executed than a memory barrier then process b is executed to ensure
process a precedes process b. Inteprocess synchronization requires that one operation proceeds
another operation in different processes. However, interprocess communication can only ensure
that an operation starts before the other has finish. If one operation can see the effect of another
operation the only conclusion that can be made is that the operation started before the other has
finished. A interprocess solid arrow relation however can be deduced from interprocess and
intraprocess relations. In modern multicore processes multiple operations are executed at the
same time. The memory barrier is used the get instructions to happen one after another in the
same process.

Producer-Consumer Synchronization is another fundamental concurrency problem

identified by Edsger Djisktra in 1965. Lamport describes the problem as a bounded FIFI queue
(first in first out). A bounded FIFO queue has an infinite sequence of input values, a buffer than
can hold up to n values, and a sequence of output values. There are two processes the producer
and consumer process. The producer process moves values form the input to the buffer and the
consumer processes moves vales from the buffer to the output.