Sei sulla pagina 1di 186

The Relative Power of Synchronization Operations

Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit

Shared-Memory Computability?

Shared Memory

10011

Mathematical model of concurrent computation What is (and is not) concurrently computable Efficiency (mostly) irrelevant
Art of Multiprocessor 2 Programming

Wait-Free Implementation
Every method call completes in finite number of steps Implies no mutual exclusion

Art of Multiprocessor Programming

From Weakest Register


Single writer

Single reader

0 1
Safe Boolean register
Art of Multiprocessor Programming 4

All the way to a Wait-free Implementation of Atomic Snapshots


MRMW MRSW SRSW

M-valued Boolean

Safe Regular Atomic Snapshot


Art of Multiprocessor Programming 5

Rationale for wait-freedom


We wanted atomic registers to implement mutual exclusion

Art of Multiprocessor Programming

Rationale for wait-freedom


We wanted atomic registers to implement mutual exclusion So we couldnt use mutual exclusion to implement atomic registers

Art of Multiprocessor Programming

Rationale for wait-freedom


We wanted atomic registers to implement mutual exclusion So we couldnt use mutual exclusion to implement atomic registers But wait, theres more!

Art of Multiprocessor Programming

Why is Mutual Exclusion so wrong?

Art of Multiprocessor Programming

Asynchronous Interrupts
??? ???

Art of Multiprocessor Programming

10

Heterogeneous Processors
??? ???
yawn

supercomputer

supercomputer
Art of Multiprocessor Programming

toaster
11

Fault-tolerance
??? ???

Art of Multiprocessor Programming

12

Machine Level Instruction Granularity


Amdahls Law

Art of Multiprocessor Programming

13

Basic Questions
Wait-Free synchronization might be a good idea in principle

Art of Multiprocessor Programming

14

Basic Questions
Wait-Free synchronization might be a good idea in principle But how do you do it

Art of Multiprocessor Programming

15

Basic Questions
Wait-Free synchronization might be a good idea in principle But how do you do it
Systematically?

Art of Multiprocessor Programming

16

Basic Questions
Wait-Free synchronization might be a good idea in principle But how do you do it
Systematically? Correctly?

Art of Multiprocessor Programming

17

Basic Questions
Wait-Free synchronization might be a good idea in principle But how do you do it
Systematically? Correctly? Efficiently?

Art of Multiprocessor Programming

18

FIFO Queue: Enqueue Method

q.enq( )

Art of Multiprocessor Programming

19

FIFO Queue: Dequeue Method

q.deq()/

Art of Multiprocessor Programming

20

Two-Thread Wait-Free Queue


head public class WaitFreeQueue { 0 1 capacity-1 int head = 0, tail = 0; y z Item[QSIZE] items; public void enq(Item x) { while (tail-head == QSIZE) {}; items[tail % QSIZE] = x; tail++; } public Item deq() { while (tail-head == 0) {} Item item = items[head % QSIZE]; head++; return item; }} Art of Multiprocessor Programming tail
2

21

What About Multiple Dequeuers?

Art of Multiprocessor Programming

22

Grand Challenge

Implement a FIFO queue

Art of Multiprocessor Programming

23

Grand Challenge

Implement a FIFO queue


Wait-free

Art of Multiprocessor Programming

24

Grand Challenge

Implement a FIFO queue


Wait-free Linearizable

Art of Multiprocessor Programming

25

Grand Challenge

Implement a FIFO queue


Wait-free Linearizable From atomic read-write registers
Art of Multiprocessor Programming 26

Grand Challenge

Implement a FIFO queue


Wait-free Linearizable From atomic read-write registers Multiple dequeuers
Art of Multiprocessor Programming 27

Grand Challenge

Only new Implement a FIFO queue aspect


Wait-free Linearizable From atomic read-write registers Multiple dequeuers
Art of Multiprocessor Programming

28

Puzzle
While you are ruminating on the grand challenge We will give you another puzzle Consensus!

Art of Multiprocessor Programming

29

Consensus: Each Thread has a Private Input

32

19

21

Art of Multiprocessor Programming

30

They Communicate

Art of Multiprocessor Programming

31

They Agree on One Threads Input

19

19

19

Art of Multiprocessor Programming

32

Formally: Consensus
Consistent:
all threads decide the same value

Art of Multiprocessor Programming

33

Formally: Consensus
Consistent:
all threads decide the same value

Valid:
the common decision value is some thread's input

Art of Multiprocessor Programming

34

No Wait-Free Implementation of Consensus using Registers ??? ???

Art of Multiprocessor Programming

35

Formally
Theorem
There is no wait-free implementation of n-thread consensus from read-write registers

Art of Multiprocessor Programming

36

Formally
Theorem
There is no wait-free implementation of n-thread consensus from read-write registers

Implication
Asynchronous computability different from Turing computability

Art of Multiprocessor Programming

37

Proof Strategy
Assume otherwise Reason about the properties of any such protocol Derive a contradiction Quod Erat Demonstrandum Enough to consider binary consensus and n=2
Art of Multiprocessor Programming 38

Protocol Histories as State Transitions


A
B x.read() y.read(y) y.read() x.write() final state

init state

time time
Art of Multiprocessor Programming 39

Wait-Free Computation
A moves B moves

Either A or B moves Moving means


Register read Register write
Art of Multiprocessor Programming 40

The Two-Move Tree


Final states

Initial state

Art of Multiprocessor Programming

41

Decision Values

1
42

Art of Multiprocessor Programming

Bivalent: Both Possible


bivalent

1
43

Art of Multiprocessor Programming

Univalent: Single Value Possible


univalent

1
44

Art of Multiprocessor Programming

x-valent: x Only Possible Decision


1-valent

1
45

Art of Multiprocessor Programming

Summary
Wait-free computation is a tree

Art of Multiprocessor Programming

46

Summary
Wait-free computation is a tree Bivalent system states
Outcome not fixed

Art of Multiprocessor Programming

47

Summary
Wait-free computation is a tree Bivalent system states
Outcome not fixed

Univalent states
Outcome is fixed May not be known yet

Art of Multiprocessor Programming

48

Summary
Wait-free computation is a tree Bivalent system states
Outcome not fixed

Univalent states
Outcome is fixed May not be known yet

1-Valent and 0-Valent states


Art of Multiprocessor Programming 49

Claim
Some initial state is bivalent

Art of Multiprocessor Programming

50

Claim
Some initial state is bivalent Outcome depends on
Chance Whim of the scheduler

Art of Multiprocessor Programming

51

Claim
Some initial state is bivalent Outcome depends on
Chance Whim of the scheduler

Multicore gods procrastinate

Art of Multiprocessor Programming

52

Claim
Some initial state is bivalent Outcome depends on
Chance Whim of the scheduler

Multicore gods procrastinate Lets prove it


Art of Multiprocessor Programming 53

What if inputs differ?

Art of Multiprocessor Programming

54

Must Decide 0

In this solo execution by A


Art of Multiprocessor Programming 55

Must Decide 1

In this solo execution by B


Art of Multiprocessor Programming 56

Mixed Initial State Bivalent

Solo execution by A must decide 0

Solo execution by B must decide 1


57

Art of Multiprocessor Programming

Critical States
critical

0-valent
Art of Multiprocessor Programming

1-valent
58

From a Critical State


c

0-valent If A goes first, protocol decides 0

1-valent
If B goes first, protocol decides 1
59

Art of Multiprocessor Programming

Critical States
Claim: starting from a bivalent initial state the protocol will reach a critical state
Lets prove it

Art of Multiprocessor Programming

60

Reaching Critical State


initially bivalent

CB
CA
univalent univalent

CB CA
univalent

univalent 0-valent

c
1-valent
61

Critical States
Claim: Starting from a bivalent initial state the protocol will reach a critical state
Proof:
Otherwise we could stay bivalent forever And the protocol would not be wait-free

Art of Multiprocessor Programming

62

Model Dependency
So far, memory-independent! True for
Registers Message-passing Carrier pigeons Any kind of asynchronous computation

Art of Multiprocessor Programming

63

Closing the Deal


Start from a critical state

Art of Multiprocessor Programming

Closing the Deal


Start from a critical state Each thread fixes outcome by
Reading or writing Same or different registers

Art of Multiprocessor Programming

Closing the Deal


Start from a critical state Each thread fixes outcome by
Reading or writing Same or different registers

Leading to a 0 or 1 decision

Art of Multiprocessor Programming

Closing the Deal


Start from a critical state Each thread fixes outcome by
Reading or writing Same or different registers

Leading to a 0 or 1 decision And a contradiction.

Art of Multiprocessor Programming

Possible Interactions
x.read() x.read() y.read() x.write() y.write()

y.read()

?
? ?

?
? ?
Art of Multiprocessor Programming

?
? ?

?
? ?
68

x.write()

y.write()

Possible Interactions
A reads x
x.read() x.read() y.read() x.write() y.write()

y.read()

?
? ?

?
? ?
Art of Multiprocessor Programming

?
? ?

?
? ?
69

x.write()

y.write()

Possible Interactions
A reads x A reads y
x.read() x.read() y.read() x.write() y.write()

y.read()

?
? ?

?
? ?
Art of Multiprocessor Programming

?
? ?

?
? ?
70

x.write()

y.write()

Some Thread Reads


c

Art of Multiprocessor Programming

71

Some Thread Reads


A runs solo, eventually decides 0

Art of Multiprocessor Programming

72

Some Thread Reads


A runs solo, eventually decides 0

B reads x

Art of Multiprocessor Programming

73

Some Thread Reads


A runs solo, eventually decides 0

B reads x

0
1
A runs solo, eventually decides 1

Art of Multiprocessor Programming

74

Some Thread Reads


A runs solo, eventually decides 0

B reads x

0
1
States look the same to A
Art of Multiprocessor Programming 75

A runs solo, eventually decides 1

Some Thread Reads


A runs solo, eventually decides 0

B reads x

0
1
States look the same to A
Art of Multiprocessor Programming 76

A runs solo, eventually decides 1

Possible Interactions
x.read() x.read() y.read() x.write() y.write()

no

no

no

no

y.read()

no
no no

no
no no
Art of Multiprocessor Programming

no
? ?

no
? ?
77

x.write()

y.write()

Writing Distinct Registers


c

Art of Multiprocessor Programming

78

Writing Distinct Registers


A writes y

Art of Multiprocessor Programming

79

Writing Distinct Registers


A writes y

B writes x

Art of Multiprocessor Programming

80

Writing Distinct Registers


A writes y

B writes x

Art of Multiprocessor Programming

81

Writing Distinct Registers


A writes y

B writes x

B writes x

Art of Multiprocessor Programming

82

Writing Distinct Registers


A writes y

B writes x

B writes x

A writes y

Art of Multiprocessor Programming

83

Writing Distinct Registers


A writes y

B writes x

B writes x

A writes y

Art of Multiprocessor Programming

84

Writing Distinct Registers


A writes y

B writes x

B writes x

A writes y

Same story

85

Writing Distinct Registers


A writes y

B writes x

B writes x

A writes y

Same story

86

Possible Interactions
x.read() x.read() y.read() x.write() y.write()

no

no

no

no

y.read()

no
no no

no
no no
Art of Multiprocessor Programming

no
? no

no
no ?
87

x.write()

y.write()

Writing Same Registers


c
A writes x A runs solo, eventually decides 0 B writes x

A writes x

0
States look the same to A

1
Art of Multiprocessor Programming

A runs solo, eventually decides 1


88

Thats All, Folks!


x.read() x.read() y.read() x.write() y.write()

no

no

no

no

y.read()

no
no no

no
no no
Art of Multiprocessor Programming

no
no no

no
no no
89

x.write()

y.write()

Recap: Atomic Registers Cant Do Consensus


If protocol exists
It has a bivalent initial state Leading to a critical state

Whats up with the critical state?


Case analysis for each pair of methods As we showed, all lead to a contradiction

Art of Multiprocessor Programming

90

What Does Consensus have to do with Concurrent Objects?

Art of Multiprocessor Programming

91

Consensus Object
public interface Consensus<T> { T decide(T value); }

Art of Multiprocessor Programming

92

Concurrent Consensus Object


We consider only one time objects:
each thread calls method only once

Linearizable to sequential consensus object:


Winners call went first

Art of Multiprocessor Programming

93

Java Jargon Watch


Define Consensus protocol as an abstract class We implement some methods You do the rest

Art of Multiprocessor Programming

94

Generic Consensus Protocol


abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N]; protected void propose(T value) { proposed[ThreadID.get()] = value; } abstract public T decide(T value); }

Art of Multiprocessor Programming

95

Generic Consensus Protocol


abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N]; protected void propose(T value) { proposed[ThreadID.get()] = value; }

Each value); threads abstract public T decide(T proposed value }

Art of Multiprocessor Programming

96

Generic Consensus Protocol


abstract class ConsensusProtocol<T> implements Consensus<T> { protected T[] proposed = new T[N]; protected void propose(T value) { proposed[ThreadID.get()] = value; } abstract public T decide(T value); } Propose a value

Art of Multiprocessor Programming

97

Generic Consensus Protocol


abstract class ConsensusProtocol<T> Decide a value: abstract method implements Consensus { means subclass does= the real work protected T[] proposed new T[N]; protected void propose(T value) { proposed[ThreadID.get()] = value; } abstract public T decide(T value); }

Art of Multiprocessor Programming

98

Can a FIFO Queue Implement Consensus?

Art of Multiprocessor Programming

99

FIFO Consensus

proposed array FIFO Queue with red and black balls


Art of Multiprocessor Programming 100

Coveted red ball

Dreaded black ball

Protocol: Write Value to Array

Art of Multiprocessor Programming

101

Protocol: Take Next Item from Queue

0
8

Art of Multiprocessor Programming

102

Protocol: Take Next Item from Queue


I got the 0coveted 1 red ball, so I will decide my value I got the dreaded black ball, so I will decide the others value from the array
8

Art of Multiprocessor Programming

103

Consensus Using FIFO Queue


public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; public QueueConsensus() { queue = new Queue(); queue.enq(Ball.RED); queue.enq(Ball.BLACK); } }

Art of Multiprocessor Programming

104

Initialize Queue
public class QueueConsensus extends ConsensusProtocol { private Queue queue; public QueueConsensus() { this.queue = new Queue(); this.queue.enq(Ball.RED); this.queue.enq(Ball.BLACK); } }
8

Art of Multiprocessor Programming

105

Who Won?
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; public T decide(T value) { propose(value); Ball ball = queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } }
Art of Multiprocessor Programming 106

Who Won?
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; public T decide(T value) { propose(value); Ball ball = queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-ij]; } Race to dequeue } first queue item
Art of Multiprocessor Programming 107

Who Won?
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; public T decide(T value) { propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; I win if I was first } }
Art of Multiprocessor Programming 108

Who Won?
public class QueueConsensus<T> extends ConsensusProtocol<T> { private Queue queue; Other thread wins if I public T decide(T value) was { second propose(value); Ball ball = this.queue.deq(); if (ball == Ball.RED) return proposed[i]; else return proposed[1-i]; } }
Art of Multiprocessor Programming 109

Why does this Work?


If one thread gets the red ball Then the other gets the black ball Winner decides her own value Loser can find winners value in array
Because threads write array Before dequeueing from queue

Art of Multiprocessor Programming

110

Theorem
We can solve 2-thread consensus using only
A two-dequeuer queue, and Some atomic registers

Art of Multiprocessor Programming

111

Implications
Given
A consensus protocol from queue and registers

Assume there exists


A queue implementation from atomic registers

Substitution yields:
A wait-free consensus protocol from atomic registers

Art of Multiprocessor Programming

112

Corollary
It is impossible to implement
a two-dequeuer wait-free FIFO queue from read/write memory.

Art of Multiprocessor Programming

113

Consensus Numbers
An object X has consensus number n
If it can be used to solve n-thread consensus
Take any number of instances of X together with atomic read/write registers and implement n-thread consensus

But not (n+1)-thread consensus

Art of Multiprocessor Programming

114

Consensus Numbers
Theorem
Atomic read/write registers have consensus number 1

Theorem
Multi-dequeuer FIFO queues have consensus number at least 2

Art of Multiprocessor Programming

115

Consensus Numbers Measure Synchronization Power


Theorem
If you can implement X from Y And X has consensus number c Then Y has consensus number at least c

Art of Multiprocessor Programming

116

Synchronization Speed Limit


Conversely
If X has consensus number c And Y has consensus number d < c Then there is no way to construct a waitfree implementation of X by Y

This theorem will be very useful


Unforeseen practical implications!
Art of Multiprocessor Programming 117

Earlier Grand Challenge


Snapshot means
Write any array element Read multiple array elements atomically

What about
Write multiple array elements atomically Scan any array elements

Call this problem multiple assignment


Art of Multiprocessor Programming 118

Multiple Assignment Theorem


Atomic registers cannot implement multiple assignment Weird or what?
Single write/multi read OK Multi write/multi read impossible

Art of Multiprocessor Programming

119

Proof Strategy
If we can write to 2/3 array elements
We can solve 2-consensus Impossible with atomic registers

Therefore
Cannot implement multiple assignment with atomic registers

Art of Multiprocessor Programming

120 (1)

Proof Strategy
Take a 3-element array
A writes atomically to slots 0 and 1 B writes atomically to slots 1 and 2 Any thread can scan any set of locations

Art of Multiprocessor Programming

121

Initially
A
Writes to 0 and 1

Writes to 1 and 2
Art of Multiprocessor Programming 125

Thread A wins if
A

B
Art of Multiprocessor Programming

Thread B didnt move


126

Thread A wins if
A

B
Art of Multiprocessor Programming

Thread B moved later


127

Thread A loses if
A

B
Art of Multiprocessor Programming

Thread B moved earlier


128

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }}

Art of Multiprocessor Programming

129

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }}

Not seen: Extends ConsensusProtocol Decide first sets j=i-1 and proposes value
Art of Multiprocessor Programming 130

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else Three slots return proposed[j]; initialized to }} EMPTY

Art of Multiprocessor Programming

131

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else Assign ID 0 to entries 0,1 return proposed[j]; (or ID 1 to entries 1,2) }}

Art of Multiprocessor Programming

132

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; Read the register my }} thread didnt assign

Art of Multiprocessor Programming

133

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else Other thread didnt return proposed[j]; move, so I win }}

Art of Multiprocessor Programming

134

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else Other thread moved return proposed[j]; later so I win }}

Art of Multiprocessor Programming

135

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i]; else return proposed[j]; }} OK, I win.

Art of Multiprocessor Programming

136

Multi-Consensus Code
class MultiConsensus extends { Assign2 a = new Assign2(3, EMPTY); public T decide(T value) { a.assign(i, i, i+1, i); int other = a.read((i+2) % 3); if (other==EMPTY||other==a.read(1)) return proposed[i];(1) else return proposed[j]; Other thread moved }} first, so I lose

Art of Multiprocessor Programming

137

Summary
If a thread can assign atomically to 2 out of 3 array locations Then we can solve 2-consensus Therefore
No wait-free multi-assignment From read/write registers

Art of Multiprocessor Programming

138

Read-Modify-Write Objects
Method call
Returns objects prior value x Replaces x with mumble(x)

Art of Multiprocessor Programming

139

Read-Modify-Write
public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; } }

Art of Multiprocessor Programming

140

Read-Modify-Write
public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; } }

Return prior value

Art of Multiprocessor Programming

141

Read-Modify-Write
public abstract class RMWRegister { private int value; public int synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; } }

Apply function to current value

Art of Multiprocessor Programming

142

RMW Everywhere!
Most synchronization instructions
are RMW methods

The rest
Can be trivially transformed into RMW methods

Art of Multiprocessor Programming

143

Example: Read
public abstract class RMWRegister { private int value; public int synchronized read() { int prior = value; value = value; return prior; } }

Art of Multiprocessor Programming

144

Example: Read
public abstract class RMW { private int value; public int synchronized read() { int prior = this.value; value = value; return prior; } }

apply f(x)=x, the identity function


Art of Multiprocessor Programming

145

Example: getAndSet
public abstract class RMWRegister { private int value; public int synchronized getAndSet(int v) { int prior = value; value = v; return prior; } }
Art of Multiprocessor Programming 146

Example: getAndSet (swap)


public abstract class RMWRegister { private int value; public int synchronized getAndSet(int v) { int prior = value; value = v; return prior; } }

f(x)=v is constant
Art of Multiprocessor Programming 147

getAndIncrement
public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = value; value = value + 1; return prior; } }
Art of Multiprocessor Programming 148

getAndIncrement
public abstract class RMWRegister { private int value; public int synchronized getAndIncrement() { int prior = value; value = value + 1; return prior; } }

f(x) = x+1

Art of Multiprocessor Programming

149

getAndAdd
public abstract class RMWRegister { private int value; public int synchronized getAndAdd(int a) { int prior = value; value = value + a; return prior; } }
Art of Multiprocessor Programming 150

Example: getAndAdd
public abstract class RMWRegister { private int value; public int synchronized getAndIncrement(int a) { int prior = value; value = value + a; return prior; } f(x) = x+a
Art of Multiprocessor Programming 151

compareAndSet
public abstract class CASObject { private int value; public boolean synchronized compareAndSet(int expected, int update) { if (value==expected) { value = update; return true; } return false; } }

Art of Multiprocessor Programming

152

compareAndSet
public abstract class CASObject { private int value; public boolean synchronized compareAndSet(int expected, int update) { if (value==expected) { value = update; return true; } return false; } }

If value is as expected,
Art of Multiprocessor Programming 153

compareAndSet
public abstract class CASOBJECT{ private int value; public boolean synchronized compareAndSet(int expected, int update) { if (value==expected) { value = update; return true; } return false; } }

replace it

Art of Multiprocessor Programming

154

compareAndSet
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { if (value==expected) { value = update; return true; } return false; } }

Report success
Art of Multiprocessor Programming 155

compareAndSet
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { if (value==expected) { value = update; return true; } return false; Otherwise report failure } }

Art of Multiprocessor Programming

156

Read-Modify-Write
public abstract class RMWRegister { private int value; public void synchronized getAndMumble() { int prior = value; value = mumble(value); return prior; } }

Lets characterize f(x)


Art of Multiprocessor Programming 157

Definition
A RMW method
With function mumble(x) is non-trivial if there exists a value v Such that v mumble(v)

Art of Multiprocessor Programming

158

Par Example
Identity(x) = x
is trivial

getAndIncrement(x) = x+1
is non-trivial

Art of Multiprocessor Programming

159

Theorem
Any non-trivial RMW object has consensus number at least 2 No wait-free implementation of RMW registers from atomic registers Hardware RMW instructions not just a convenience

Art of Multiprocessor Programming

160

Reminder
Subclasses of consensus have
propose(x) method
which just stores x into proposed[i] built-in method

decide(object value) method


which determines winning value customized, class-specific method

Art of Multiprocessor Programming

161

Proof
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}

Art of Multiprocessor Programming

162

Proof
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else Initialized to v return proposed[j]; }}

Art of Multiprocessor Programming

163

Proof
public class RMWConsensus extends Consensus { Am I first? private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) return proposed[i]; else return proposed[j]; }}

Art of Multiprocessor Programming

164

Proof
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { Yes, return propose(value); if (r.getAndMumble() == v) my input return proposed[i]; else return proposed[j]; }}

Art of Multiprocessor Programming

165

Proof
public class RMWConsensus extends ConsensusProtocol { private RMWRegister r = v; public T decide(T value) { propose(value); if (r.getAndMumble() == v) No, return return proposed[i]; else others input return proposed[j]; }}

Art of Multiprocessor Programming

166

Proof
We have displayed
A two-thread consensus protocol Using any non-trivial RMW object

Art of Multiprocessor Programming

167

Interfering RMW
Let F be a set of functions such that for all fi and fj, either
Commute: fi(fj(v))=fj(fi(v)) Overwrite: fi(fj(v))=fi(v)

Claim: Any set of RMW objects that commutes or overwrites has consensus number exactly 2
Art of Multiprocessor Programming 168

Examples
test-and-set getAndSet(1) f(v)=1
Overwrite fi(fj(v))=fi(v)

swap getAndSet(x) f(v,x)=x


Overwrite fi(fj(v))=fi(v)

fetch-and-inc getAndIncrement() f(v)=v+1


Commute fi(fj(v))= fj(fi(v))
Art of Multiprocessor Programming 169

Meanwhile Back at the Critical State


A about to apply fA

B about to apply fB

0-valent

1-valent

Art of Multiprocessor Programming

170

Maybe the Functions Commute


A applies fA

B applies fB

B applies fB

A applies fA C runs solo

C runs solo
0-valent

0
Art of Multiprocessor Programming

1
1-valent
171

Maybe the Functions Commute


A applies fA

B applies fB

These states look the same to C


B applies fB

A applies fA C runs solo

C runs solo
0-valent

0
Art of Multiprocessor Programming

1
1-valent
172

Maybe the Functions Overwrite


A applies fA

B applies fB

C runs solo

A applies fA

0
C runs solo
0-valent
Art of Multiprocessor Programming

1
1-valent
173

Maybe the Functions Overwrite


These states look the same to C
A applies fA

B applies fB

C runs solo

A applies fA

0
C runs solo
0-valent
Art of Multiprocessor Programming

1
1-valent
174

Impact
Many early machines provided only these weak RMW instructions
Test-and-set (IBM 360) Fetch-and-add (NYU Ultracomputer) Swap (Original SPARCs)

We now understand their limitations


But why do we want consensus anyway?
Art of Multiprocessor Programming 175

compareAndSet
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = value; if (value==expected) { value = update; return true; } return false; } }
Art of Multiprocessor Programming 176

compareAndSet
public abstract class RMWRegister { private int value; public boolean synchronized compareAndSet(int expected, int update) { int prior = this.value; if (value==expected) { this.value = update; return true; } return false; replace value if its what we } }

expected,

Art of Multiprocessor Programming

177

compareAndSet Has Consensus Number


public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } }

Art of Multiprocessor Programming

178

compareAndSet Has Consensus Number


public class RMWConsensus extends ConsensusProtocol { private AtomicInteger r = new AtomicInteger(-1); public T decide(T value) { propose(value) r.compareAndSet(-1,i); return proposed[r.get()]; } } Initialized to -1

Art of Multiprocessor Programming

179

compareAndSet Has Consensus Number


public class RMWConsensus extends ConsensusProtocol { Try private AtomicInteger r to = swap in my id new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } }

Art of Multiprocessor Programming

180

compareAndSet Has Consensus Number


public class RMWConsensus extends ConsensusProtocol { winners private AtomicInteger Decide r = preference new AtomicInteger(-1); public T decide(T value) { propose(value); r.compareAndSet(-1,i); return proposed[r.get()]; } }

Art of Multiprocessor Programming

181

The Consensus Hierarchy


1 Read/Write Registers, Snapshots 2 getAndSet, getAndIncrement, . . .

compareAndSet,
Art of Multiprocessor Programming 182

Multiple Assignment
Atomic k-assignment Solves consensus for 2k-2 threads Every even consensus number has an object (can be extended to odd numbers)

Art of Multiprocessor Programming

183

Lock-Freedom
Lock-free:
in an infinite execution infinitely often some method call finishes

Pragmatic approach Implies no mutual exclusion

Art of Multiprocessor Programming

184

Lock-Free vs. Wait-free


Wait-Free: each method call takes a finite number of steps to finish Lock-free: infinitely often some method call finishes

Art of Multiprocessor Programming

185

Lock-Freedom
Any wait-free implementation is lock-free. Lock-free is the same as waitfree if the execution is finite.

Art of Multiprocessor Programming

186

Lock-Free Implementations
Lock-free consensus is as impossible as wait-free consensus All these results hold for lock-free algorithms also.

Art of Multiprocessor Programming

187

There is More: Universality


Consensus is universal From n-thread consensus
Wait-free/Lock-free Linearizable n-threaded Implementation Of any sequentially specified object
Art of Multiprocessor Programming 188

This work is licensed under a Creative Commons AttributionShareAlike 2.5 License.


You are free:
to Share to copy, distribute and transmit the work to Remix to adapt the work

Under the following conditions:


Attribution. You must attribute the work to The Art of Multiprocessor Programming (but not in any way that suggests that the authors endorse you or your use of the work). Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to
http://creativecommons.org/licenses/by-sa/3.0/.

Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.

Art of Multiprocessor Programming

189

Potrebbero piacerti anche