Multi-threading in Java

Jeff HUANG
Software Engineering Group @HKUST

Do you use them?

Do u know their internals?

Lets see

File

How can they service so many clients simultaneously? li i l l

DB

Multi-threading
A thread is a single sequential flow of control In a single process, there could be multiple threads Threads share memory The server program spawns multiple threads Each thread handles a client request
5

Example
A single core CPU and a task
compute readfile compute readfile compute writefile compute

7 sec

2 such tasks
6

A simple accounting
Single thread

T=14

Two threads

T=8

Whats more
Multi-core & multi-processor

Dual-core AMD Athlon X2

32 processor p Pentium Xeon

4096 processor Cray p y X1

More accounting
Number of cores = 1,000,000 = 1M Multi-threading can do 2M such tasks in 8 seconds! What b t th i l th Wh t about the single-threaded version? d d i ?

More benefits
Performance User responsiveness p Resource utilization Simplicity f Si li it of modeling d li Simplified handling of asynchronous events p g y

10

Multi-threads are everywhere

Server programs (web/database server) Client programs (Downloading tools/Browser) p g ( g ) Editors (Word/Photoshop/Adobe Reader) Search engine S h i IDE (Matlab/Eclipse/Visual studio) ( p ) JVM Swing and AWT
11

Where are we now? How to program? p g

12

Various ways of multithreading

Thread Runnable Exectutor F E Framework k Timer FutureTask

13

Two major ways

Thread Runnable Exectutor F E Framework k Timer FutureTask

14

Creating threads in Java

Thread class
public class Thread { public void run(){}; }

Runnable interface
public interface Runnable { public void run(); // work thread }

15

Thread Class
public class Thread extends Object implements Runnable { public Thread(); public Thread(String name); // Thread name p public Thread(Runnable R); // Thread R.run() ( ); () public Thread(Runnable R, String name); public void run(); // if no R, work for thread bli id () R kf h d public void start(); // begin thread execution ... }

16

More Thread Class Methods

public class Thread extends Object implements R i l t Runnable { bl public static Thread currentThread() public String getName() p public void interrupt() p () public boolean isAlive() public void join() public void setDaemon() bli id tD () public void setName() public void setPriority() public static void sleep() p public static void yield() y () }
17

Creating Threads in Java

1. Thread class
Extend Thread class and override the run method

Example
public class MyThread extends Thread { public void run() { // work for thread } } MyThread M Th d T = new M Th d () ; // create th d MyThread t thread T.start(); // begin running thread // thread executing in parallel

18

Creating Threads in Java

2. Runnable interface
Create object implementing Runnable interface Pass it to Thread object via Thread constructor

Example E l
public class MyRunnable implements Runnable { public void run() { // work for thread } } MyRunnable myR = new MyRunnable(); // create runnable object Thread T = new Thread(myR); // create thread T.start(); // begin running thread // thread executing in parallel

19

Producer - Consumer
class Producer extends Thread { private final BlockingQueue queue; Producer(BlockingQueue q){queue = q;} public void run() { while(true) { queue.put(produce()); queue put(produce()); } } private Message produce() { return new Message(); } }

public static void main (String[] args) Consumer(BlockingQueue q) { queue = q; } { public void run() { BlockingQueue q = new SomeQueue(); while(true) { Producer p = new Producer(q); consume(queue.take()); consume(queue take()); Consumer c = new Consumer(q); } } p.start(); private void consume(Message mes) { c.start(); } }
}

class Consumer extends Thread { private final BlockingQueue queue;

Example 1
20

Simple WebServer
public class MyRunnable implements Runnable { public void run() { handleRequest(connection); Process request } } public class SimpleWebServer { public static void main (String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { Accept client request final Socket fi l S k t connection = socket.accept(); ti k t t() Runnable task = new MyRunnable(connection); Create Thread t = new Thread(task); Create thread Runnable t.start(); t start(); start th t t thread d } }

Example 2

21

Lets look a bit deeper

Thread states Thread scheduler Difficulties in Multithreaded Diffi l i i M l i h d d p g programming g

22

Thread States
Java thread can be in one of these states
New Runnable Running Blocked Dead thread allocated & waiting for start() thread can begin execution thread currently executing thread waiting for event (I/O, etc.) thread finished

Transitions between states caused by y

Invoking methods in class Thread
new(), start(), y (), (), yield(), sleep(), wait(), notify() (), p(), (), y()

Other (external) events

Scheduler, I/O, returning from run() , , g ()
23

Thread States

new new

start runnable scheduler y yield, time slice notify, notifyAll, notify notifyAll IO complete, sleep expired

running terminate dead IO, sleep, wait, join

blocked

24

Thread Scheduling
Scheduler
Determines which runnable threads to run Can be based on thread priority Part of OS or Java Virtual Machine (JVM)

Scheduling policy

Preempted by scheduler

Non preemptive Non-preemptive (cooperative) scheduling Preemptive scheduling

25

Lets do a bit analysis

Two threads A and B
They both do the following work print 1, print 2, print 3

The main thread first starts A, then starts B

main (): thread A, thread B thread A: println 1, println 2, println 3 thread B: println 1, println 2, println 3
26

Lets do a bit analysis

Non-preemptive scheduler Single-core 1, 2, 3, 1, 2, 3 , , , , , Multi-core Preemptive scheduler

???

??? ???

main (): thread A, thread B thread A: println 1, println 2, println 3 thread B: println 1, println 2, println 3
27

Java Thread Example

public class ThreadExample extends Thread { public void run() { for (int i = 1; i <= 3; i++) System.out.println(i); y p ( ); try { sleep((int)(Math.random() * 1000)); } catch (InterruptedException e) { } } public static void main(String[] args) { Thread A = new ThreadExample(); Thread B = new ThreadExample(); A.start(); A t t() B.start(); System.out.println("Done"); } }
28

Java Thread Example

Possible outputs
1,1,Done,2,2,3,3 1,Done,1,2,2,3,3 1 Done 1 2 2 3 3 1,1,Done,2,3,2,3 Done,1,1, 2,2,3,3 Done 1 1 2 2 3 3

Multi-threads can interleave their executions!!!

29

Another example
public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { First increase x by 1 x = x + 1; Then decrease x by 1 x = x 1; } } public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) Start 1000 threads new Worker().start(); // wait for all threads exit System.out.println(x); } }
30

Data Race
public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { First increase x by 1 x = x + 1; Then decrease x by 1 x = x 1; } } public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) Start 1000 threads new Worker().start(); // wait for all threads exit System.out.println(x); } }

X is not always 0 !!
31

Quiz time

32

Answer: Yes!

33

How Can This Happen?

Compiler can reorder statements
Or keep values in registers p g

The Java memory model is designed to allow aggressive optimization On multi-processor, values not synchronized to global memory Good for performance p

34

Observations
Multithread execution is non-deterministic
Depends on scheduler & single or multi-core

Thread scheduling may cause data races

Modifying same data from multiple threads Result depends on thread execution order

Complier can reorder statements

The memory model is designed to allow aggressive optimization

35

How to deal with it?

We need protocol to make sure the shared p data is manipulated safely We need to force the compiler to ensure that the modified shared data visible to other threads th d Synchronization
Lock p Semaphore Monitor

y Java synchronization

36

Java synchronization
Java uses the concept of monitors Java uses the concept that every object is i t d ith l k associated with a lock use synchronized keyword Two ways
Synchronized methods Synchronized blocks

37

Synchronized method
Calling a synchronized method attempts to possess the lock
If no one owns the lock, then this thread has/owns the lock and proceeds. Otherwise, it has to wait until the lock is released by some other thread

Lock
For static method, it is the lock associated with the Class object to which the static method belongs to For instance method, it is the lock associated with the bj t th object on which the method call is being made hi h th th d ll i b i d
38

Synchronized blocks
Very similar to synchronized method but it is method, Blocks of code, rather than entire methods
public void someMethod() { // non-critical section synchronized(someobject) { // critical section } // non-critical section }

synchronized(this)

39

Data Race
public class Worker extends Thread { static int x = 0; public void run() { for (int i = 0; i < 1000; i++) { First increase x by 1 x = x + 1; Then decrease x by 1 x = x 1; } } public static void main (String[] args) { x = 0; for (int i = 0; i < 1000; i++) Start 1000 threads new Worker().start(); // wait for all threads exit System.out.println(x); } }

X is not always 0 !!
40

Using synchronization
public class Worker extends Thread { static int x = 0; static Object lock = new Object(); j j public void run() { synchronized(lock) { for (int f (i t i = 0 i < 1000 i++) { 0; 1000; x = x + 1; x = x 1; } } } }
41

Questions
What would happen if the lock field were not static? Why dont we just make the run method synchronized? Why dont we just synchronize on x?

42

But, it is hard in practice

Holding locks is a global property
affects entire program, cannot be hidden behind an abstract interface

Results in lack of modularity

callers cannot ignore what locks their callees acquire or what locations they access id b t l f l b l necessary f race avoidance, but also for global for ordering to avoid deadlock part of a methods protocol which lock needs to be held when called, which locks it acquires
43

Bank Account Example

Atomicity violation
class Account { l A t private int balance = 0; public read() { int r; synchronized(this) { y r = balance; } return r; } } public void deposit(int n) { int r = read(); other threads can update balance synchronized(this) { balance = r + n; b l } }

Race-freedom Race freedom is not sufficient

44

Optimized Bank Account

class Account { l A t private int balance = 0; public read() { return balance; } }

public void deposit(int n) { synchronized(this) { int r = balance; balance = r + n; } }

45

Another Account Example

class Account { private int balance = 0; public synchronized int withdraw(int amt){} public synchronized void deposit(int i) {} } class Client1 { public synchronized void move(Account a1, Account a2) { a2.deposit(a1.withdraw(1000000)); } } class Client2 // same as Client1

Client move(a1,a2); Client1: move(a ,a ); Client2: move(a2,a1);

46

Deadlock
client1 holds lock for account a1, waits for account a2 client2 holds lock for account a2, waits for , account a1 Both of them can not proceed
Client1 Cli 1 Client2

a1

a2
47

Impair performance
Frequent lock acquire and lock release operations We are sacrificing performance for data safety But sometimes there is no conflict On some extreme situations, performance is greatly impaired

48

Is it hard?

49

Lets make a conclusion

Multithreading is quite useful g q But you need to be careful Bugs are everywhere

50

What we have learned

Benefits of multithreads
Performance User responsiveness p

How to make multithreaded Java programs

Thread class/Runnable interface

Diffic lties in M ltithreading Difficulties Multithreading

Statements reorder/Data races/Atomicity violation/Deadlock i l ti /D dl k

Java synchronization
Syncrhonized methods/blocks
51

Questions
What would happen if the lock field were not static? Why dont we just make the run method synchronized? Why dont we just synchronize on x?

52

Using synchronization
public class Worker extends Thread { static int x = 0; static Object lock = new Object(); j j public void run() { synchronized(lock) { for (int f (i t i = 0 i < 1000 i++) { 0; 1000; x = x + 1; x = x 1; } } } }
53

See you next time

Backups p

Daemon Threads
Java threads types
User Daemon
Provide general services Typically never terminate Call setDaemon() before start()

Program termination
1. All user threads finish 2. Daemon threads are terminated by JVM 3. Main program finishes p g
56

Atomicity violation

57

Thread flow graph

58

Non-preemptive Scheduling
Threads continue execution until
Thread terminates Executes instruction causing wait (e.g., IO) Thread volunteering to stop (invoking yield or sleep)

59

Preemptive Scheduling
Threads continue execution until
Same reasons as non-preemptive scheduling Preempted by scheduler

60

Transactional Memory
Instead of using synchronizations, the runtime system ensures that the critical section executes transactionally t ti ll You can just go ahead when entering a critical section; when you are exiting the critical section, the ti h iti th iti l ti th runtime system will check the data safety for you
if th there is no conflict, just proceed i fli t j t d Otherwise, rollback

The transaction memory system guarantees

other threads cannot see intermediate values of the ( g) transaction (all-or-nothing) the transaction cannot see values of other transactions in the middle of its execution

61

Hardware & software TM

Hardware transactional memory(HTM) Software transactional memory (STM)

62

Advantages

No deadlock!
we never refer to locks explicitly

Composability & modularity p y y

no need to know what callees do w.r.t. synchronization

Performance could better in some situations

63