Chapter Mpi

Parallel Programming MPI
Pham Quang Dung
Hanoi, 2012
Pham Quang Dung ()
Hanoi, 2012
1 / 31
Outline
Introduction
Installation
MPI routines
Pham Quang Dung ()
Hanoi, 2012
2 / 31
Introduction
Message passing model Distributed memory architecture Communication is based on send and receive operations Documentation : http ://www.open-mpi.org/doc/
Pham Quang Dung ()
Hanoi, 2012
3 / 31
Outline
Introduction
Installation
MPI routines
Pham Quang Dung ()
Hanoi, 2012
4 / 31
Installation
Download openmpi-1.6.2.tar.gz from http ://www.open-mpi.org/software/ompi/v1.6/ Extract and cd openmpi-1.6.2 Run : ./congure Run : sudo make all install Congure library : export LD LIBRARY PATH=$LD LIBRARY PATH :/usr/local/lib/
Pham Quang Dung ()
Hanoi, 2012
5 / 31
Outline
Introduction
Installation
MPI routines
Pham Quang Dung ()
Hanoi, 2012
6 / 31
MPI routines
8 cores functions allowing to write an MPI application

MPI MPI MPI MPI MPI MPI MPI MPI Init Finalize Comm size Comm rank Send Recv Isend Irecv
Pham Quang Dung ()
Hanoi, 2012
7 / 31
MPI Init & MPI Finalize
int MPI Init(int* argc, char***argv)

Return MPI SUCCESS or an error code This function initializes the MPI execution environment This function must be called by each MPI process before any other MPI functions is executed
int MPI Finalize()

Free any resources Each MPI process must call this function before it exits
Pham Quang Dung ()
Hanoi, 2012
8 / 31
MPI Comm size
int MPI Comm size(MPI Comm comm, int* size)

IN comm : Communicator OUT size : the number of processes in the communication group Communicator : identies a process group and denes the communication context. All message tags are unique with respect to a communicator MPI COMM WORLD : the processes group includes all processes of a parallel application MPI Comm size : returns the number of processes in the group of the given communicator
Pham Quang Dung ()
Hanoi, 2012
9 / 31
MPI Comm rank
int MPI Comm rank(MPI Comm comm, int* rank)

IN comm : Communicator OUT rank : id of the process in the communication group Communicator : identies a process group and denes the communication context. All message tags are unique with respect to a communicator MPI COMM WORLD : the processes group includes all processes of a parallel application MPI Comm rank : returns the id of process in the group of the given communicator
Pham Quang Dung ()
Hanoi, 2012
10 / 31
Hello world
1 3 5 7 9
#i n c l u d e < s t d i o . h> #i n c l u d e <mpi . h> #i n c l u d e < u n i s t d . h> i n t main ( i n t a r g c , c h a r a r g v ) { i n t rank , s i z e ; M P I I n i t (& a r g c ,& a r g v ) ; / s t a r t s MPI / MPI Comm rank (MPI COMM WORLD,& r a n k ) ; / g e t c u r r e n t p r o c e s s i d / MPI Comm size (MPI COMM WORLD,& s i z e ) ; / g e t number o f p r o c e s s e s / p r i n t f ( H e l l o w o r l d from p r o c e s s %d o f %d \ n , r a n k , s i z e ) ; s l e e p (10) ; MPI Finalize () ; return 0; }
11
13
Pham Quang Dung ()
Hanoi, 2012
11 / 31
Hello world
Compile : mpic++ -o helloworld helloworld.cpp Run : mpirun -np 4 helloworld Run with hosts le : mpirun hostle myhosts.txt -np 4 helloworld
myhosts.txt is a text le each line is an IP address of a host in the system
Pham Quang Dung ()
Hanoi, 2012
12 / 31
Hello world
2 4
Hello Hello Hello Hello
world world world world
from from from from
process process process process
0 2 3 1
of of of of
4 4 4 4
Pham Quang Dung ()
Hanoi, 2012
13 / 31
MPI Send
MPI Send(void* buf, int count, MPI Datatype dtype, int dest, int tag, MPI Comm comm)
IN IN IN IN IN IN buf : address of the send buer count : number of items to be sent dtype : type of the items dest : Receiver id tag : message tags comm : Communicator
It is a blocking function : it terminates when the send buer can be reused

either the message was delivered, or the data were copied to a system buer
Pham Quang Dung ()
Hanoi, 2012
14 / 31
MPI Recv
MPI Recv(void* buf, int count, MPI Datatype dtype, int source, int tag, MPI Comm comm, MPI Status* status)
IN buf : address of the receive buer IN count : number of items to be received IN dtype : type of the items IN dest : Sender id IN tag : message tags IN comm : Communicator OUT status : the status information
It is a blocking function : it terminates when the message is available in the reciever buer The message must not be larger than the receiver buer The remaining part of the buer not used for the recieved message will be unchanged
Pham Quang Dung ()
Hanoi, 2012
15 / 31
MPI Recv - properties
A message to be recieved must match the sender, the tag, and the communicator Sender and tag can be specied as wild card : MPI ANY SOURCE, MPI ANY TAG The actual length of the received message can be determined via MPI Get count function
int MPI Get count( MPI Status *status, MPI Datatype datatype, int *count )
Output : count is the number of received elements
Pham Quang Dung ()
Hanoi, 2012
16 / 31
Two processes send and receive an array

i n t main ( i n t a r g c , c h a r a r g v ) { i n t rank , s i z e ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& r a n k ) ; MPI Comm size (MPI COMM WORLD,& s i z e ) ; MPI Status s t a t ; i n t s [MAX] = { 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 } ; i n t r [MAX ] ; MPI Send ( s , 1 0 , MPI INT ,1 r a n k , r a n k , MPI COMM WORLD) ; MPI Recv ( r , 1 0 , MPI INT ,1 r a n k ,1 r a n k , MPI COMM WORLD, &s t a t ) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , r a n k ) ; f o r ( i n t i = 0 ; i < 1 0 ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; return 0; }
2 4 6 8
10
12
14
16
Pham Quang Dung ()
Hanoi, 2012
17 / 31
Deadlock
i n t main ( i n t a r g c , c h a r a r g v ) { i n t rank , s i z e ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& r a n k ) ; MPI Comm size (MPI COMM WORLD,& s i z e ) ; MPI Status s t a t ; i n t s [MAX] = { 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 } ; i n t r [MAX ] ; MPI Recv ( r , 1 0 , MPI INT ,1 r a n k , r a n k , MPI COMM WORLD, &s t a t ) ; MPI Send ( s , 1 0 , MPI INT ,1 r a n k ,1 r a n k , MPI COMM WORLD) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , r a n k ) ; f o r ( i n t i = 0 ; i < 1 0 ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; return 0; }
2 4 6 8
10
12
14
16
Pham Quang Dung ()
Hanoi, 2012
18 / 31
Deadlock - avoid with nonblocking functions

2 4 6 8
10
12
i n t main ( i n t a r g c , c h a r a r g v ) { i n t rank , s i z e ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& r a n k ) ; MPI Comm size (MPI COMM WORLD,& s i z e ) ; MPI Status s t a t ; MPI Request r e q s ; MPI Request r e q r ; i n t s [MAX] = { 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 } ; i n t r [MAX ] ; M P I I r e c v ( r , 1 0 , MPI INT ,1 r a n k , r a n k , MPI COMM WORLD, &r e q r ) ; M P I I s e n d ( s , 1 0 , MPI INT ,1 r a n k ,1 r a n k , MPI COMM WORLD, &r e q s ) ; MPI Wait(& r e q r , &s t a t ) ; // b l o c k i n g f u n c t i o n , message i s r e c e i v e d p r i n t f ( P r o c e s s %d r e c e i v e d : , r a n k ) ; f o r ( i n t i = 0 ; i < 1 0 ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; return 0; }
Pham Quang Dung () Parallel Programming MPI Hanoi, 2012 19 / 31
14
f i n i s h e s when
16
18
20
22
MPI Sendrecv(void* sendbuf, int sendcount, MPI Datatype sendtype, int dest, int sendtag, void*recvbuf, int recvcount, MPI Datatype recvtype, int source, int recvtag, MPI Comm comm, MPI Status stat)
Equivalent to the execution of MPI Send and MPI Recv in parallel threads sendbuf and recvbuf are dierent buers
Pham Quang Dung ()
Hanoi, 2012
20 / 31
Example - two processes

1 3 5 7 9
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; i n t s [MAX] , r [MAX ] ; i n t n = 10; MPI Status s t a t ; f o r ( i n t i = 0 ; i < n ; i ++) s [ i ] = (1 2 i d ) i ; M P I S e n d r e c v ( s , n , MPI INT ,1 i d , i d , r , n , MPI INT ,1 i d ,1 i d , MPI COMM WORLD,& s t a t ) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; }
11
13
15
17
Pham Quang Dung ()
Hanoi, 2012
21 / 31
Example - multiple processes - roundrobin

2 4 6 8
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; i n t s [MAX] , r [MAX ] ; i n t n = 10; MPI Status s t a t ; for ( int i = 0; i < n ; i ++) s [ i ] = i d ;
10
12
14
i n t d e s t = i d + 1 ; i f ( d e s t >= s z ) d e s t = 0 ; i n t s r c = i d 1 ; i f ( s r c < 0 ) s r c = s z 1; M P I S e n d r e c v ( s , n , MPI INT , d e s t , i d , r , n , MPI INT , s r c , s r c , MPI COMM WORLD,& s t a t ) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; }
16
18
20
22
MPI Bcast(void* sendbuf, int count, MPI Datatype type, int root, MPI Comm comm)
The content of sendbuf of the process root is copied to all other processes Function type : Blocking
Pham Quang Dung ()
Hanoi, 2012
23 / 31
MPI Bcast(void* sendbuf, int count, MPI Datatype type, int root, MPI Comm comm)
1 3 5 7 9
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; i n t root = a t o i ( argv [ 1 ] ) ; p r i n t f ( P r o c e s s %d s t a r t e d , r o o t = %d \ n , i d , r o o t ) ; i n t s [MAX ] ; i n t n = 10; MPI Status s t a t ; i f ( i d == r o o t ) for ( int i = 0; i < n ; s[i] = i ;
11
13
i ++)
15
MPI Bcast ( s , n , MPI INT , r o o t , MPI COMM WORLD) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , s [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; Pham Quang Dung () Parallel Programming MPI }
17
19
21
23
Hanoi, 2012
24 / 31
MPI Gather(void* sendbuf, int sendcount, MPI Datatype sendtype, void*recvbuf, int recvcount, MPI Datatype recvtype, int root, MPI Comm comm)
Process root receives the data in the send buer of all processes The received data is stored in the receive buer ordered by the process id of the senders Note : recvcount is the number of items to be received from each process
Pham Quang Dung ()
Hanoi, 2012
25 / 31
2 4 6 8
10
12
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; i n t root = a t o i ( argv [ 1 ] ) ; i n t s [MAX] , r [MAX ] ; i n t n = 10; MPI Status s t a t ; f o r ( i n t i = 0 ; i < n ; i ++) s [ i ] = i d ; p r i n t f ( P r o c e s s %d b u f f e r s : , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , s [ i ] ) ; M P I B a r r i e r (MPI COMM WORLD) ;
p r i n t f ( \ n ) ;
14
M P I G a t h e r ( s , 1 , MPI INT , r , 1 , MPI INT , r o o t , MPI COMM WORLD) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , r [ i ] ) ; MPI Finalize () ; }
16
18
p r i n t f ( \ n ) ;
20
Run with 5 processes
2 4 6
10
Process 0 buffer s : 0 0 0 0 0 0 0 0 0 0 Process 1 buffer s : 1 1 1 1 1 1 1 1 1 1 Process 2 buffer s : 2 2 2 2 2 2 2 2 2 2 Process 3 buffer s : 3 3 3 3 3 3 3 3 3 3 Process 4 buffer s : 4 4 4 4 4 4 4 4 4 4 P r o c e s s 1 r e c e i v e d : 327932596 32640 4220900 0 35656272 32767 0 0 327982656 32640 P r o c e s s 2 r e c e i v e d : 243923636 32620 4220900 0 1196785472 32767 0 0 243973696 32620 P r o c e s s 4 r e c e i v e d : 1950198452 32607 4220900 0 396994064 32767 0 0 1950248512 32607 P r o c e s s 3 r e c e i v e d : 724302516 32588 4220900 0 1209642080 32767 0 0 724352576 32588 Process 0 received : 0 0 1 1 2 2 3 3 4 4
Pham Quang Dung ()
Hanoi, 2012
27 / 31
MPI Scatter(void* sendbuf, int sendcount, MPI Datatype sendtype, void*recvbuf, int recvcount, MPI Datatype recvtype, int root, MPI Comm comm)
Process root sends the data in the sendbuf to all other processes in the communicator comm sendcount is the number of items to be sent to each process sendbuf is divided into chunks of sendcount items recvcount is the number of items to be received for each process
Pham Quang Dung ()
Hanoi, 2012
28 / 31
2 4 6 8
10
12
14
16
18
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; int root = 1; i n t s [MAX] , r [MAX ] ; i n t n = 10; i n t sendcount = 2; int recvcount = 2; MPI Status s t a t ; i f ( i d == r o o t ) f o r ( i n t i = 0 ; i < n ; i ++) s [ i ] = i ; M P I S c a t t e r ( s , s e n d c o u n t , MPI INT , r , r e c v c o u n t , MPI INT , r o o t , MPI COMM WORLD) ; p r i n t f ( P r o c e s s %d r e c e i v e d : , i d ) ; f o r ( i n t i = 0 ; i < r e c v c o u n t ; i ++) p r i n t f ( %d , r [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; }

2 4
Process Process Process Process Process
1 0 4 3 2
received received received received received
: : : : :
2 0 8 6 4
3 1 9 7 5
Pham Quang Dung ()
Hanoi, 2012
30 / 31
MPI Reduce(void* sendbuf, void* recvbuf, int sendcount, MPI Datatype sendtype, MPI Op op, int root, MPI Comm comm)
Reduces values of all processes to a single process root op is the operator :

MPI MPI MPI MPI .. MAX MIN SUM PROD
Pham Quang Dung ()
Hanoi, 2012
31 / 31
1 3 5 7 9
i n t main ( i n t a r g c , c h a r a r g v ) { i n t id , sz ; M P I I n i t (& a r g c , &a r g v ) ; MPI Comm rank (MPI COMM WORLD,& i d ) ; MPI Comm size (MPI COMM WORLD,& s z ) ; int root = 1; i n t n = 2 , S [ n ] , S1 [ n ] ; f o r ( i n t i = 0 ; i < n ; i ++){ S [ i ] = 0 ; S1 [ i ] = i 100+ i d ; } MPI Reduce(&S1 ,&S , 2 , MPI INT , MPI SUM , r o o t , MPI COMM WORLD) ; p r i n t f ( P r o c e s s i d = %d h a s S = , i d ) ; f o r ( i n t i = 0 ; i < n ; i ++) p r i n t f ( %d , S [ i ] ) ; p r i n t f ( \ n ) ; MPI Finalize () ; }
11
13
15
17
Pham Quang Dung ()
Hanoi, 2012
32 / 31
1 3 5 7 9
Process Process Process Process Process Process Process Process Process Process
id id id id id id id id id id
= = = = = = = = = =
6 2 0 9 1 4 8 7 3 5
has has has has has has has has has has
S S S S S S S S S S
= = = = = = = = = =
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 45 1045 0 0
Pham Quang Dung ()
Hanoi, 2012
33 / 31

Chapter Mpi

Caricato da

Informazioni sul documento

Descrizione originale:

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Chapter Mpi

Caricato da

Copyright:

Formati disponibili

Parallel Programming MPI

Pham Quang Dung

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

8 cores functions allowing to write an MPI application

Pham Quang Dung ()

Parallel Programming MPI

MPI Init & MPI Finalize

int MPI Init(int* argc, char***argv)

int MPI Finalize()

Pham Quang Dung ()

Parallel Programming MPI

MPI Comm size

int MPI Comm size(MPI Comm comm, int* size)

Pham Quang Dung ()

Parallel Programming MPI

MPI Comm rank

int MPI Comm rank(MPI Comm comm, int* rank)

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Hello Hello Hello Hello

world world world world

from from from from

process process process process

Pham Quang Dung ()

Parallel Programming MPI

It is a blocking function : it terminates when the send buer can be reused

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

MPI Recv - properties

Pham Quang Dung ()

Parallel Programming MPI

Two processes send and receive an array

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()

Parallel Programming MPI

Deadlock - avoid with nonblocking functions

Pham Quang Dung ()

Parallel Programming MPI

Example - two processes

Pham Quang Dung ()

Parallel Programming MPI

Example - multiple processes - roundrobin

Pham Quang Dung ()

Parallel Programming MPI

Pham Quang Dung ()