Message Passing Interface (MPI) I

Message Passing Interface (MPI) I İstanbul Teknik Üniversitesi Bilişim Enstitüsü Yüksek Başarımlı Hesaplama Laboratuvarı yayınıdır. Laboratuvar kulla...
Author: Oswald Newman
2 downloads 0 Views 491KB Size
Message Passing Interface (MPI) I

İstanbul Teknik Üniversitesi Bilişim Enstitüsü Yüksek Başarımlı Hesaplama Laboratuvarı yayınıdır. Laboratuvar kullanıcılarının eğitimleri için düzenlenmiştir. İzin alınmaksızın ve referans verilmeksizin hiçbir bölümü yayınlanamaz veya kullanılamaz. Bu yayın, erişildiği ve görüntülendiği takdirde bu açıklama kabul edilmiş sayılır. (2009)

Message Passing Programming Based on MPI

Function categories: Initialization/finalization Point-to-point communication functions Collective communication functions Communicator topologies User-defined data types Utilities (e.g.- timing)

2

Message Passing Interface Minimum MPI Program Skeleton #include or include “mpi.h”

Initialize MPI environment

MPI_Init

Do work and make message passing calls

Terminate MPI Environment

MPI_Finalize

3

Message Passing Interface ●



MPI_Comm_size: Determines communictor.

the

size of the group associated with a

MPI_Comm_rank: Determines the rank of the calling process in the communicator.

1

9

4

7

0 2

3

MPI_COMM_WORLD

6

5

We called “master”

8

4

Messsage Passing Interface



Rank within a communicator: Every process has its own unique, Integer identifier assigned by the system when the process initializes. A rank is sometimes also called a "process ID". Ranks are contiguous and begin at zero.

5

Message Passing Interface

Environment Management Routines



Purposes of MPI environment management routines: Initializing the MPI environment, Terminating the MPI environment, Querying the environment, Identity, etc.

6

Message Passing Interface Environment Management Routines

MPI_Init ●

Initializes the MPI execution environment.



Must be called in every MPI program,



Must be called before any other MPI functions



Must be called only once in an MPI program.



For C programs, MPI_Init may be used to pass the command line arguments to all processes. int MPI_Init(int *argc, char **argv[]) 7

Message Passing Interface

Environment Management Routines

MPI_Finalize ●

Terminates MPI execution environment.



After this routine, any of MPI routines does not int MPI_Finalize (void)

8

Message Passing Interface Environment Management Routines

MPI_Comm_size ●

Determines the number of processes in the group associated with a communicator.



Generally used within the communicator MPI_COMM_WORLD to determine the number of processes being used by your application.

int MPI_Comm_size (MPI_Comm comm,int *size)

9

Message Passing Interface Environment Management Routines       MPI_Comm_rank ●





Determines the rank of the calling process within the communicator. A unique integer rank between communicator MPI_COMM_WORLD.

0

and

(#OfProcs-1)

within

the

The rank is referred to as a task ID.

int MPI_Comm_rank (MPI_Comm comm,int *rank)

10

Message Passing Interface Environment Management Routines

MPI_Get_processor_name ●

Returns the processor name.



Also returns the length of the name.

int

MPI_Get_processor_name (char *name,int *length)

11

Message Passing Interface Environment Management Routines

MPI_Wtime ●

Returns an elapsed wall clock time in seconds (double precision) on the calling processor.

double MPI_Wtime()

12

Message Passing Interface Environment Management Routines

   MPI_Abort ●



Terminates all MPI processes associated with the communicator. In most MPI implementations it terminates ALL processes regardless of the communicator specified.

                                       int MPI_Abort( MPI_Comm comm, int errorcode )

13

Message Passing Interface Sending Data       MPI_Send ●

Performs a blocking send of the specified data to the specified destination.

int MPI_Send( void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm )

14

Message Passing Interface Receving Data

MPI_Recv ●

Performs a blocking receive of the specified data from the  specified source. 

                     int MPI_Recv( void *buf, int count, MPI_Datatype datatype, int source, int tag, MPI_Comm comm, MPI_Status *status )

15

Message Passing Interface

MPI Datatypes MPI_CHAR

signed char

MPI_INT

signed int

MPI_FLOAT

float

MPI_DOUBLE

double

MPI_LONG_DOUBLE

long double

MPI_SHORT

signed short int

MPI_LONG

signed long int

16

Message Passing Interface

MPI Datatypes MPI_UNSIGNED

unsigned int

MPI_UNSIGNED_CHAR

unsigned char

MPI_UNSIGNED_SHORT unsigned short int MPI_UNSIGNED_LONG

unsigned long int

MPI_BYTE

8 binary digits

MPI_PACKED

data packed

17

Message Passing Interface Point to Point Communication Communication Mode

Blocking Routines

Non-Blocking Routines

Synchronous

MPI_Ssend

MPI_Issend

Ready

MPI_Rsend

MPI_Irsend

Buffered

MPI_Bsend

MPI_Isend

Standard

MPI_Send

MPI_Isend

MPI_Recv

MPI_Irecv

MPI_Sendrecv MPI_Sendrecv_replace The receive routine is simply blocking (MPI_Recv) or non-blocking (MPI_Irecv).

18

Message Passing Interface

Point to Point Communication The communication mode is selected with the send routine. ●

Four blocking send routines



Four non-blocking send routines.

The receive routine does not specify communication mode. The receive routine is simply blocking or non-blocking.

19

Message Passing Interface Synchronous Send The sending task sends the receiving task a "ready to send" message. When the receiver executes the receive call, it sends a "ready to receive" message. The data are then transferred. When a synchronous mode send operation is completed, the sending process may assume the destination process has begun receiving the message. The destination process need not be done receiving the message, but it must have begun receiving the message. 20

Message Passing Interface Blocking Communication

Synchronous Send MPI_Ssend Sender 1

2

3

Receiver 1 “Ready to send”

MPI_Recv

2“Ready to receive” 3 Data 21

Message Passing Interface Ready Send - It requires that the "ready to receive" notification has arrived, indicating that the receiving task has posted the receive. - If the "ready to receive" message hasn't arrived, the ready mode send will incur an error. - This mode should not be used unless the user is certain that the corresponding receive has been posted. Overhead: - Ready mode aims to minimize system overhead and synchronization overhead incurred by the sending task.

22

Message Passing Interface Blocking Communication Ready Send MPI_Rsend Sender 1

2

Receiver MPI_Recv 1“Ready to receive” 2 Data 23

Message Passing Interface Buffered Send - The blocking buffered send MPI_Bsend copies the data from the message buffer to a user-supplied buffer, and then returns. - The sending task can then proceed with calculations that modify the original message buffer, knowing that these modifications will not be reflected in the data actually sent. - The data will be copied from the user-supplied buffer over the network once the "ready to receive" notification has arrived. Overheads: –

Buffered mode incurs extra system overhead, because of the additional copy from the message buffer to the user-supplied buffer.



Synchronization overhead is eliminated on the sending task -- the timing of the receive is now irrelevant to the sender. 24

Message Passing Interface Blocking Communication

Buffered Send 1 Sender MPI_Bsend

3 2

Receiver MPI_Recv 1 Copies the data to the system buffer 2 “Ready to receive” 3 Data is transferred from system buffer. 25

Message Passing Interface

Buffered Blocking Send Communication

• • •

MPI_Buffer_attach Provides to MPI a buffer in the user memory to be used for buffering outgoing messages. For using buffered send routine, there must be a user defined buffer. User defined buffer must be attached with MPI_Buffer_attach.

int MPI_Buffer_attach(void *buf, Initial buffer address

int size)

Buffer size, in bytes 26

Message Passing Interface

Buffered Blocking Send Communication





MPI_Buffer_detach Detach the buffer currently associated with MPI. The call returns the address and the size of the detached buffer. This operation will block until all messages currently in the buffer have been transmitted.

int MPI_Buffer_detach(void *buf, Initial buffer address

int *size)

Buffer size, in bytes 27

Message Passing Interface Blocking Point to Point Conclusion Mode

Advantages

Disadvantages

Synchronous

Safest, therefore most portable No need for extra buffer space SEND/RECV order not critical

Can incur substantial synchronization overhead

Ready

Lowest total overhead No need for extra buffer space SEND/RECV handshake not required, but...

RECV must precede SEND

Buffered

Decouples SEND from RECV; no sync overhead on SEND Programmer can control size of buffer space SEND/RECV order irrelevant

Copying to buffer incurs additional system overhead

Standard

Good for many cases For small messages: ready send, buffered receive For large messages: synchronous

Your program may not be suitable

28

Message Passing Interface Deadlock -Deadlock occurs when 2 (or more) processes are blocked and each is waiting for the other to make progress.

-Neither process makes progress because each depends on the other to make progress first. -The program shown below is an example -- it fails to run to completion because processes 0 and 1 deadlock.  if( myrank == 0 ) {       /* Receive, then send a message */       MPI_Recv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &status );       MPI_Send( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );     }     else if( myrank == 1 ) {       /* Receive, then send a message */       MPI_Recv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &status );          MPI_Send( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );     } 29

Message Passing Interface Avoiding Deadlock ➢The program shown below is similar to the program in the preceding section, but its communication is better organized and the program does not deadlock. ➢Once again, process 0 attempts to exchange messages with process 1. ➢Process 0 receives, then sends; process 1 sends, then receives. The protocol is safe.

if( myrank == 0 ) {       /* Receive a message, then send one */       MPI_Recv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &status );       MPI_Send( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );     }     else if( myrank == 1 ) {       /* Send a message, then receive one */       MPI_Send( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );       MPI_Recv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &status );        }

30

Message Passing Interface Non-Blocking Communication ➢After posting a send or receive with a call to a nonblocking routine, the posting process needs some way to refer to the posted operation. ➢MPI uses request handles for this purpose. ➢Nonblocking send and receive routines all return request handles, which are used to identify the operation posted by the call. ➢In summary, ➢ Sends and receives may be posted (initiated) by calling nonblocking routines. ➢ Posted operations are identified by request handles. ➢ Using request handles, processes can check the status of posted operations or wait for their completion.

31

Message Passing Interface

Posting Sends without Blocking ➢A process calls the routine MPI_Isend to post (initiate) a send without blocking on completion of the send operation. ➢MPI_Isend needs additional output argument, a request handle. ➢The request handle identifies the send operation that was posted. ➢The request handle can be used to check the status of the posted send or to wait for its completion. ➢None of the arguments passed to MPI_Isend should be read or written until the send operation is completed. int MPI_Isend(void *buf, int count, MPI_Datatype datatype,                  int dest, int tag, MPI_Comm comm,                MPI_Request *request); 32

Message Passing Interface Posting Receives without Blocking ➢A process calls the routine MPI_Irecv to post (initiate) a receive without blocking on its completion. ➢ The status argument of MPI_Irecv is replaced by a request handle; both are output arguments. ➢The request handle identifies the receive operation that was posted and can be used to check the status of the posted receive or to wait for its completion. ➢None of the arguments passed to MPI_Irecv should be read or written until the receive operation is completed.

int MPI_Irecv(void *buf, int count, MPI_Datatype datatype, int source,              int tag, MPI_Comm comm,               MPI_Request *request);

33

Message Passing Interface Completion: Waiting and Testing ➢Posted sends and receives must be completed. ➢If a send or receive is posted by a nonblocking routine, then its completion status can be checked by calling one of a family of completion routines. ➢MPI provides both blocking and nonblocking completion routines. ➢The blocking routines are MPI_Wait and its variants. ➢The nonblocking routines are MPI_Test and its variants.

34

Message Passing Interface Completion: Waiting ➢A process that has posted a send or receive by calling a nonblocking routine (for instance, MPI_Isend or MPI_Irecv) can subsequently wait for the posted operation to complete by calling MPI_Wait.

int MPI_Wait( MPI_Request *request, MPI_Status *status ); int MPI_Wait( MPI_Request *request, MPI_Status *status ); ➢The request argument is expected to identify a previously posted send or receive. ➢MPI_Wait returns when the send or receive identified by the request argument is complete.

35

Message Passing Interface Completion: Testing ➢A process that has posted a send or receive by calling a nonblocking routine (e.g. MPI_Isend or MPI_Irecv) can subsequently wait for the posted operation to complete by calling MPI_Test.

int MPI_Test( MPI_Request *request, int *flag, MPI_Status *status );

➢The request argument is expected to identify a previously posted send or receive. ➢MPI_Test returns immediately.

36

Hazırlayanlar: H. Hakan Gürel, Özden Akıncı, Berk Onat, Muhammet Yazıcı