Message Passing Programming Based on MPI

Message Passing Programming Based on MPI Derived Data Types Bora AKAYDIN 15.06.2012 Outline Derived Data Types • Derived Datatypes • Packing/...
Author: Claire Price
4 downloads 0 Views 230KB Size
Message Passing Programming Based on MPI Derived Data Types

Bora AKAYDIN

15.06.2012

Outline

Derived Data Types



Derived Datatypes



Packing/Unpacking Datatypes

15.06.2012

Derived Datatypes



How to send only the red elements of V in a single communication?

V(0) V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) V(9)

T(0) T(1) T(2)

MPI_Send

This method requires inefficient copy of noncontiguous data

One possibility, copy these elements to a temporary array before sending.

Derived Data Types

15.06.2012

Non-struct Derived Data Types



There are routines available in MPI library that are more suitable for an array or vector like data structures: • MPI_Type_contiguous • MPI_Type_vector • MPI_Type_indexed



Derived Data Types

All above functions work with a single data type!

15.06.2012

MPI_Type_contiguous (C)



Constructs a type consisting of the replication of a data type into continuous locations.

int MPI_Type_contiguous( int count, MPI_Datatype old_type, MPI_Datatype *newtype)

old type

x count (4)

= new type

Data constructor with MPI_Type_contiguous

Derived Data Types

15.06.2012

mpi_type_contiguous (Fortran)



Constructs a type consisting of the replication of a data type into continuous locations.

MPI_TYPE_CONTIGUOUS( count, MPI_Datatype old_type, MPI_Datatype newtype,ierr)

old type

x count (4)

= new type

Data constructor with mpi_type_contiguous

Derived Data Types

15.06.2012

MPI_Type_contiguous



In C, if we create a matrix with static memory allocation, we can say that data of the matrix will be in contiguous memory.

double A[3][3]; A A A (0,0) (0,1) (0,2) A A A (1,0) (1,1) (1,2) A A A (2,0) (2,1) (2,2)

Derived Data Types

in the memory (C) A A A A A A A A A (0,0) (0,1) (0,2) (1,0) (1,1) (1,2) (2,0) (2,1) (2,2)

in the memory (fortran) A A A A A A A A A (0,0) (1,0)(2,0)(0,1) (1,1)(2,1) (0,2) (1,2) (2,2)

15.06.2012

MPI_Type_contiguous (C) double A[3][3]; A A A (0,0) (0,1) (0,2) A A A (1,0) (1,1) (1,2) A A A (2,0) (2,1) (2,2)

in the memory A A A A A A A A A (0,0) (0,1) (0,2) (1,0) (1,1) (1,2) (2,0) (2,1) (2,2)

count =3 old_type =MPI_DOUBLE new_type =rowtype

MPI_Type_contiguous(int count, MPI_Datatype old_type, MPI_Datatype *newtype);

Derived Data Types

15.06.2012

mpi_type_contiguous (Fortran)

A A A (0,0) (1,0)(2,0) A A A (0,1) (1,1) (2,1) A A A (0,2) (1,2)(2,2)

in the memory (fortran) A A A A A A A A A (0,0) (0,1) (0,2) (1,0) (1,1) (1,2) (2,0) (2,1) (2,2)

count =3 old_type =MPI_REAL new_type =columntype

call mpi_type_contiguous(count, MPI_Datatype old_type, MPI_Datatype newtype,ierr);

Derived Data Types

15.06.2012

Handling Non-Contiguous Data



How to send only the red elements of V, while avoiding copying non-contiguous data to a temporary array? V(0) V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) V(9)



Derived Data Types

Define a new data type, in this case a vector with stride of two from original. vType

15.06.2012

MPI_Type_vector



old type

Similar to contiguous, but allows for regular gaps (stride) in the displacements. (MPI_INT)

new type blocklength=3 stride=5 count=2

Data constructor with MPI_Type_vector.

Derived Data Types

15.06.2012

MPI_Type_vector



In C, if we create a matrix with static memory allocation, we can say that data of the matrix will be in contiguous memory. • Suppose that, we want to send columns to the each task, instead of rows. double A[3][3]; A A A (0,0) (0,1) (0,2) A A A (1,0) (1,1) (1,2) A A A (2,0) (2,1) (2,2)

Derived Data Types

in the memory A A A A A A A A A (0,0) (0,1) (0,2) (1,0) (1,1) (1,2) (2,0) (2,1) (2,2)

We can use, MPI_Type_vector to create vector (strided) data type.

15.06.2012

MPI_Type_vector double A[3][3]; A A A (0,0) (0,1) (0,2) A A A (1,0) (1,1) (1,2) A A A (2,0) (2,1) (2,2)

in the memory A A A A A A A A A (0,0) (0,1) (0,2) (1,0) (1,1) (1,2) (2,0) (2,1) (2,2)

blocklength=1 stride=3 count=3

Derived Data Types

15.06.2012

MPI_Type_vector (C)

blocklength=1 stride=3 count=3

MPI_Type_vector(int count, int blocklength, int stride, MPI_Datatype old_type, MPI_Datatype *new_type);

Derived Data Types

15.06.2012

mpi_type_vector (Fortran)

blocklength=1 stride=3 count=3

mpi_type_vector(count, blocklength, stride, MPI_Datatype old_type, MPI_Datatype new_type,ierr);

Derived Data Types

15.06.2012

MPI_Type_vector (C) V(0) V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) V(9)



Sending reds

vType

blocklength = 1 blocklength= ?1 stride= ?2 count= ?3

stride

=2

count

=3

old_type

= MPI_DOUBLE

new_type

= vtype

MPI_Type_vector( count, blocklength, stride, old_type, &vType); MPI_Send(&V[2], 1, vType, dest, tag, MPI_COMM_WORLD);

Derived Data Types

15.06.2012

mpi_type_vector (Fortran) V(0) V(1) V(2) V(3) V(4) V(5) V(6) V(7) V(8) V(9)



Sending reds

type

blocklength = 1 blocklength= ?1 stride= ?2 count= ?3

stride

=2

count

=3

old_type

= MPI_INTEGER

new_type

= type

call mpi_type_vector(count,blocklength,stride,old_type,type,ierr); call mpi_send(V(2), 1, type, dest, tag, MPI_COMM_WORLD,ierr);

Derived Data Types

15.06.2012

MPI_Type_indexed



Indexed constructor allows one to specify a noncontiguous data layout where displacements between successive blocks need not be equal.

double A[4][4];

Derived Data Types

15.06.2012

A A A A (0,0) (0,1) (0,2)(0,3) A A A A (1,0) (1,1) (1,2)(1,3) A A A A (2,0) (2,1) (2,2)(2,3) A A A A (3,0) (3,1) (3,2)(3,3)

MPI_Type_indexed



Derived Data Types

This allows: • Gathering of arbitrary entries from an array and sending them in one message, or • Receiving one message and scattering the received message entries into arbitrary locations in an array.

15.06.2012

MPI_Type_indexed (C)

int MPI_Type_indexed( int count, int blocklength[], int indices[], MPI_Datatype old_type, MPI_Datatype *newtype ) • • •

Derived Data Types

count : number of blocks blocklength : number of elements in each block indices : displacement for each block, measured as number of elements 15.06.2012

mpi_type_indexed (Fortran)

mpi_type_indexed( count, blocklength(), indices(), MPI_Datatype old_type, MPI_Datatype newtype,ierr ) • • •

Derived Data Types

count : number of blocks blocklength : number of elements in each block indices : displacement for each block, measured as number of elements 15.06.2012

MPI_Type_indexed

old type blen[0]= 2

blen[1]= 3

blen[2]= 1

new type indices[0]=0

indices[1]=3 count= 3

Derived Data Types

15.06.2012

indices[2]=8

MPI_Type_indexed

• •

Suppose that, we have a matrix A(4x4) We want to send upper triangular matrix

double A[4][4]; A A A A (0,0) (0,1) (0,2)(0,3) A A A A (1,0) (1,1) (1,2)(1,3) A A A A (2,0) (2,1) (2,2)(2,3) A A A A (3,0) (3,1) (3,2)(3,3)

old type

= MPI_DOUBLE

new type = upper count

= 4

blocklen[ ] = (4, 3, 2, 1) indices[ ] = (0, 5, 10, 15)

MPI_Type_indexed(count, blocklen, indices, MPI_DOUBLE, upper )

Derived Data Types

15.06.2012

MPI_Type_commit

• •

Commits new datatype to the system. Required for all user constructed (derived) data types.

int MPI_Type_commit( MPI_Datatype *datatype ) MPI_TYPE_COMMIT( MPI_Datatype datatype,ierr )

Derived Data Types

15.06.2012

MPI_Type_free

• •

Deallocates the specified data type object. Use of this routine is especially important to prevent memory exhaustion if many data type objects are created, as in a loop.

int MPI_Type_free( MPI_Datatype *datatype ) MPI_TYPE_FREE( MPI_Datatype datatype,ierr )

Derived Data Types

15.06.2012

Packing Different Data

Derived Data Types



Sometimes, users need to send non-contiguous data in a single package.



MPI allows them to • explicitly pack data into a contiguous buffer before sending it, and • unpack it from a contiguous buffer after receiving.



Several messages can be successively packed into one packing unit.

15.06.2012

MPI_Pack (C) data to be buffered int MPI_Pack ( void *packdata, number of input data items datatype of each input data item output buffer start size of buffer, in bytes current position in buffer, in bytes communicator for packed message

Derived Data Types

int count, MPI_Datatype datatype, void *buffer, int size, int *position, MPI_Comm comm )

15.06.2012

mpi_pack (Fortran) data to be buffered MPI_PACK ( number of input data items datatype of each input data item output buffer start size of buffer, in bytes current position in buffer, in bytes communicator for packed message

Derived Data Types

packdata, count, MPI_Datatype datatype, buffer, size, position, MPI_Comm comm,ierr )

15.06.2012

Packing Data

char: (1 Byte)

integer: (4 Byte)

char c[25]: T o d a y int date[3]:

18

i s 7

a 2007

w o n d e r f u l integer array with 3 elements

buffer

Derived Data Types

15.06.2012

d a y !

char array with 25 element

Packing Data

buffer T o d a y

i s

a

w o n d e r f u l

d a y !

18

7

• At the beginning position=0 MPI_Pack(c, 25, MPI_CHAR, buffer, 37, 37 &position, MPI_Comm comm ) • position value is updated by MPI_Pack as position=25 MPI_Pack(date, 3, MPI_INT, buffer, 37, &position, MPI_Comm comm ) • position value is updated by MPI_Pack as position=37

Derived Data Types

15.06.2012

2007

Packing Data

buffer T o d a y

i s

a

w o n d e r f u l

d a y !

18

7

2007

• At the beginning position=0 call MPI_PACK(c, 25, MPI_CHARACTER, buffer, 37, 37 &position, MPI_Comm comm,ierr ) • position value is updated by MPI_Pack as position=25 MPI_Pack(date, 3, MPI_INTEGER, buffer, 37, &position, MPI_Comm comm,ierr ) • position value is updated by MPI_Pack as position=37

Derived Data Types

15.06.2012

Sending Packed Data

Derived Data Types



MPI_Send function used to send packed data,



MPI_PACKED type is used as datatype.



Size of the data must be specified in Bytes.



Now, buffer size is 37 Bytes

15.06.2012

Sending Packed Data

char: (1 Byte)

buffer T o d a y

i s

a

integer: (4 Byte)

w o n d e r f u l

MPI_Send( buffer, position, MPI_PACKED, dest, tag, MPI_Comm comm);

Derived Data Types

15.06.2012

d a y !

18

7

2007

Sending Packed Data

char: (1 Byte)

buffer T o d a y

i s

a

integer: (4 Byte)

w o n d e r f u l

d a y !

call MPI_SEND( buffer, position, MPI_PACKED, dest, tag, MPI_Comm comm,ierr)

Derived Data Types

15.06.2012

18

7

2007

Receiving Packed Data

Derived Data Types



MPI_Recv function used to receive packed data,



MPI_PACKED type is used as datatype.



Size of the data must be specified in Bytes.

15.06.2012

Receiving Packed Data

MPI_Recv( Rbuffer, 37, MPI_PACKED, source, tag, comm, &status);

Rbuffer T o d a y

Derived Data Types

i s

a

w o n d e r f u l

15.06.2012

d a y !

18

7

2007

Receiving Packed Data

MPI_RECV( Rbuffer, 37, MPI_PACKED, source, tag, comm, status,ierr);

Rbuffer T o d a y

Derived Data Types

i s

a

w o n d e r f u l

15.06.2012

d a y !

18

7

2007

MPI_Unpack (C) input buffer start int MPI_Unpack(void *buffer, size of buffer, in bytes current position in buffer, in bytes output buffer start number of items to be unpacked datatype of each output data item communicator for packed message

Derived Data Types

int size, int *position, void *packdata, int count, MPI_Datatype datatype, MPI_Comm comm )

15.06.2012

mpi_unpack (Fortran) input buffer start int MPI_UNPACK( buffer, size of buffer, in bytes

size,

current position in buffer, in bytes

position,

output buffer start

packdata,

number of items to be unpacked datatype of each output data item communicator for packed message

Derived Data Types

count, MPI_Datatype datatype, MPI_Comm comm,ierr )

15.06.2012

MPI_Unpack

• •

• •

Derived Data Types

The output buffer can be any communication buffer allowed in MPI_Recv. The buffer is a contiguous storage area containing size bytes, starting at address buffer. The value of position is the first location in the buffer occupied by the packed message. position is incremented by the size of the packed message, so that the output value of position is the first location in the buffer after the locations occupied by the message that was unpacked.

15.06.2012

Unpacking Packed Data (C)

MPI_Unpack( buffer, 37, &position, First value position=0 Updated to position=25 &Rc, 25, MPI_CHAR, MPI_Comm comm);

char Rc[25]: T o d a y

Derived Data Types

i s

a

15.06.2012

w o n d e r f u l

d a y !

Unpacking Packed Data (Fortran)

MPI_UNPACK( buffer, 37, position, First value position=0 Updated to position=25 Rc, 25, MPI_CHARACTER, MPI_Comm comm,ierr);

char Rc[25]: T o d a y

Derived Data Types

i s

a

15.06.2012

w o n d e r f u l

d a y !

Unpacking Packed Data (C)

MPI_Unpack( buffer, 37, &position, First value position=25 Updated to position=37 &Rdate, 3, MPI_INT, MPI_Comm comm);

int Rdate[3]:

Derived Data Types

18

15.06.2012

7

2007

Unpacking Packed Data (Fortran)

MPI_UNPACK( buffer, 37, position, First value position=25 Updated to position=37 Rdate, 3, MPI_INTEGER, MPI_Comm comm,ierr);

int Rdate[3]:

Derived Data Types

18

15.06.2012

7

2007

Programming Activities

Writing parallel MPI codes using following routines – Contiguous – Indexed – Vector – Pack-unpack

Derived Data Types

15.06.2012