c++ - How to create new Type in MPI

Question

I am new to MPI and I want to create a new datatype for Residence struct. I just want to see if I can create the new type right way. struct Residence { double x; double y; };

My new MPI Type

MPI_Datatype createRecType()
{
    // Set-up the arguments for the type constructor
    MPI_Datatype new_type;

    int count = 2;
    int blocklens[] = { 1,1 };

    MPI_Aint indices[2];
    //indices[0]=0;
    MPI_Type_extent( MPI_DOUBLE, &indices[0] );
    MPI_Type_extent( MPI_DOUBLE, &indices[1] );

    MPI_Datatype old_types[] = {MPI_DOUBLE,MPI_DOUBLE};

    MPI_Type_struct(count,blocklens,indices,old_types,&new_type);
    MPI_Type_commit(&new_type);

}

Answer 1

You've almost got it right except that indices is supposed to give the offset of each structure field in bytes from the beginning of the structure. The correct way to construct such a type would be to use the offsetof operator, defined in stddef.h:

#include <stddef.h>  // or <cstddef> for C++

struct Residence
{
   double x;
   double y;
};

MPI_Datatype createRecType()
{
    // Set-up the arguments for the type constructor
    MPI_Datatype new_type;

    int count = 2;
    int blocklens[] = { 1,1 };

    MPI_Aint indices[2];
    indices[0] = (MPI_Aint)offsetof(struct Residence, x);
    indices[1] = (MPI_Aint)offsetof(struct Residence, y);

    MPI_Datatype old_types[] = {MPI_DOUBLE,MPI_DOUBLE};

    MPI_Type_struct(count,blocklens,indices,old_types,&new_type);
    MPI_Type_commit(&new_type);

    return new_type;
}

While this would suffice for that particular structure, in general one has to adjust the structured type length in order to account for any trailing padding that the compiler might insert at the end of the structure. This is only necessary if one wants to send multiple items of that structured type, i.e. an array of structure elements. The old way to do that was to add a third member to the structure of type MPI_UB (UB comes from Upper Bound) and set the offset of that member to be equal to sizeof(struct Residence) (padding is accounted in the structure size as returned by sizeof). The modern way is to use MPI_Type_create_resized, which creates a new MPI type with the same type signature as the original one but with a different extent:

    MPI_Type_struct(count,blocklens,indices,old_types,&new_type);
    // Create a resized type
    MPI_Type resized_new_type;
    MPI_Type_create_resized(new_type,
                            // lower bound == min(indices) == indices[0]
                            indices[0],
                            (MPI_Aint)sizeof(struct Residence),
                            &resized_new_type);
    MPI_Type_commit(&resized_new_type);
    // Free new_type as it is no longer needed
    MPI_Type_free(&new_type);

    return resized_new_type;

Only the relevant code lines are shown. The code above assumes that indices[0] gives the offset of the first structure element. One could instead use MPI_Type_get_extent to get the true lower bound and that would work for structure types with negative offsets. It is not necessary to commit new_type as it is only used to construct the resized type. It is also not necessary to keep it around and that's why it is freed after resized_new_type has been created.