BG/P specific extensions to MPI

IBM provides extensions to MPICH2 in order to ease the use of the BG/P hardware. These extensions start with MPIX instead of MPI. Currently, only a C/C++ interface and no Fortran interface is available.

In order to use the extensions, please include mpix.h:

          #include <mpix.h>

and compile your program with the usual mpi compiler wrapper (e.g. mpixlc, mpixlcxx, ...).

The following routines are available:

int MPIX_Cart_comm_create(MPI_Comm *cart_comm)
int MPIX_Pset_same_comm_create(MPI_Comm *pset_comm)
int MPIX_Pset_diff_comm_create(MPI_Comm *pset_comm)

unsigned MPIX_torus2rank(unsigned x,unsigned y,unsigned z,unsigned t)
void MPIX_rank2torus(unsigned rank,unsigned *x,unsigned *y,unsigned *z,unsigned *t)

unsigned MPIX_Comm_torus2rank(MPI_Comm comm,unsigned x,unsigned y,unsigned z,unsigned t)
void MPIX_Comm_rank2torus(MPI_Comm comm,unsigned rank,unsigned *x,unsigned *y,
unsigned *z,unsigned *t)

int MPIX_Get_properties(MPI_Comm comm, int *prop_array)
int MPIX_Get_property(MPI_Comm comm, int prop, int *result)
int MPIX_Set_property(MPI_Comm comm, int prop, int value)

Explanations:

MPIX_Cart_comm_create
creates a four-dimensional Cartesian communicator that mimics the exact hardware on which it is run. It will only work properly if the application runs on ALL nodes of a partition, i.e. if you reserve 512 nodes your application must not use less nodes. Because of MPICH2 dimension ordering, the associated arrays (i.e. coords, sizes, and periods) are in [t, z, y, x,] order so that the rank in cart_comm matches the rank in MPI_COMM_WORLD.

MPIX_Pset_same_comm_create
creates a communicator such that all nodes in the same communicator are served by the same I/O node.

MPIX_Pset_diff_comm_create creates a communicator such that all nodes in the same communicator are served by a different I/O node.

MPIX_torus2rank
returns the mapped rank based on the physical X, Y, Z, and T coords.

MPIX_rank2torus
returns the physical X, Y, Z, and T coords based on the mapped rank.

MPIX_Comm_torus2rank
returns the communicator rank based on the physical X, Y, Z, and T coords.

MPIX_Comm_rank2torus
returns the physical X, Y, Z, and T coords based on the communicator rank.

MPIX_Get_properties
retrieves the values of all the properties of the specified communicator.

MPIX_Get_property/MPIX_Set_property
gets/sets the corresponding property for the specified communicator.

For more information, please see the Blue Gene/P Application Development Redbook and the table IBM Blue Gene extensions to MPICH2.