c - How does MPI_IN_PLACE work with MPI_Scatter?

Question

What exactly does MPI_IN_PLACE do when given as an argument to MPI_Scatter and how should it be used? I can't make sense of man MPI_Scatter:

When the communicator is an intracommunicator, you can perform a gather operation in-place (the output buffer is used as the input buffer). Use the variable MPI_IN_PLACE as the value of the root process recvbuf. In this case, recvcount and recvtype are ignored, and the root process sends no data to itself. Because the in-place option converts the receive buffer into a send-and-receive buffer, a Fortran binding that includes INTENT must mark these as INOUT, not OUT.

What I want to do is use the same buffer that contains the data on the root as the receive buffer at each other process (like in MPI_Bcast). Will MPI_Scatter with MPI_IN_PLACE let me do this?

Answer 1

The sendbuf of MPI_scatter is only relevant on the root according to mpich,

sendbuf -- address of send buffer (choice, significant only at root)

From this discussion,

For scatter/scatterv, MPI_IN_PLACE should be passed as the recvbuf. For gather and most other collectives, MPI_IN_PLACE should passed as the sendbuf.

You therefore need to use MPI_IN_PLACE in the recv buffer location on the root process, e.g.

if (rank == iroot)
     MPI_Scatter(buf, sendcount, MPI_Datatype, MPI_IN_PLACE, sendcount,  
                 MPI_Datatype, iroot, MPI_COMM_WORLD);
else
     MPI_Scatter(dummy, sendcount, MPI_Datatype, buf, sendcount, MPI_Datatype,  
                 iroot, MPI_COMM_WORLD);

You could then use buf in the send on root and the same buf in the recv position on each other process. The dummy buffer on the receiving processors could probably also be replaced by MPI_IN_PLACE.