Tracefile for MPI_Isend and MPI_Irecv
Dear all,
I am using trace of SimGrid to view the resource usage.
I create a cluster of 2 hosts and connect them by 2 uni-directional links (forward and backward). Then I use MPI_Send and MPI_Recv to exchange the data between them.
if (rank == 0) {
MPI_Send(sendbuf, bufsize, MPI_FLOAT, 1, 0, MPI_COMM_WORLD);
MPI_Recv(sendbuf, bufsize, MPI_FLOAT, 1, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
}
if (rank == 1) {
MPI_Recv(recvbuf, bufsize, MPI_FLOAT, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);
MPI_Send(sendbuf, bufsize, MPI_FLOAT, 0, 0, MPI_COMM_WORLD);
}
and I got the following figure from the tracefile Y axis is the percentage of bandwidth used and X axis is the timeline. The figure shows normal behavior of the software. One process use a link in one direction and the other process use the link in the remain direction. I want to use 2 links simultaneously, so I use asynchronous point to point functions (MPI_Isend and MPI_Irecv).
if (rank == 0) {
MPI_Irecv(recvbuf, bufsize, MPI_FLOAT, 1, 0, MPI_COMM_WORLD, &recv);
MPI_Isend(sendbuf, bufsize, MPI_FLOAT, 1, 0, MPI_COMM_WORLD, &send);
}
if (rank == 1) {
MPI_Irecv(recvbuf, bufsize, MPI_FLOAT, 0, 0, MPI_COMM_WORLD, &recv);
MPI_Isend(sendbuf, bufsize, MPI_FLOAT, 0, 0, MPI_COMM_WORLD, &send);
}
MPI_Wait(&send, MPI_STATUS_IGNORE);
MPI_Wait(&recv, MPI_STATUS_IGNORE);
and I got the following figure. All the traffic moves to one link and I cannot explain this. Please help me explain why only one link handle the traffic of two communication. testmpisendrecv.zip The attachment is the code and platform file I used to do the experiment. Thank you all.