Hi,
I have written a parallel code in C++/MPI. The parallel code uses strip
partitioning method to didvide the bigger block (a 3D cuboid) into several
smaller blocks/slices. Each smaller block is assigned to a processor for
updating data within it. Interface data has to be exchanged at each iteration.
Within a slice, all the planes (say rows) of data are marked red & black
alternately. Red rows are always at the both boundaries of the block/slice
hence only interfacial red rows require to exchange data but black rows do not,
as they need data from neighboruing red rows. Here is the pseudo code:
for(time=1'time<=Nt;time++)
{
do
{
/* overlapped_comm_comp_subroutine */
/* MPI_Allredue(..to find global max error..) */
} while(convergence criteria is met)
}
I wish to reduce the wall-clock runtime as much as possible by overlapping comm
with computaion. For this I used non-blocking MPI_Isend/Irecv with MPI_wait().
Still my code is not showing good scaling? What can be the reason for the poor
scalability? Is it due to the reason that my code is not well optimized? Is
there any other way I can optimize it more? Pls see the pseudo code for
overlapped_comm_comp_subrouine and suggest me if anything is wrong with the
code.
overlapped_comm_comp_subroutine:
if(rank is even)
{
/* update rightmost boundary */
MPI_Isend(..rightmost boundary to next neighbour..)
/* update last half red rows */
MPI_wait()
}
else if(rank is odd)
{
MPI_Irecv(..from prev neighbour..)
/* update first half black rows */
MPI_Wait()
}
if(rank is odd)
{
/* update leftmost red boundary */
MPI_Isend(..leftmost red boundary to prev neighbr..)
/* update first half red rows */
MPI_Wait()
}
else if(rank is even)
{
MPI_Irecv(..updated red bndry from next neghbr..)
/* update last half black rows */
MPI_wait()
}
// SAME THING IS REPEATED FOR OTHER HALF OF THE ROWS //
if(rank is even)
{
/* update leftmost boundary */
MPI_Isend(..leftmost boundary to prev neighbour..)
/* update first half red rows */
MPI_wait()
}
else if(rank is odd)
{
MPI_Irecv(..from next neighbour..)
/* update last half black rows */
MPI_Wait()
}
if(rank is odd)
{
/* update rightmost red boundary */
MPI_Isend(..rightmost red boundary to next neighbr..)
/* update last half red rows */
MPI_Wait()
}
else if(rank is even)
{
MPI_Irecv(..updated red bndry from prev neghbr..)
/* update first half black rows */
MPI_wait()
}
//END of Subroutine //
Thanks a lot!
Ravi R. Kumar
|
