Hello,
Thanks for the reply!
Below is the subroutine I am using for data exchange between different
processes. In my code, I need to solve for 101x101x101 points in a 3D domain.
For this I defined a 3D array T[101][101][10] dynamically and to parallelize
the problem I divided T[Nz][Nx][Ny], along Nz, into several slices. Each
processor works on a slice and needs interface data from neighbouring nodes.
For exchanging interface data, I am using non-blocking MPI_Send/MPI_Recv, see
the subroutine below:
void exchange_interface_data_T(.....)
{
MPI_Status status;
MPI_Request request;
if(rank%2==0 && rank != num_processes-1){
MPI_Isend(&T[local_Nz][0][0], Nx*Ny, MPI_DOUBLE, rank+1, comm_tag,
MPI_COMM_WORLD,&request);
MPI_Request_free(&request);
}
else if(rank%2==1){
MPI_Irecv(&T[0][0][0],Nx*Ny,MPI_DOUBLE,rank-
1,comm_tag,MPI_COMM_WORLD,&request);
MPI_Wait(&request,&status);
}
if(rank%2==1){
MPI_Isend(&T[1][0][0],Nx*Ny, MPI_DOUBLE, rank-1,comm_tag+51,
MPI_COMM_WORLD,&request);
MPI_Request_free(&request);
}
else if(rank%2==0 && rank != num_processes-1){
MPI_Irecv(&T[local_Nz+1][0]
[0],Nx*Ny,MPI_DOUBLE,rank+1,comm_tag+51,MPI_COMM_WORLD,&request);
MPI_Wait(&request,&status);
}
if(rank%2==0 && rank != 0){
MPI_Isend(&T[1][0][0],Nx*Ny,MPI_DOUBLE,rank-
1,comm_tag+101,MPI_COMM_WORLD,&request);
MPI_Request_free(&request);
}
else if(rank%2==1 && rank != num_processes-1){
MPI_Irecv(&T[local_Nz+1][0]
[0],Nx*Ny,MPI_DOUBLE,rank+1,comm_tag+101,MPI_COMM_WORLD,&request);
MPI_Wait(&request,&status);
}
if(rank%2==1 && rank != num_processes-1){
MPI_Isend(&T[local_Nz][0]
[0],Nx*Ny,MPI_DOUBLE,rank+1,comm_tag+201,MPI_COMM_WORLD,&request);
MPI_Request_free(&request);
}
else if(rank%2==0 && rank != 0){
MPI_Irecv(&T[0][0][0],Nx*Ny,MPI_DOUBLE,rank-
1,comm_tag+201,MPI_COMM_WORLD,&request);
MPI_Wait(&request,&status);
}
}
This is how I am approaching data exchange between neighbouring nodes (slices).
Am I doing something wrong in data exchange? Pls suggest me.
Thanks a lot!
Ravi R. Kumar
Quoting Brian Barrett <brbarret_at_[hidden]>:
> On Mar 20, 2005, at 12:12 AM, Kumar, Ravi Ranjan wrote:
>
> > I wrote a code in C++ using MPI. It works fine and gives correct
> > result for
> > smaller 3D array size case for e.g. T[51][51][51]. However, my code
> > hangs when
> > I try to run the same for larger size case i.e T[101][101][101] with
> > an error
> > message as below:
> >
> > MPI_Recv: message truncated (rank 0, MPI_COMM_WORLD)
> > Rank (0, MPI_COMM_WORLD): Call stack within LAM:
> > Rank (0, MPI_COMM_WORLD): - MPI_Recv()
> > Rank (0, MPI_COMM_WORLD): - main()
>
> <snip>
>
> > I read sometime ago that this may be due to mismatch in number of data
> > sent and
> > number of data received in MPI_Send/MPI_Recv process. I have checked
> > this thing
> > many times and found no mismatch in number of data exchanged, still I
> > am
> > getting this error. What can be the reason for this? Could anyone
> > please
> > explain?
>
> The reason is exactly as you surmised. For some reason, a message has
> arrived that is bigger than the buffer you posted. It's hard to tell
> why this is occurring, but I would look carefully at your send/recv
> pairs again. These are hard ones to debug, as LAM is in an error
> condition and doesn't give you much information about what happened. I
> notice you are using blocking receives - this helps a little bit, in
> that you can print out what messages are being printed (and their
> sizes) and you can print out the size of the buffer you are providing
> to MPI_Recv. If you send a big message and post an ANY_SOURCE recv,
> Murphy's law pretty much guarantees it will happen in the worst order
> possible.
>
>
> Hope that helps,
>
> Brian
>
> --
> Brian Barrett
> LAM/MPI developer and all around nice guy
> Have an LAM/MPI day: http://www.lam-mpi.org/
>
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>
|
