Hi,
I just looked at the LAM 7.0.3 source and found that MPI_Alltoall eventually
boils down to this code:
err = MPI_Irecv(rbuf, rcount, rdtype, src, rtag, comm, &req);
if (err != MPI_SUCCESS) {
return(lam_errfunc(comm, BLKMPISENDRECV, err));
}
err = MPI_Send(sbuf, scount, sdtype, dest, stag, comm);
if (err != MPI_SUCCESS) {
return(lam_errfunc(comm, BLKMPISENDRECV, err));
}
err = MPI_Wait(&req, status);
if (err != MPI_SUCCESS) {
return(lam_errfunc(comm, BLKMPISENDRECV, err));
}
i.e
A non-blocking RECEIVE followed by a blocking SEND and a WAIT.
From this is it obvious that using the same buffer for send and receive is
exceedingly dangerous. The program is non-deterministic in that the state of the
data in the shared buffer cannot be relied upon. The data being sent can be
overwritten by the receive part way through and the receiving node will
consequently get the wrong data.
I would suggest using different buffers, then copy the data after the
MPI_Alltoall completes.
The user is only free to modify the SEND and/or RECEIVE buffer AFTER the
MPI_Alltoall function COMPLETES. NOT DURING IT, which is what you are doing by
having the same address for these buffers.
Regards
Neil
Shi Jin wrote:
> Hi there,
>
> We are having a problem on the MPI_ALLTOALL call.
> We have a code developed by somebody else, which uses
> the MPI_ALLTOALL like this:
> call
> MPI_ALLTOALL(temp,block_c,dtype1,temp,block_c,dtype1,comm,ierr)
>
> Note that the sending and receving buffer are the
> same.
> His code runs well with lam-6.5.9.
>
> But after we upgraded our lam to version 7.0.4, the
> code is giving wrong results. I am nervous that the
> sending and receiving buffer are the same. I guess the
> data in temp could be overwritten by new data it
> receives before it can send out its own. If I change
> the call to
> MPI_ALLTOALL(temp,block_c,dtype1,newtemp,block_c,dtype1,comm,ierr)
> It is working again.
>
> So I guess what I am worrying is about right.
> But I don't understand why the code was working
> correctly with the old verion of lam. Is there any
> changes in the implementation of the MPI_ALLTOALL
> call?
>
> Thank you very much.
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - More reliable, more storage, less spam
> http://mail.yahoo.com
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
--
+-----------------+---------------------------------+------------------+
| Neil Storer | Head: Systems S/W Section | Operations Dept. |
+-----------------+---------------------------------+------------------+
| ECMWF, | email: neil.storer_at_[hidden] | //=\\ //=\\ |
| Shinfield Park, | Tel: (+44 118) 9499353 | // \\// \\ |
| Reading, | (+44 118) 9499000 x 2353 | ECMWF |
| Berkshire, | Fax: (+44 118) 9869450 | ECMWF |
| RG2 9AX, | | \\ //\\ // |
| UK | URL: http://www.ecmwf.int/ | \\=// \\=// |
+--+--------------+---------------------------------+----------------+-+
| ECMWF is the European Centre for Medium-Range Weather Forecasts |
+-----------------------------------------------------------------+
|
