On Sat, 26 Jun 2004 hye_at_[hidden] wrote:
> I'm debugging my parallel application under LAM 6.5.9/MPI 2 C++/, Redhat
> 7.3 Beowulf cluster.
<mandatory-question>
Can you upgrade to LAM 7.0.6? 6.5.9 is actually quite old. Although it's
generally stable (it's been out in the wild for quite some time and we're
not aware of any major problems with it), 7.0.6 has a lot of new
functionality and addresses some known issues with the 6.5 series.
Also, we don't formally support the 6.5 series anymore.
</mandatory-question>
> The problem seems to be that some "later" messages are received prior to
> the "previous" sent messages. Here's the structure of my code:
>
> Sender processors:
> ... MPI_Isend(...)/MPI_Send(...) with various tag...
>
> Recver processors:
> ...
> MPI_Iprobe(MPI_ANY_SOURCE, MPI_ANY_TAG, MPI_COMM_WORLD, &rc,
> &status);
> if (rc) {
> MPI_Recv(&msg, sizeof(msg), MPI_BYTE, status.MPI_SOURCE,
> status.MPI_TAG, MPI_COMM_WORLD, &status1);
> ...
> }
This code looks ok. But you might consider against using Probe and/or
MPI_ANY_TAG -- see below.
> My question is: is the First-sent-First-Received can be guaranteed? For
> example, if one processor sends out message to other processor using
> MPI_Isend or MPI_Send or both, in the sequence of TAG1, TAG2, TAG1,
> TAG3, TAG2. Can it be guaranteed that, in the receiver side, the
> sequence is still TAG1, TAG2, TAG1, TAG3, TAG2?
No.
MPI only guarantees that the order of messages sent with the same
signature (i.e., sender and receiver, communicator, and tag) are ordered.
Hence, it is possible (and legal) for MPI to allow messages with different
tags to be delivered out of order (even with ANY_TAG).
However, I recommend against using MPI_Probe and/or MPI_ANY_TAG.
<shameless plug>
In this month's Cluster World magazine (July 2004), I write about my Top
10 List of Evils to Avoid in Parallel ("In Parallel, Everyone Hears You
Scream"). MPI_PROBE is #8; MPI_ANY_SOURCE is #3 (numbers 10-6 are this
month; 5-1 are next month).
</shameless plug>
>From your code, it looks like you're attempting to not receive until the
message has actually arrived. MPI actually gives you a better way to
effect this kind of behavior -- use non-blocking receives. Post a few
non-blocking receives for the messages that you're expecing and then use
the various flavors of MPI_TEST and MPI_WAIT to know when they have
actually been received. Indeed, using MPI_PROBE frequently causes the MPI
implementation to perform an additional memory copy (LAM does, at least).
This is because LAM has to receive the message into a temporary buffer and
then when you finally post the matching MPI_RECV, LAM copies it out of the
temporary buffer into the destination buffer.
So try to avoid MPI_PROBE.
MPI_ANY_SOURCE isn't as bad, but you should avoid it if you can. I'll
skip the rationale here; using it (or not) is probably not the problem
that you're seeing here.
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
