On Sat, 5 Jun 2004, Michael Lees wrote:
> I'm attempting to design a system which has an asynchronous MPI
> interface in the form of a 'communication port'. This port interfaces
> with two queues, send and receive. The port should always listen to MPI,
> ie., always have some form of MPI_RECV posted, and place any message
> that comes from MPI into the receive queue of the port. The port should
> also monitor the send queue and make an MPI_SEND command when items are
> placed into the send queue.
>
> Now intuitively I'd have a port with two fairly simple threads, send and
> receive which use condition variables to wake the appropriate threads.
>
> Send Thread
> {
> Wait(sendQinsert)
> get Item from sendQ
> MPI_Send...
> }\\loop
>
>
> Receive Thread
> {
> MPI_RECV...(block until Recv arrives)
> put msg in recv queue
> signal(recvQinsert)
> }\\loop
>
>
> Now this isn't possible with the current lam-mpi as it doesn't support
> MPI_THREAD_MULTIPLE. I'm sure I can't be the only person trying to
> perform asynchronous send and receives to an MPI process? Is there a
> standard method to implementing this kind of design, either by
> collapsing into a single sendANDreceive thread or by applying
> appropriate mutex locks?
This is a difficult problem without proper threading. :-(
It can be done by collapsing into a single thread, but then you have the
problem of having to poll on two different things: when a new send request
arrives on your queue and when data is available from MPI_Recv. Since
these are unlike tasks, you cannot block on them and instead have to poll.
This is easy enough to do, but then you have the problem of the polling
frequency -- too high, and you steal too many cycles from your
application. Too low, and you increase latencies on message passing
(which may or may not be important to you).
The other typical approach is to simply have a "progress()" routine that
processes pending sends and pending receives, and you call that from your
main thread wherever you want. I.e., the polling frequency becomes
controlled through targetted invocations in the main thread.
I realize that neither of these are optimal, but these are pretty much
your options with single-threaded MPI implementations. :-\
That being said, if you come up with something more clever, I'm sure this
list would love to hear about it! :-)
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
