Hi Again Jeff,
I was thinking about the proposed two thread solution which I had
originally posted. This is of course only efficient if the MPI_recv call
is also implemented with signaling and condition variables? It's no use
removing the polling from the threads if the blocking recv polls too?
How does lam implement a blocking receive is it done with signaling or
with polling?
Thanks
-Mike
Jeff Squyres wrote:
> On Sat, 5 Jun 2004, Michael Lees wrote:
>
>> I'm attempting to design a system which has an asynchronous MPI
>> interface in the form of a 'communication port'. This port interfaces
>> with two queues, send and receive. The port should always listen to
>> MPI, ie., always have some form of MPI_RECV posted, and place any
>> message that comes from MPI into the receive queue of the port. The
>> port should also monitor the send queue and make an MPI_SEND command
>> when items are placed into the send queue.
>>
>> Now intuitively I'd have a port with two fairly simple threads, send
>> and receive which use condition variables to wake the appropriate
>> threads.
>>
>> Send Thread
>> {
>> Wait(sendQinsert)
>> get Item from sendQ
>> MPI_Send...
>> }\\loop
>>
>>
>> Receive Thread
>> {
>> MPI_RECV...(block until Recv arrives)
>> put msg in recv queue
>> signal(recvQinsert)
>> }\\loop
>>
>>
>> Now this isn't possible with the current lam-mpi as it doesn't support
>> MPI_THREAD_MULTIPLE. I'm sure I can't be the only person trying to
>> perform asynchronous send and receives to an MPI process? Is there a
>> standard method to implementing this kind of design, either by
>> collapsing into a single sendANDreceive thread or by applying
>> appropriate mutex locks?
>
>
> This is a difficult problem without proper threading. :-(
>
> It can be done by collapsing into a single thread, but then you have the
> problem of having to poll on two different things: when a new send
> request arrives on your queue and when data is available from MPI_Recv.
> Since these are unlike tasks, you cannot block on them and instead have
> to poll. This is easy enough to do, but then you have the problem of the
> polling frequency -- too high, and you steal too many cycles from your
> application. Too low, and you increase latencies on message passing
> (which may or may not be important to you).
>
> The other typical approach is to simply have a "progress()" routine that
> processes pending sends and pending receives, and you call that from
> your main thread wherever you want. I.e., the polling frequency becomes
> controlled through targetted invocations in the main thread.
>
> I realize that neither of these are optimal, but these are pretty much
> your options with single-threaded MPI implementations. :-\
>
> That being said, if you come up with something more clever, I'm sure
> this list would love to hear about it! :-)
>
This message has been scanned but we cannot guarantee that it and any
attachments are free from viruses or other damaging content: you are
advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.
|
