Jeff Squyres wrote:
> Internal LAM buffering is not going to be your big issue here; aside
> from some MPI_Request and associated data structures, LAM doesn't
> really allocate much per message. More specifically, the overhead for
> each message is essentially constant, regardless of size. Whenever
> possible, LAM will try to receive directly into your buffer (the
> exceptions are small messages on low-latency networks like IB and GM).
>
> So posting a lot of receives is not going to dramatically increase the
> memory usage in LAM itself -- it's going to increase your process'
> memory usage, because assumedly you have unique message buffers
> associated with each of those requests.
The system is a multi-threaded design and I'm trying to determine the
limits of the design.
ie., with x number of messages from the slaves the master thread won't
be able to service its MPI recv thread fast enough and frequently enough
to ensure all x*16 messages are received before the next x*16 messages
are sent (ie., within 100 milliseconds).
So really I'm interested in the CPU overhead associated with context
switching, buffering etc.
Presumably x does exist, there must be some number of messages which
will overload the system so as to cause a 'backlog' in the MPI buffer?
> You can't really monitor the internals of LAM/MPI itself, but you can
> monitor the entire process with getrusage(). See the man page for
> details.
>
Is getrusage only milliseconds resolution? GetTimeOfDay seems to be
better resolution but it doesn't do CPU time?
>
> On Nov 8, 2005, at 9:43 AM, Michael Lees wrote:
>
>
>>Hi all,
>>
>>I'm trying to understand lam performance, in particular the overhead
>>associated with communication.
>>I have a simple program with 17 mpi processes, 16 of the processes
>>(call
>>them slaves) operate in cycles of message generation. Each cycle begins
>>with the process sleeping for 100 miliseconds and then waking and
>>sending 6 messages to the single master node.
>>This results in the master receiving a burst of 16*6 messages every 100
>>milliseconds. All the communication uses asynchronous sends and
>>receives. Ideally I'd like all 96 messages of a cycle to be received
>>before the next cycle starts.
>>
>>Is there anyway I can monitor the size of the internal lam buffers and
>>their current usage. I want to try and figure out the maximum number of
>>messages that I can send and receive reliably within the 100
>>milliseconds period.
>>
>>
>>Thanks
>>
>>--
>>Michael Lees
>>
>>
>>This message has been checked for viruses but the contents of an
>>attachment
>>may still contain software viruses, which could damage your computer
>>system:
>>you are advised to perform your own checks. Email communications with
>>the
>>University of Nottingham may be monitored as permitted by UK
>>legislation.
>>
>>_______________________________________________
>>This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>>
>
>
This message has been checked for viruses but the contents of an attachment
may still contain software viruses, which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.
|
