On Apr 8, 2005, at 6:36 AM, Philip Mason wrote:
> I am running a LAM(7.1.1) application on Linux(RH7_IA32)/GM2 with
> the memory manager off.
>
> The LAM-"Installation Guide suggests configuring LAM with
> "-with-memory-manager=external" and then defining your own
> "sbrk"-routine.
Only for those who need it, for which cases are extremely rare.
> Alternatively, is it OK to configure LAM with
> "--with-memory-manager=none" and then use the
> MPI_ALLOC_MEM/MPI_FREE_MEM routines to allocate/deallocate any
> MPI-send/recieve buffers? I assume these routines automatically
> pin/upin memory when runing in "gm-mode"(or ib)?
Sometimes the world is full of coincidences. Someone asked me the
exact same question in an off-list e-mail just yesterday. :-)
I'll tell you the same thing I told him: yes, it should work. We have
not explicitly tested this configuration, but I don't see why it
wouldn't work. Let us know if you run into problems.
As you mentioned, the memory manager is to catch when the user free's
memory that has been secretly pinned by LAM/MPI. This allows LAM to
[secretly] unpin the memory before it actually goes back to the
operating system (if you don't unpin before returning memory to the OS,
Bad Things happen).
You can avoid this problem in two different ways:
1. the approach you outlined, where you effectively will never free() a
buffer that may have secretly been pinned (instead, you use
MPI_FREE_MEM, which will allow LAM to explicitly unpin it)
2. use the rpi_gm_nopin SSI parameter to tell LAM to never pin user
buffers. This is the default on Solaris, for example (because Solaris
in incapable of pinning arbitrary user buffers).
> Have tried the above, seems to work. However, I also tried using
> non-MPI allocate/deallocate routines for MPI-send/recieve buffers and
> still works OK -> was under the impression freeing buffers without
> explicitly unpinning memory is liable to fail? - worried this might
> have worked by luck.
It was probably luck. For small messages (by default, <=16k), we
memcpy from the user buffer into pinned buffers. LAM only pins "large"
messages (i.e., >16k).
> Side-question: If I allocate an array BUFFER and then MPI-send/recieve
> this BUFFER in sub-sections - is it sufficient to MPI_ALLOC_MEM/FREE
> BUFFER and not have to MPI_ALLOC_MEM/FREE each subsection
> sent/recieved?
Yes.
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
