Does this application have a lot of pending non-blocking sends and
receives ongoing when the process fails? LAM *should* try to release
pinned memory in order to malloc/pin more, and therefore it should only
fail in this situation if all the pinned GM memory is actively being
used when you are trying to alloc more.
On Nov 30, 2004, at 2:41 PM, Yaakoub Y El Khamra wrote:
>
>
> Greetings
> When running mpi code on 32 procs, with gm, lam 7.1.1 gives me the
> following error:
> -----------------------------------------------------------------------
> ------
> It seems that rank 13 was not able to allocate additional
> DMA-accessible memory for Myrinet. DMA-accessible memory is memory
> which the Myrinet cards can access. Typically, OS's have fixed limits
> on how the total amount of DMA memory can be allocated at one
> time. Long MPI messages require that a large amount of DMA-accessible
> memory be allocated. If possible, try using smaller messages or
> adjusting the OS's DMA limit. :-(
>
> GM failed to allocate a DMA block of 16428 bytes.
> -----------------------------------------------------------------------
> ------
>
> The same code runs without any problems on mpich 1.2.6..13b (with gm)
> and also runs without any problems with lam when rpi is set to tcp. Any
> ideas, suggestions?
>
>
> NB This is on CCT's supercomputer: supermike, x86 arch, and everything
> was compiled with intel 8.0 C/C++/F90 compilers.
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Yaakoub Y El Khamra
> System Analyst II
> Center for Computation and Technology
> 324 Johnston Hall
> Louisiana State University
> Baton Rouge, Louisiana 70803
> voice: 225 578 8904
> fax: 225 578 5362
> aim: warp mechanic
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
