On Nov 21, 2005, at 10:52 AM, Jaime Perea wrote:
> First I like the most lam-mpi, so I decided to install it. But
> since it seems that the future is in open-mpi I decided to
> check it also. During the last few months we have been
> using mpich (well, nobody's perfect). After doing some
> testing with an old version of link-checker I found
>
> latency bandwidth
> lam 10.1 215
> openmpi 9.5 190
> mpich 9.5 216
>
> My questions:
>
> Is the higher latency of lam a result of having the
> lamd stuff also? do you think that those values
> will affect my codes in any sense?. I mean for me
> those results are pretty close and I suppose that
> at the end they are the more or less the same.
> I have to say that openmpi is built with the intel
> compilers while the others are gcc.... something
> more to check :-)
The higher latency of LAM/MPI is due to some design choices in how we
setup communication for small messages. We do a little bit more work
than MPICH-gm, so our latency is a little bit higher. Open MPI has a
much cleaner send path for short messages, so it performs better than
LAM/MPI and usually quite comparably (on modern hardware, we're often
a tenth of a microsecond or so faster) with MPICH-gm. Both LAM/MPI
and MPICH-gm play some really nasty linker tricks to intercept sbrk
() / munmap() so that they can lazily unpin memory. Open MPI instead
uses a rolling pin / rdma / unpin protocol by default. The advantage
is that we don't have to play evil linker tricks as much. The
disadvantage is a lower peak bandwidth, especially on slightly older
hardware (it's a function of the PCI bus performance).
If you did not enable thread support for Open MPI, there is the
ability to enable the lazy unpin behavior in Open MPI - I believe
something like:
mpirun -mca mpi_leave_pinned 1 -mca mpool_base_use_mem_hooks 1
<normal options>
should do the trick. We're still working on this mode of operation
-- calls to free() are much more expensive than we would like and
more tuning is required (which is why this is not the default mode of
operation for 1.0).
Brian
--
Brian Barrett
LAM/MPI developer and all around nice guy
Have a LAM/MPI day: http://www.lam-mpi.org/
|
