On Thu, 16 Jun 2005 08:11:44 -0600, Galen Arnold <arnoldg_at_[hidden]>
wrote:
> Is anybody else trying out new lam builds [nightly snapshots] with
> infiniband?
>
> I'm seeing sharp scaling performance drops in bandwidth as I scale from
> 8x2 -> 16x2 [couple hundred MB/s drops to 10s of MB/s]. mpich2-1.0.1
> doesn't have this issue and performs well out to at least 64x2.
>
> If anybody's got hints on ways to get better performance from:
>
> setenv LAM_MPI_SSI_rpi ib
>
> ...I'd like to hear them.
>
> -Galen
I'd certainly like to see a bit more about what you are running when you
see the performance drop; I have access to a 64-node IB cluster that I
could probably throw your test onto with little difficulty, and then we
could compare notes.
Of course, I would need to know the following for the comparison to be
particularly meaningful (or possible):
* Compiler used (gcc/Intel/Pathscale)
* OS (Linux?)
* IB Driver (Mellanox IB Gold 1.7?)
* What does your IB fabric topology look like? (Just one switch/many,
etc?)
--
Troy Telford
Linux Networx
ttelford_at_[hidden]
(801) 649-1356
|
