On Tue, 5 Oct 2004, Pierre Valiron wrote:
> Has anybody some experience bonding Myrinet, Infiniband, etc,
> between powerful SMP 64-bit machines (quad Opterons or Power5) ?
What makes you think that a 64-bit machine is faster for this purpose
than a 32-bit one ?
When you say 'bonding' do you still think of making these interfaces
appear as normal network devices in Linux and then using the bonding
module and TCP/IP on top of this ? I doubt that this will work better
for exactly the same reasons as for Gigabit. Furthermore, as you
increase the number of CPUs that have can receive interrupts from the
NICs and have to process packets, you increase the amount of lock
contention for accessing shared data in the TCP/IP stack which can
actually make your CPUs look busy without any actual work being done
while they wait for revalidation of their cache lines and so on.
> which might be usable provided the network is fast (Myrinet,
> Infiniband, etc) and N is small enough.
This works well for applications where most (or all) of the
communication is done between neighbours. Otherwise, to reach a node
from another node, a message might need to pass through a few others
which will increase latency a lot, especially if the NIC is not
prepared to do this operation on its own and the host CPU(s) have to
actively be involved.
> I doubt this type of configuration is supported in present LAM/MPI.
Provided that you set up routing properly at IP level, it does work
with LAM for as long as I've used it. But performance might not be the
best...
I won't mention FNN as I'm sure that somebody else is just dying to
say something :-)
--
Bogdan Costescu
IWR - Interdisziplinaeres Zentrum fuer Wissenschaftliches Rechnen
Universitaet Heidelberg, INF 368, D-69120 Heidelberg, GERMANY
Telephone: +49 6221 54 8869, Telefax: +49 6221 54 8868
E-mail: Bogdan.Costescu_at_[hidden]
|
