LAM/MPI logo

LAM/MPI General User's Mailing List Archives

  |   Home   |   Download   |   Documentation   |   FAQ   |   all just in this list

From: Robin Humble (rjh_at_[hidden])
Date: 2003-11-10 12:51:33


On Mon, Nov 10, 2003 at 01:27:22PM +0100, jess michelsen wrote:
>I replaced the Intel gigabit driver (e1000) version 4.3.2-k1 NAPI by the
>latest version (5.2.20). Also, I replaced the OS kernel
>(linux-2.4.18-14) by linux-2.4.20-20.8. Since then, the MPI

2.4.22's e1000 driver has much worse performance than the 4.4.12-k1
version with a 2.4.21-pre2 kernel that we are currently using.
NAPI or non-NAPI really doesn't seem to make any difference for MPI
codes.
I have yet to sort out why newer kernels and/or newer e1000 drivers
are lame.

we don't see any TCP or MPI hangs with our drivers and kernels.

>communication between two nodes seems to be much more stable, i.e. I
>didn't see any 'hangs' until now. However, both latency and bandwidth
>are deteriorated. Latency is now around 250 usec and bandwidth around
>300 Mbit/sec. (they were 120 usec,600Mbit/sec).

All of those numbers are poor if you are talking about Netpipe
TCP performance http://www.scl.ameslab.gov/netpipe/.
Typically you should see ~65usec latency and ~800Mbit/s with the e1000
via a direct machine-machine link or via a switch. I'm haven't run any
MPI Netpipe's for a while so I'm not sure what to expect for those.

>3) Has anybody studied, how the parameters for the e1000 driver (which
>are set when the ethernet devices are activated - the e1000 driver is a
>module, not compiled into the kernel) affect performance. Is there an

try asking the e1000 driver development people at Intel. They are
friendly :)

this is a bit old, but may also be useful:
   http://www.cs.uni.edu/~gray/gig-over-copper/hsln-lcn.ps

cheers,
robin