On Tue, Apr 13, 2004 at 06:44:12PM +0200, Andy Manners wrote:
>We have just installed a small gigabit ethernet cluster using Linux
>2.4.21 and an e1000 gigabit ethernet driver. Performance was woeful
recent e1000 drivers seems to be tuned for low cpu load by default which
is bad for Beowulf.
We find these are good options for high bandwidth, and fairly low
latency (dual e1000's assumed) which you'd put into /etc/modules.conf
(/etc/modprobe.conf on a 2.6 kernel system)
options e1000 InterruptThrottleRate=15000,15000
Documentation/networking/e1000.txt has more info on other options.
Also, stick with the default MTU of 1500.
it has significantly better bandwidth and smoother bandwidth curves
than larger MTU (9000), and the interrupt load of MTU=1500 doesn't seem
much different.
>50usec. and 900 Mbit/sec. However, things do not look quite right and
>the controller hangs for larger message sizes without interrupt
>throttling.
we don't see any NIC hangs here. e1000 is a very stable hardware and
driver combo. Are you using NAPI in the kernel?
>Pointers for parameters to prod and tune in the 2.4 kernel also
>gratefully received.
there are webpages with tweaks to some kernel params (socket buffer
sizes, tcp latency) for beowulf activities, but none of them seem to
make any real difference at all... The driver options and which MPI you
use (LAM is best of course :-) are easily the biggest effect.
cheers,
robin
|
