Hi Andy!
Seems like the problems, we initially had with the e1000. All problems,
both latency/bandwidth and stability, were resolved by installing the
newest Intel e1000 driver version. It is now 5.2.30.1. In our
applications, the communications are ultimately controlled by the
computations, so we can safely increase InterruptThrottleRate all the
way. With NetPipe, we have recorded latencies as low as 16-32
microseconds and bandwidth up to just below 900 Mbit/sec. We have gained
a little bit of performance by setting both Initial and Absolute write
delay to 32 microseconds, so several sends can be performed at one
interrupt.
Best regards, Jess Michelsen
On Tue, 2004-04-13 at 18:44, Andy Manners wrote:
> We have just installed a small gigabit ethernet cluster using Linux
> 2.4.21 and an e1000 gigabit ethernet driver. Performance was woeful
> using the default configuration with 250 usec. small message latencies
> and only 200-300 Mbit/sec throughput for large messages. Some
> experimentation with the driver parameters improved things somewhat to
> 50usec. and 900 Mbit/sec. However, things do not look quite right and
> the controller hangs for larger message sizes without interrupt
> throttling.
>
> Can someone point us towards information on tuning gigabit ethernet on
> linux for numerical cluster purposes? That is, where low latencies are
> important and maximum throughput less so?
>
> In particular, is it possible to set minimum latencies for small
> messages (blocking mpi calls) and coalescing for message sizes
> exceeding the packet size?
>
> Pointers for parameters to prod and tune in the 2.4 kernel also
> gratefully received.
>
> Apologies if the subject is slightly off topic.
>
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
|
