Hi --
LAM/MPI over Infiniband is ready in the repository. It should be in the
next release (7.1) of LAM. There is no formal documentation available yet,
but I am including a few in this email.
You can download the LAM/MPI source from the repository
(http://www.lam-mpi.org/svn/) -- you can go either for the nightly tarball
or the subversion checkout.
While most of the installation instructions are the same as documented on
the LAM website, some tips for the Infiniband module would be as follows:
- : Configure time switches : -
--------------------------------------
--with-rpi=ib
This allows the Infiniband module to be the default runtime module during
mpirun.
--with-rpi-ib=<your/ib/installation/dir>
This specifies where your Infiniband installation dir is, which will be
used by LAM to pick IB libraries.
An example to sum up the above cases would be:
./configure --with-rpi-ib=/usr/mellanox --with-rpi=ib
-: Runtime switches to mpirun :-
------------------------------------
-ssi rpi=ib
This allows you to pick up Infiniband module during runtime. Other modules
available are "tcp", "gm", "lamd". If you are using the configure time
switch --with-rpi=ib, then "ib" would be the default module and you won't
need this runtime switch here. Only if you are building LAM with different
modules, then you can select one during runtime using this switch.
-ssi rpi_ib_tinymsglen=<NUM>
This allows you to configure the "eager" or the "tiny" message length for
Infiniband module. Default is 1024 (1K)
-ssi rpi_ib_num_envelopes=<NUM>
This allows you to play with the number of preposted envelopes to the
receive queue for IB. Default is 64. It should not be more than 256
usually (too much memory consumption)
-ssi rpi_ib_port=<NUM>
This allows you to specify a specific IB port you want to use. If not
given, the LAM IB module will itself search for an available port.
-ssi rpi_ib_hca_id=<STRING>
This allows you to specify a specific "hca id", in case you have multiple
IB cards and hcas. If not provided, LAM IB module will search for the
first one it gets. (Usually its "Infinihost0" for Mellanox hcas)
Having said all these, you *won't* need most of the above runtime
switches.
So in a nutshell, installation of LAM would be:
---------------------------------------------------------------
$ ./configure --with-rpi-ib=/usr/mellanox --with-rpi=ib
$ make
$ make install
.....
$ mpirun prog
----------------------------------------------------------------
Known problems
----------------
While LAM with IB works for mostly everything, I have recognized it to
fail during *repeated* MPI_Intercomm_merge()'s. This can cause a couple of
tests in the lamtests/dynamic to hang and we are fully aware of that. It
should be fixed sometime soon.
Let me know if you have any questions!
-Vishal
On Mon, 7 Jun 2004 Kian_Chang_Low_at_[hidden] wrote:
# Hi,
#
# I'm thinking of testing LAM-MPI over infiniband hardware and wondering
# whether is LAM-MPI infiniband-ready? If yes, is there any documentation on
# getting LAM-MPI to communicate over the infiniband network?
#
# Thanks,
# Kian Chang.
|
