Hi,
I am new to the LAM-MPI mailing-group and if this is not the appropriate forum
for my query, please let me know.
Because IBM's AIX implementation of MPI does not support process creation and
management, some of our users have asked if it would be possible to install
LAM-MPI on our system.
In the next few months we will be installing an IBM Cluster-1600 system, which
will consist of over 50 SMP nodes, each node being a 32-CPU POWER4 p690 server
running AIX 5.2. The nodes will be interconnected using IBM's latest pSeries
high-performance switch (previously known as "Federation").
I have a few queries, that I hope someone with the relevant knowledge could answer:
1. Do you see any problem in installing LAM-MPI on such a system at that level
of operating system?
2. Are there any observations or general advice that you could make on such a
proposal?
3. Do you see any reason why using LAM-MPI would not be a good thing to do on
our system?
4. IBM's MPI uses LAPI as the underlying protocol and this will be tuned and
developed for Federation. My understanding is that LAM-MPI would have to use
TCP/IP on our system (whereas it does understand how to get the best out of
Myrinet/Infiniband), with a possible degradation in performance caused by this.
From past experience with LAM-MPI on systems with the older "Colony" switch, do
you have any idea of the amount of performance degradation I could expect to see?
5. We use LoadLeveler as our BATCH system. Is LAM-MPI "LoadLeveler-aware" i.e.
can I set up a BATCH job that LoadLeveler will schedule onto a set of nodes
that will then be used by my (LAM-)MPI program?
6. We use AIX's Workload Manager (WLM) on the "batch" nodes to allocate and
schedule consumable resources (CPU and physical memory). Would this interfere
with the way LAM-MPI works?
7. We only allow a very small INTERACTIVE service that is restricted to using
just a few CPUs in one node. "telnet", "rsh", "ssh" and so on are prevented from
running on the other nodes as these are reserved for BATCH jobs. Since it isn't
possible to "rsh" onto the BATCH nodes, would this be a problem for LAM-MPI?
8. Some of our programs use a hybrid parallel programming padigm of MPI across
nodes, with OpenMP within the nodes. In fact it is more general than this,
allowing any number of MPI tasks within a node with these having any number of
OpenMP threads (normally we use "power-of-2" numbers - 1 MPI x 32 OpenMP, 2 x
16, 4 x 8, 8 x 4, 16 x 2, 32 x 1, etc...) For this of course we have to use
thread-safe libraries. Can such an approach be utilised with LAM-MPI too?
Thank you for taking the time to read (and hopefully reply) this.
Best wishes
--
+-----------------+---------------------------------+------------------+
| Neil Storer | Head: Systems S/W Section | Operations Dept. |
+-----------------+---------------------------------+------------------+
| ECMWF, | email: neil.storer_at_[hidden] | //=\\ //=\\ |
| Shinfield Park, | Tel: (+44 118) 9499353 | // \\// \\ |
| Reading, | (+44 118) 9499000 x 2353 | ECMWF |
| Berkshire, | Fax: (+44 118) 9869450 | ECMWF |
| RG2 9AX, | | \\ //\\ // |
| UK | URL: http://www.ecmwf.int/ | \\=// \\=// |
+--+--------------+---------------------------------+----------------+-+
| ECMWF is the European Centre for Medium-Range Weather Forecasts |
+-----------------------------------------------------------------+
|
