We have a cluster of Intel Xeon processors running RedHat 9. We decided
to install LAM/MPI 7.0, with icc 7.1 as our compiler and Sun Grid Engine
5.3p3 as a batch scheduler. We chose icc over g++ based on a 50% speed
up for our single processor code.
In hopes of helping others who might find themselves in a similar
situation, we thought we would post the required steps:
Compiling and installing LAM/MPI with icc
I. Downloaded source code from:
http://www.lam-mpi.org/7.0/download.php
II. Used a script (bash) and instructions for setting icc environment
variables written by Michael Sabielny (from LAM General User's Mailing
List):
http://216.239.33.104/search?q=cache:Q0xfiyXzFJkJ:www.thinkingnerds.com/
lam/MailArchives/lam/msg06110.php+making+lam+with+the+intel+sabielny
<http://216.239.33.104/search?q=cache:Q0xfiyXzFJkJ:www.thinkingnerds.com
/lam/MailArchives/lam/msg06110.php+making+lam+with+the+intel+sabielny&hl
=en&ie=UTF-8> &hl=en&ie=UTF-8
intel.sh
#
export INTELHOME=/opt/intel/compiler70
export PATH=$INTELHOME/ia32/bin:$PATH
export LD_LIBRARY_PATH=$INTELHOME/ia32/lib:$LD_LIBRARY_PATH
export MANPATH=$INTELHOME/man:$MANPATH
#
Added source <path>/intel.sh to /etc/profile
III. Building the code yielded a few errors, which were avoided by doing
the following
(1) Error 'compilation aborted for registry c'
Added:
#define __WORDSIZE 32
At line 30 of the header file:
/opt/intel/compiler70/ia32/substitute_headers/bits/types.h
(2) Error "__SWORD_TYPE undefined"
Added:
# define__SWORD_TYPE int
At line 25 of the file
/usr/include/bits/statfs.h
IV. Exported environment variables for the installation
export CC=icc
export CXX=icc
export CXXFLAGS=-i_dynamic
export CXXLDFLAGS=-i_dynamic
V. Ran the configuration file using
./configure -prefix=/opt/intel/compiler70/lam
VI. Followed instruction available at http://www.lam-mpi.org
<http://www.lam-mpi.org/> to finish installation
Integration of LAM/MPI and Sun Grid Engine
I. Sun Grid Engine uses csh, so .cshrc and .login were edited with
appropriate paths for LAM executables
II. Configured parallel environment for LAM/MPI based on example
configuration file provided in SGE (<sgeroot>/mpi/mpich.template) with
the following settings
pe_name mpi
queue_list all
slots 32
user_lists NONE
xuser_lists NONE
start_proc_args /opt/sge/mpi/startmpi.sh $pe_hostfile
stop_proc_args /opt/sge/mpi/stopmpi.sh
allocation_rule $fill_up
control_slaves FALSE
job_is_first_task TRUE
III. Edited function for creating an LAM/MPI hostfile based on SGE
hostfile in <sgeroot>/mpi/startmpi.sh
PeHostfile2MachineFile()
{
cat $1 | while read line; do
# echo $line
host=`echo $line|cut -f1 -d" "|cut
-f1 -d"."`
nslots=`echo $line|cut -f2 -d" "
slotpref="np="
i=1
# while [ $i -le $nslots ]; do
# add here code to map regular
hostnames into ATM hostnames
echo "$host $slotpref$nslots"
# i=`expr $i + 1`
# done
done
}
IV. Edited sample job submission script <sgeroot>/mpi/mpi_cpi.sh to the
following
#
echo "Got $NSLOTS slots."
/opt/intel/compiler70/lam/bin/mpiexec -machinefile $TMPDIR/machines -n
$NSLOTS <FILE>
#
where <FILE> is an mpiCC compiled executable file
|
