LAM/MPI logo

LAM/MPI General User's Mailing List Archives

  |   Home   |   Download   |   Documentation   |   FAQ   |   all just in this list

From: damien_at_[hidden]
Date: 2006-01-17 17:44:08


That looks like a mismatch between the underscoring on the BLAS and
SuperLU, The linking on SuperLU is looking for a BLAS built with a single
underscore convention. Can you post your make.inc for SuperLU and the one
for your BLAS?

Damien

> I found out what the problem was with it not creating the library. I
> needed
> to set my make.inc like this:
>
> ARCH = ar
> ARCHFLAGS = cr
>
> Now I am getting the following error:
> Any suggestions would be appreciated.
>
>
> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0
> [root_at_fc4vm1 SuperLU_DIST_2.0]# make
> ( cd INSTALL; make )
> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/INSTALL'
> make[1]: Nothing to be done for `all'.
> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/INSTALL'
> ( cd SRC; make )
> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/SRC'
> ar cr /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a \
> dlangs.o dgsequ.o dlaqgs.o dutil.o dmemory.o dmyblas2.o dsp_blas2.o
> dsp_blas3.o dldperm.o ddistribute.o pdgstrf_irecv.o pdgstrs_Bglobal.o
> pdgstrs1.o pdgssvx_ABglobal.o pdgsrfs_ABXglobal.o pdgsmv_AXglobal.o
> pdgssvx.o pdgstrs.o pddistribute.o pdlangs.o pdutil.o pdgsequ.o pdlaqgs.o
> pdgsrfs.o pdgsmv.o pdgstrs_lsum.o symbfact.o sp_ienv.o etree.o
> sp_colorder.o
> get_perm_c.o mmd.o comm.o memory.o util.o superlu_grid.o pxerbla.o
> superlu_timer.o GetDiagU.o mc64ad.o mc21.o lsame.o xerbla.o slamch.o
> dlamch.o
> echo /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a
> ar cr /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a \
> dcomplex.o zlangs.o zgsequ.o zlaqgs.o zutil.o zmemory.o zmyblas2.o
> dmemory.o zsp_blas2.o zsp_blas3.o zldperm.o zdistribute.o
> pzgssvx_ABglobal.o
> pzgstrf_irecv.o pzgstrs1.o pzgstrs_Bglobal.o pzgsrfs_ABXglobal.o
> pzgsmv_AXglobal.o pzgssvx.o pzgstrs.o pzdistribute.o pzlangs.o pzutil.o
> pzgsequ.o pzlaqgs.o pzgsrfs.o pzgsmv.o pzgstrs_lsum.o symbfact.o sp_ienv.o
> etree.o sp_colorder.o get_perm_c.o mmd.o comm.o memory.o util.o
> superlu_grid.o pxerbla.o superlu_timer.o GetDiagU.o mc64ad.o mc21.o
> lsame.o
> xerbla.o slamch.o dlamch.o
> echo /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a
> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/SRC'
> ( cd EXAMPLE; make )
> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/EXAMPLE'
> mpif77 pddrive.o dcreate_matrix.o sp_ienv.o dreadhb.o
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a -L/usr/lib/libblas.so.3 -lm
> -o
> pddrive
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs.o)(.text+0xf3f): In
> function `pdgstrs':
> pdgstrs.c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs.o)(.text+0x123f):pdgstrs
> .c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs.o)(.text+0x190e):pdgstrs
> .c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs.o)(.text+0x1ba9):pdgstrs
> .c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs_lsum.o)(.text+0xee):
> In
> function `dlsum_fmod':
> pdgstrs_lsum.c: undefined reference to `dgemm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs_lsum.o)(.text+0x384):pdg
> strs_lsum.c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrs_lsum.o)(.text+0x7e8):
> In
> function `dlsum_bmod':
> pdgstrs_lsum.c: undefined reference to `dtrsm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(dsp_blas2.o)(.text+0x4e9): In
> function `sp_dtrsv_dist':
> dsp_blas2.c: undefined reference to `dtrsv_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(dsp_blas2.o)(.text+0xa46):dsp_bl
> as2.c: undefined reference to `dtrsv_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrf_irecv.o)(.text+0x16b):
> In function `pdgstrf2':
> pdgstrf_irecv.c: undefined reference to `dger_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrf_irecv.o)(.text+0x36f):pd
> gstrf_irecv.c: undefined reference to `dger_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrf_irecv.o)(.text+0xdb4):
> In function `pdgstrf':
> pdgstrf_irecv.c: undefined reference to `dgemm_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrf_irecv.o)(.text+0x1338):p
> dgstrf_irecv.c: undefined reference to `dtrsv_'
> /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a(pdgstrf_irecv.o)(.text+0x1dd1):p
> dgstrf_irecv.c: undefined reference to `dgemm_'
> collect2: ld returned 1 exit status
> mpif77: No such file or directory
> make[1]: *** [pddrive] Error 1
> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/EXAMPLE'
> make: *** [example] Error 2
> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0
> [root_at_fc4vm1 SuperLU_DIST_2.0]#
>
>
> Sam Adams
> General Dynamics - Network Systems
> Phone: 210.536.5945
>
> -----Original Message-----
> From: lam-bounces_at_[hidden] [mailto:lam-bounces_at_[hidden]] On Behalf
> Of
> damien_at_[hidden]
> Sent: Friday, January 13, 2006 2:40 PM
> To: General LAM/MPI mailing list
> Subject: Re: LAM: installing superLU-dist 2.0
>
> Your DSUPERLULIB is defined to be in the top directory for SuperLU, but
> the linker couldn't find the library in there. Were there any compile
> errors? If not, look around inside the SuperLU directory's subdirectories
> and see if it ended up in one of those. I don't have my SuperLU backups
> here in front of me, but I'll dig them up tonight and see what I did. I
> vaguely remember the library directories being wonky on mine too.
>
> Damien
>
>> Ok, I did some changes like you recommended and this is what I get now.
>>
>> Script started on Sat 07 Jan 2006 03:33:10 AM CST
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> make
>> ( cd INSTALL; make )
>> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/INSTALL'
>> mpicc -c dlamch.c
>> mpicc -pipe -O2 -c lsame.c
>> mpicc -pipe -O2 -c dlamchtst.c
>> mpif77 -o testdlamch dlamch.o lsame.o dlamchtst.o
>> mpicc -c slamch.c
>> mpicc -pipe -O2 -c slamchtst.c
>> mpif77 -o testslamch slamch.o lsame.o slamchtst.o
>> mpicc -c superlu_timer.c
>> mpicc -pipe -O2 -c timertst.c
>> mpif77 -o testtimer superlu_timer.o timertst.o \
>>
>> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/INSTALL'
>> ( cd SRC; make )
>> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/SRC'
>> mpicc -pipe -O2 -DAdd__ -c dlangs.c
>> mpicc -pipe -O2 -DAdd__ -c dgsequ.c
>> mpicc -pipe -O2 -DAdd__ -c dlaqgs.c
>> mpicc -pipe -O2 -DAdd__ -c dutil.c
>> mpicc -pipe -O2 -DAdd__ -c dmemory.c
>> mpicc -pipe -O2 -DAdd__ -c dmyblas2.c
>> mpicc -pipe -O2 -DAdd__ -c dsp_blas2.c
>> mpicc -pipe -O2 -DAdd__ -c dsp_blas3.c
>> mpicc -pipe -O2 -DAdd__ -c dldperm.c
>> mpicc -pipe -O2 -DAdd__ -c ddistribute.c
>> mpicc -pipe -O2 -DAdd__ -c pdgstrf_irecv.c
>> mpicc -pipe -O2 -DAdd__ -c pdgstrs_Bglobal.c
>> mpicc -pipe -O2 -DAdd__ -c pdgstrs1.c
>> mpicc -pipe -O2 -DAdd__ -c pdgssvx_ABglobal.c
>> mpicc -pipe -O2 -DAdd__ -c pdgsrfs_ABXglobal.c
>> mpicc -pipe -O2 -DAdd__ -c pdgsmv_AXglobal.c
>> mpicc -pipe -O2 -DAdd__ -c pdgssvx.c
>> mpicc -pipe -O2 -DAdd__ -c pdgstrs.c
>> mpicc -pipe -O2 -DAdd__ -c pddistribute.c
>> mpicc -pipe -O2 -DAdd__ -c pdlangs.c
>> mpicc -pipe -O2 -DAdd__ -c pdutil.c
>> mpicc -pipe -O2 -DAdd__ -c pdgsequ.c
>> mpicc -pipe -O2 -DAdd__ -c pdlaqgs.c
>> mpicc -pipe -O2 -DAdd__ -c pdgsrfs.c
>> mpicc -pipe -O2 -DAdd__ -c pdgsmv.c
>> mpicc -pipe -O2 -DAdd__ -c pdgstrs_lsum.c
>> mpicc -pipe -O2 -DAdd__ -c symbfact.c
>> mpicc -pipe -O2 -DAdd__ -c sp_ienv.c
>> mpicc -pipe -O2 -DAdd__ -c etree.c
>> mpicc -pipe -O2 -DAdd__ -c sp_colorder.c
>> mpicc -pipe -O2 -DAdd__ -c get_perm_c.c
>> mpicc -pipe -O2 -DAdd__ -c mmd.c
>> mpicc -pipe -O2 -DAdd__ -c comm.c
>> mpicc -pipe -O2 -DAdd__ -c memory.c
>> mpicc -pipe -O2 -DAdd__ -c util.c
>> mpicc -pipe -O2 -DAdd__ -c superlu_grid.c
>> mpicc -pipe -O2 -DAdd__ -c pxerbla.c
>> mpicc -pipe -O2 -DAdd__ -c superlu_timer.c
>> mpicc -pipe -O2 -DAdd__ -c GetDiagU.c
>> mpif77 -c mc64ad.f
>> mpif77 -c mc21.f
>> mpicc -pipe -O2 -DAdd__ -c lsame.c
>> mpicc -pipe -O2 -DAdd__ -c xerbla.c
>> xerbla.c: In function ‘xerbla_’:
>> xerbla.c:35: warning: incompatible implicit declaration of built-in
>> function
>> ‘printf’
>> mpicc -DAdd__ -c slamch.c
>> mpicc -DAdd__ -c dlamch.c
>> mpicc -march=i686 /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a \
>> dlangs.o dgsequ.o dlaqgs.o dutil.o dmemory.o dmyblas2.o dsp_blas2.o
>> dsp_blas3.o dldperm.o ddistribute.o pdgstrf_irecv.o pdgstrs_Bglobal.o
>> pdgstrs1.o pdgssvx_ABglobal.o pdgsrfs_ABXglobal.o pdgsmv_AXglobal.o
>> pdgssvx.o pdgstrs.o pddistribute.o pdlangs.o pdutil.o pdgsequ.o
>> pdlaqgs.o
>> pdgsrfs.o pdgsmv.o pdgstrs_lsum.o symbfact.o sp_ienv.o etree.o
>> sp_colorder.o
>> get_perm_c.o mmd.o comm.o memory.o util.o superlu_grid.o pxerbla.o
>> superlu_timer.o GetDiagU.o mc64ad.o mc21.o lsame.o xerbla.o slamch.o
>> dlamch.o
>> i386-redhat-linux-gcc: /opt/SuperLU_DIST_2.0/libsuperlu_dist_2.0.a: No
>> such
>> file or directory
>> mpicc: No such file or directory
>> make[1]: *** [double] Error 1
>> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/SRC'
>> make: *** [superlulib] Error 2
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]# cat
>> make.inc
>>
> ############################################################################
>> #
>> # Program: SuperLU_DIST
>> #
>> # Module: make.inc
>> #
>> # Purpose: Top-level Definitions
>> #
>> # Creation date: February 4, 1999 version alpha
>> #
>> # Modified: September 1, 1999 version 1.0
>> # March 15, 2003 version 2.0
>> #
>>
> ############################################################################
>> #
>> # The machine (platform) identifier to append to the library names
>> #
>> PLAT = _i686
>>
>> #
>> # The name of the libraries to be created/linked to
>> #
>> DSuperLUroot = /opt/SuperLU_DIST_2.0
>> DSUPERLULIB = $(DSuperLUroot)/libsuperlu_dist_2.0.a
>> #
>> BLASDEF =
>> BLASLIB = -lblas -lm
>> MPILIB =
>>
>> #PERFLIB = -L/vol1/VAMPIR/lib -lVT
>> LIBS = $(DSUPERLULIB) $(BLASLIB) $(PERFLIB) $(MPILIB)
>>
>> #
>> # The archiver and the flag(s) to use when building archive (library)
>> # If your system has no ranlib, set RANLIB = echo.
>> #
>> ARCH = mpicc -march=i686
>> ARCHFLAGS =
>> RANLIB = ranlib
>>
>>
> ############################################################################
>> # C compiler setup
>> CC = mpicc
>> # CFLAGS should be set to be the C flags that include optimization
>> CFLAGS = -pipe -O2
>> #
>> # NOOPTS should be set to be the C flags that turn off any optimization
>> NOOPTS =
>>
> ############################################################################
>> # FORTRAN compiler setup
>> FORTRAN = mpif77
>> FFLAGS =
>>
> ############################################################################
>> LOADER = mpif77
>>
>> # 32-bit:
>> LOADOPTS =
>>
> ############################################################################
>> # C preprocessor defs for compilation (-DNoChange, -DAdd_, or -DUpCase)
>> #
>> # Need follow the convention of how C calls a Fortran routine.
>> #
>> CDEFS = -DAdd__
>> ]0;sam_at_fc4vm1:/opt/SuperLU_DIST_2.0 [root_at_fc4vm1 SuperLU_DIST_2.0]#
>> exit
>>
>> Script done on Sat 07 Jan 2006 03:33:54 AM CST
>>
>> Sam Adams
>> General Dynamics - Network Systems
>> Phone: 210.536.5945
>>
>> -----Original Message-----
>> From: lam-bounces_at_[hidden] [mailto:lam-bounces_at_[hidden]] On Behalf
>> Of
>> damien_at_[hidden]
>> Sent: Thursday, January 12, 2006 3:08 PM
>> To: General LAM/MPI mailing list
>> Subject: Re: LAM: installing superLU-dist 2.0
>>
>> Sam,
>>
>> the ufffff looks like corruption, unless there's some unusual temporary
>> directories somewhere. I'd re-extract everything and try again. It's
>> been a while since I built and used SuperLU-Dist, but if you're building
>> with gfortran in gcc, the CDEFS should be -DAdd__ . GCC-based fortran
>> adds two underscores.
>>
>> Having said that, can I make a couple of non-LAM suggestions? Use the
>> MUMPS solver, http://graal.ens-lyon.fr/MUMPS/ instead. It's a minimum
>> of
>> 2x faster than SuperLU-Dist on every problem I've tried, and usually 3
>> or
>> 4x, and it's better at badly-scaled matrices. If you have trouble
>> getting
>> the code, email me off-list. MUMPS is also faster than the threaded
>> SuperLU on a multiprocessor box. Also, the g95 fortran compiler (
>> g95.sourceforge.net ) is in better shape than gfortran. Mail me
>> off-list
>> if you have any problems getting it going, I run MUMPS daily on a LAM
>> cluster and it's rock-solid.
>>
>> Damien
>>
>>> I was wondering if any of you guy have had any luck installing
>>> SuperLD-Dist
>>> using Lam-mpi 7.1.1, and gcc 4. I was just basically trying to be
>>> generic
>>> using FC4 with the stock lam, and gcc. I guess the trick it just to
>>> get
>>> the
>>> make.inc file set up properly for lam.
>>>
>>> This is what I have in mine.
>>>
>>
> ############################################################################
>>> #
>>> # Program: SuperLU_DIST
>>> #
>>> # Module: make.inc
>>> #
>>> # Purpose: Top-level Definitions
>>> #
>>> # Creation date: February 4, 1999 version alpha
>>> #
>>> # Modified: September 1, 1999 version 1.0
>>> # March 15, 2003 version 2.0
>>> #
>>>
>>
> ############################################################################
>>> #
>>> # The machine (platform) identifier to append to the library names
>>> #
>>> PLAT = _i386
>>>
>>> #
>>> # The name of the libraries to be created/linked to
>>> #
>>> DSuperLUroot = /opt/SuperLU_DIST_2.0
>>> DSUPERLULIB = $(DSuperLUroot)/libsuperlu_dist_2.0.a
>>> #
>>> BLASDEF = #-DUSE_VENDOR_BLAS
>>> BLASLIB = -L/usr/lib/ -lblas -lm
>>> MPILIB = /usr/lib/libmpi.a /usr/lib/liblamf77mpi.a
>>> /usr/lib/liblammpi++.a /usr/lib/liblammpio.a
>>>
>>> #-L/usr/lib/ -llammpio #-L/usr/local/lib/libmpich.a
>>> -L/usr/local/lib/libpmpich.a -L/usr/local/lib/libpmpich++.a
>>> #/usr/lpp/ppe.poe/lib -lmpi_r
>>> #PERFLIB = -L/vol1/VAMPIR/lib -lVT
>>> LIBS = $(DSUPERLULIB) $(BLASLIB) $(PERFLIB) $(MPILIB)
>>>
>>> #
>>> # The archiver and the flag(s) to use when building archive (library)
>>> # If your system has no ranlib, set RANLIB = echo.
>>> #
>>> ARCH = mpic++ -march=i386 #ar
>>> # 64-bit:
>>> #ARCHFLAGS = -X64 cr
>>> ARCHFLAGS = # cr
>>> RANLIB = ranlib
>>>
>>>
>>
> ############################################################################
>>> # C compiler setup
>>> # 64-bit
>>> #CC = mpcc_r
>>> CC = mpic++
>>> # CFLAGS should be set to be the C flags that include optimization
>>> CFLAGS = -pipe -O2 #-D_SP -qarch=PWR3 -qalias=allptrs \
>>> #-DDEBUGlevel=0 -DPRNTlevel=1 -O3
>>> #-q64 #-qwarn64 -D_LONGINT
>>> #
>>> # NOOPTS should be set to be the C flags that turn off any optimization
>>> # 64-bit
>>> #NOOPTS = -q64
>>> NOOPTS =
>>>
>>
> ############################################################################
>>> # FORTRAN compiler setup
>>> # 64-bit
>>> #FORTRAN = mpxlf90_r
>>> FORTRAN = mpif77 #mpxlf90
>>> FFLAGS = #-WF,-Dsp -O3 -Q -qstrict -qfixed -qinit=f90ptr
>>> -qarch=pwr3
>>> #-q64 #-qintsize=8
>>>
>>
> ############################################################################
>>> # 64-bit
>>> #LOADER = mpxlf90_r
>>> LOADER = mpic++
>>>
>>> # 32-bit:
>>> LOADOPTS = #-bmaxdata:0x70000000
>>> # 64-bit:
>>> #LOADOPTS = -q64
>>>
>>
> ############################################################################
>>> # C preprocessor defs for compilation (-DNoChange, -DAdd_, or
>>> -DUpCase)
>>> #
>>> # Need follow the convention of how C calls a Fortran routine.
>>> #
>>> CDEFS = -DAdd_ -L/usr/lib/libmpi.a -L/usr/lib/liblamf77mpi.a
>>> -L/usr/lib/liblammpi++.a -L/usr/lib/liblammpio.a #mpxlf90-DNoChange
>>> [root_at_fc4vm1 SuperLU_DIST_2.0]#
>>> [root_at_fc4vm1 SuperLU_DIST_2.0]#
>>> [root_at_fc4vm1 SuperLU_DIST_2.0]#
>>>
>>> When I do the make this is the message that I get.
>>>
>>> [root_at_fc4vm1 SuperLU_DIST_2.0]# make
>>> ( cd INSTALL; make )
>>> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/INSTALL'
>>> make[1]: Nothing to be done for `all'.
>>> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/INSTALL'
>>> ( cd SRC; make )
>>> make[1]: Entering directory `/opt/SuperLU_DIST_2.0/SRC'
>>> mpic++ -pipe -O2 -DAdd_ -L/usr/lib/libmpi.a -L/usr/lib/liblamf77mpi.a
>>> -L/usr/lib/liblammpi++.a -L/usr/lib/liblammpio.a -c dldperm.c
>>> dldperm.c: In function \uffff\uffff\uffffvoid dldperm(int_t, int_t,
>>> int_t,
>>> int_t*, int_t*, double*, int_t*, double*, double*)\uffff\uffff\uffff:
>>> dldperm.c:122: error: \uffff\uffff\uffffmc64id_\uffff\uffff\uffff was
>>> not
>>> declared in this scope
>>> mpic++: No such file or directory
>>> make[1]: *** [dldperm.o] Error 1
>>> make[1]: Leaving directory `/opt/SuperLU_DIST_2.0/SRC'
>>> make: *** [superlulib] Error 2
>>>
>>>
>>> Anyway, if anyone has experince getting this compiled with lam, I would
>>> appriciate the help.
>>>
>>> Sam
>>> _______________________________________________
>>> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>>>
>>
>>
>> _______________________________________________
>> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>>
>> _______________________________________________
>> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
>
>
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/