Sorry for beating a dead horse, but I've tried everything I can think of.
Tryiing to lamboot between a linux machine:
Linux sophia 2.4.20-20.8smp #1 SMP Mon Aug 18 14:39:22 EDT 2003 i686
i686 i386 GNU/Linux
and
a sun-solaris machine:
Linux sophia 2.4.20-20.8smp #1 SMP Mon Aug 18 14:39:22 EDT 2003 i686
i686 i386 GNU/Linux
They are both running:
LAM 7.1.1/MPI 2 C++/ROMIO
Naturally, I have followed everything in the manual, FAQ, and tried to
scour the list. Essentially, lamboot fails at the command:
ssh -x athena -n 'echo $SHELL'
when trying to lamboot from the linux machine, and vice-versa when
trying to lamboot from the solaris machine. ssh used on the command
line works just fine from either machine. From the linux machine,
here is the result of using ssh on the command line:
-bash-2.05b$ ssh -x athena.musc.edu -n 'echo $SHELL'
/bin/csh
-bash-2.05b$
Here is a further example of ssh on the command line from the linux machine:
-bash-2.05b$ ssh -x athena.musc.edu -n 'tkill -N'
-bash-2.05b$
Is there something else that can go wrong when lamboot tries to use
ssh versus just using ssh on the command line? Thanks for any
insight.
Regards,
Jejo Koola
Here is the output of lamboot -d:
-bash-2.05b$ lamboot -d
n-1<31283> ssi:boot:open: opening
n-1<31283> ssi:boot:open: opening boot module globus
n-1<31283> ssi:boot:open: opened boot module globus
n-1<31283> ssi:boot:open: opening boot module rsh
n-1<31283> ssi:boot:open: opened boot module rsh
n-1<31283> ssi:boot:open: opening boot module slurm
n-1<31283> ssi:boot:open: opened boot module slurm
n-1<31283> ssi:boot:select: initializing boot module globus
n-1<31283> ssi:boot:globus: globus-job-run not found, globus boot will not run
n-1<31283> ssi:boot:select: boot module not available: globus
n-1<31283> ssi:boot:select: initializing boot module rsh
n-1<31283> ssi:boot:rsh: module initializing
n-1<31283> ssi:boot:rsh:agent: ssh -x
n-1<31283> ssi:boot:rsh:username: <same>
n-1<31283> ssi:boot:rsh:verbose: 1000
n-1<31283> ssi:boot:rsh:algorithm: linear
n-1<31283> ssi:boot:rsh:no_n: 0
n-1<31283> ssi:boot:rsh:no_profile: 0
n-1<31283> ssi:boot:rsh:fast: 0
n-1<31283> ssi:boot:rsh:ignore_stderr: 0
n-1<31283> ssi:boot:rsh:priority: 10
n-1<31283> ssi:boot:select: boot module available: rsh, priority: 10
n-1<31283> ssi:boot:select: initializing boot module slurm
n-1<31283> ssi:boot:slurm: not running under SLURM
n-1<31283> ssi:boot:select: boot module not available: slurm
n-1<31283> ssi:boot:select: finalizing boot module globus
n-1<31283> ssi:boot:globus: finalizing
n-1<31283> ssi:boot:select: closing boot module globus
n-1<31283> ssi:boot:select: finalizing boot module slurm
n-1<31283> ssi:boot:slurm: finalizing
n-1<31283> ssi:boot:select: closing boot module slurm
n-1<31283> ssi:boot:select: selected boot module rsh
LAM 7.1.1/MPI 2 C++/ROMIO - Indiana University
n-1<31283> ssi:boot:base: looking for boot schema in following directories:
n-1<31283> ssi:boot:base: <current directory>
n-1<31283> ssi:boot:base: $TROLLIUSHOME/etc
n-1<31283> ssi:boot:base: $LAMHOME/etc
n-1<31283> ssi:boot:base: /home/koola/lammpi-7.1.1/etc
n-1<31283> ssi:boot:base: looking for boot schema file:
n-1<31283> ssi:boot:base: lam-bhost.def
n-1<31283> ssi:boot:base: found boot schema:
/home/koola/lammpi-7.1.1/etc/lam-bhost.def
n-1<31283> ssi:boot:rsh: found the following hosts:
n-1<31283> ssi:boot:rsh: n0 sophia.musc.edu (cpu=1)
n-1<31283> ssi:boot:rsh: n1 athena.musc.edu (cpu=1)
n-1<31283> ssi:boot:rsh: resolved hosts:
n-1<31283> ssi:boot:rsh: n0 sophia.musc.edu --> 128.23.19.53 (origin)
n-1<31283> ssi:boot:rsh: n1 athena.musc.edu --> 128.23.19.23
n-1<31283> ssi:boot:rsh: starting RTE procs
n-1<31283> ssi:boot:base:linear: starting
n-1<31283> ssi:boot:base:server: opening server TCP socket
n-1<31283> ssi:boot:base:server: opened port 33594
n-1<31283> ssi:boot:base:linear: booting n0 (sophia.musc.edu)
n-1<31283> ssi:boot:rsh: starting lamd on (sophia.musc.edu)
n-1<31283> ssi:boot:rsh: starting on n0 (sophia.musc.edu): hboot -t -c
lam-conf.lamd -d -I -H 128.23.19.53 -P 33594 -n 0 -o 0
n-1<31283> ssi:boot:rsh: launching locally
hboot: performing tkill
hboot: tkill -d
tkill: setting prefix to (null)
tkill: setting suffix to (null)
tkill: got killname back: /tmp/lam-koola_at_sophia/lam-killfile
tkill: removing socket file ...
tkill: socket file: /tmp/lam-koola_at_sophia/lam-kernel-socketd
tkill: removing IO daemon socket file ...
tkill: IO daemon socket file: /tmp/lam-koola_at_sophia/lam-io-socket
tkill: f_kill = "/tmp/lam-koola_at_sophia/lam-killfile"
tkill: nothing to kill: "/tmp/lam-koola_at_sophia/lam-killfile"
hboot: booting...
hboot: fork /home/koola/bin/lammpi-7.1.1/bin/lamd
[1] 31286 lamd -H 128.23.19.53 -P 33594 -n 0 -o 0 -d
hboot: attempting to execute
n-1<31283> ssi:boot:rsh: successfully launched on n0 (sophia.musc.edu)
n-1<31283> ssi:boot:base:server: expecting connection from finite list
n-1<31286> ssi:boot:open: opening
n-1<31286> ssi:boot:open: opening boot module globus
n-1<31286> ssi:boot:open: opened boot module globus
n-1<31286> ssi:boot:open: opening boot module rsh
n-1<31286> ssi:boot:open: opened boot module rsh
n-1<31286> ssi:boot:open: opening boot module slurm
n-1<31286> ssi:boot:open: opened boot module slurm
n-1<31286> ssi:boot:select: initializing boot module globus
n-1<31286> ssi:boot:globus: globus-job-run not found, globus boot will not run
n-1<31286> ssi:boot:select: boot module not available: globus
n-1<31286> ssi:boot:select: initializing boot module rsh
n-1<31286> ssi:boot:rsh: module initializing
n-1<31286> ssi:boot:rsh:agent: ssh -x
n-1<31286> ssi:boot:rsh:username: <same>
n-1<31286> ssi:boot:rsh:verbose: 1000
n-1<31286> ssi:boot:rsh:algorithm: linear
n-1<31286> ssi:boot:rsh:no_n: 0
n-1<31286> ssi:boot:rsh:no_profile: 0
n-1<31286> ssi:boot:rsh:fast: 0
n-1<31286> ssi:boot:rsh:ignore_stderr: 0
n-1<31286> ssi:boot:rsh:priority: 10
n-1<31286> ssi:boot:select: boot module available: rsh, priority: 10
n-1<31286> ssi:boot:select: initializing boot module slurm
n-1<31286> ssi:boot:slurm: not running under SLURM
n-1<31286> ssi:boot:select: boot module not available: slurm
n-1<31286> ssi:boot:select: finalizing boot module globus
n-1<31286> ssi:boot:globus: finalizing
n-1<31286> ssi:boot:select: closing boot module globus
n-1<31286> ssi:boot:select: finalizing boot module slurm
n-1<31286> ssi:boot:slurm: finalizing
n-1<31286> ssi:boot:select: closing boot module slurm
n-1<31286> ssi:boot:select: selected boot module rsh
n-1<31286> ssi:boot:send_lamd: getting node ID from command line
n-1<31286> ssi:boot:send_lamd: getting agent haddr from command line
n-1<31286> ssi:boot:send_lamd: getting agent port from command line
n-1<31286> ssi:boot:send_lamd: getting node ID from command line
n-1<31286> ssi:boot:send_lamd: connecting to 128.23.19.53:33594, node id 0
n-1<31286> ssi:boot:send_lamd: sending dli_port 32807
n-1<31283> ssi:boot:base:server: got connection from 128.23.19.53
n-1<31283> ssi:boot:base:server: this connection is expected (n0)
n-1<31283> ssi:boot:base:server: remote lamd is at 128.23.19.53:32807
n-1<31283> ssi:boot:base:linear: booting n1 (athena.musc.edu)
n-1<31283> ssi:boot:rsh: starting lamd on (athena.musc.edu)
n-1<31283> ssi:boot:rsh: starting on n1 (athena.musc.edu): hboot -t -c
lam-conf.lamd -d -s -I "-H 128.23.19.53 -P 33594 -n 1 -o 0"
n-1<31283> ssi:boot:rsh: launching remotely
n-1<31283> ssi:boot:rsh: attempting to execute: ssh -x athena.musc.edu
-n 'echo $SHELL'
-----------------------------------------------------------------------------
LAM failed to execute a process on the remote node "athena.musc.edu".
LAM was not trying to invoke any LAM-specific commands yet -- we were
simply trying to determine what shell was being used on the remote
host.
LAM tried to use the remote agent command "ssh"
to invoke "echo $SHELL" on the remote node.
*** PLEASE READ THIS ENTIRE MESSAGE, FOLLOW ITS SUGGESTIONS, AND
*** CONSULT THE "BOOTING LAM" SECTION OF THE LAM/MPI FAQ
*** (http://www.lam-mpi.org/faq/) BEFORE POSTING TO THE LAM/MPI USER'S
*** MAILING LIST.
This usually indicates an authentication problem with the remote
agent, some other configuration type of error in your .cshrc or
.profile file, or you were unable to executable a command on the
remote node for some other reason. The following is a list of items
that you should check on the remote node:
- You have an account and can login to the remote machine
- Incorrect permissions on your home directory (should
probably be 0755)
- Incorrect permissions on your $HOME/.rhosts file (if you are
using rsh -- they should probably be 0644)
- You have an entry in the remote $HOME/.rhosts file (if you
are using rsh) for the machine and username that you are
running from
- Your .cshrc/.profile must not print anything out to the
standard error
- Your .cshrc/.profile should set a correct TERM type
- Your .cshrc/.profile should set the SHELL environment
variable to your default shell
Try invoking the following command at the unix command line:
ssh -x athena.musc.edu -n 'echo $SHELL'
You will need to configure your local setup such that you will *not*
be prompted for a password to invoke this command on the remote node.
No output should be printed from the remote node before the output of
the command is displayed.
When you can get this command to execute successfully by hand, LAM
will probably be able to function properly.
-----------------------------------------------------------------------------
n-1<31283> ssi:boot:base:linear: Failed to boot n1 (athena.musc.edu)
n-1<31283> ssi:boot:base:server: closing server socket
n-1<31283> ssi:boot:base:linear: aborted!
n-1<31288> ssi:boot:open: opening
n-1<31288> ssi:boot:open: opening boot module globus
n-1<31288> ssi:boot:open: opened boot module globus
n-1<31288> ssi:boot:open: opening boot module rsh
n-1<31288> ssi:boot:open: opened boot module rsh
n-1<31288> ssi:boot:open: opening boot module slurm
n-1<31288> ssi:boot:open: opened boot module slurm
n-1<31288> ssi:boot:select: initializing boot module globus
n-1<31288> ssi:boot:globus: globus-job-run not found, globus boot will not run
n-1<31288> ssi:boot:select: boot module not available: globus
n-1<31288> ssi:boot:select: initializing boot module rsh
n-1<31288> ssi:boot:rsh: module initializing
n-1<31288> ssi:boot:rsh:agent: ssh -x
n-1<31288> ssi:boot:rsh:username: <same>
n-1<31288> ssi:boot:rsh:verbose: 1000
n-1<31288> ssi:boot:rsh:algorithm: linear
n-1<31288> ssi:boot:rsh:no_n: 0
n-1<31288> ssi:boot:rsh:no_profile: 0
n-1<31288> ssi:boot:rsh:fast: 0
n-1<31288> ssi:boot:rsh:ignore_stderr: 0
n-1<31288> ssi:boot:rsh:priority: 10
n-1<31288> ssi:boot:select: boot module available: rsh, priority: 10
n-1<31288> ssi:boot:select: initializing boot module slurm
n-1<31288> ssi:boot:slurm: not running under SLURM
n-1<31288> ssi:boot:select: boot module not available: slurm
n-1<31288> ssi:boot:select: finalizing boot module globus
n-1<31288> ssi:boot:globus: finalizing
n-1<31288> ssi:boot:select: closing boot module globus
n-1<31288> ssi:boot:select: finalizing boot module slurm
n-1<31288> ssi:boot:slurm: finalizing
n-1<31288> ssi:boot:select: closing boot module slurm
n-1<31288> ssi:boot:select: selected boot module rsh
n-1<31288> ssi:boot:base: looking for boot schema in following directories:
n-1<31288> ssi:boot:base: <current directory>
n-1<31288> ssi:boot:base: $TROLLIUSHOME/etc
n-1<31288> ssi:boot:base: $LAMHOME/etc
n-1<31288> ssi:boot:base: /home/koola/lammpi-7.1.1/etc
n-1<31288> ssi:boot:base: looking for boot schema file:
n-1<31288> ssi:boot:base: lam-bhost.def
n-1<31288> ssi:boot:base: found boot schema:
/home/koola/lammpi-7.1.1/etc/lam-bhost.def
n-1<31288> ssi:boot:rsh: found the following hosts:
n-1<31288> ssi:boot:rsh: n0 sophia.musc.edu (cpu=1)
n-1<31288> ssi:boot:rsh: n1 athena.musc.edu (cpu=1)
n-1<31288> ssi:boot:rsh: resolved hosts:
n-1<31288> ssi:boot:rsh: n0 sophia.musc.edu --> 128.23.19.53 (origin)
n-1<31288> ssi:boot:rsh: n1 athena.musc.edu --> 128.23.19.23
n-1<31288> ssi:boot:rsh: starting RTE procs
n-1<31288> ssi:boot:base:linear: starting
n-1<31288> ssi:boot:base:linear: booting n0 (sophia.musc.edu)
n-1<31288> ssi:boot:rsh: starting wipe on (sophia.musc.edu)
n-1<31288> ssi:boot:rsh: starting on n0 (sophia.musc.edu): tkill -d
n-1<31288> ssi:boot:rsh: launching locally
tkill: setting prefix to (null)
tkill: setting suffix to (null)
tkill: got killname back: /tmp/lam-koola_at_sophia/lam-killfile
tkill: removing socket file ...
tkill: socket file: /tmp/lam-koola_at_sophia/lam-kernel-socketd
tkill: removing IO daemon socket file ...
tkill: IO daemon socket file: /tmp/lam-koola_at_sophia/lam-io-socket
tkill: f_kill = "/tmp/lam-koola_at_sophia/lam-killfile"
tkill: killing LAM...
tkill: killing PID (SIGHUP) 31286 ...
tkill: killed
tkill: all finished
n-1<31288> ssi:boot:rsh: successfully launched on n0 (sophia.musc.edu)
n-1<31288> ssi:boot:base:linear: booting n1 (athena.musc.edu)
n-1<31288> ssi:boot:rsh: starting wipe on (athena.musc.edu)
n-1<31288> ssi:boot:rsh: starting on n1 (athena.musc.edu): tkill -d
n-1<31288> ssi:boot:rsh: launching remotely
n-1<31288> ssi:boot:rsh: attempting to execute: ssh -x athena.musc.edu
-n 'echo $SHELL'
-----------------------------------------------------------------------------
LAM failed to execute a process on the remote node "athena.musc.edu".
LAM was not trying to invoke any LAM-specific commands yet -- we were
simply trying to determine what shell was being used on the remote
host.
LAM tried to use the remote agent command "ssh"
to invoke "echo $SHELL" on the remote node.
*** PLEASE READ THIS ENTIRE MESSAGE, FOLLOW ITS SUGGESTIONS, AND
*** CONSULT THE "BOOTING LAM" SECTION OF THE LAM/MPI FAQ
*** (http://www.lam-mpi.org/faq/) BEFORE POSTING TO THE LAM/MPI USER'S
*** MAILING LIST.
This usually indicates an authentication problem with the remote
agent, some other configuration type of error in your .cshrc or
.profile file, or you were unable to executable a command on the
remote node for some other reason. The following is a list of items
that you should check on the remote node:
- You have an account and can login to the remote machine
- Incorrect permissions on your home directory (should
probably be 0755)
- Incorrect permissions on your $HOME/.rhosts file (if you are
using rsh -- they should probably be 0644)
- You have an entry in the remote $HOME/.rhosts file (if you
are using rsh) for the machine and username that you are
running from
- Your .cshrc/.profile must not print anything out to the
standard error
- Your .cshrc/.profile should set a correct TERM type
- Your .cshrc/.profile should set the SHELL environment
variable to your default shell
Try invoking the following command at the unix command line:
ssh -x athena.musc.edu -n 'echo $SHELL'
You will need to configure your local setup such that you will *not*
be prompted for a password to invoke this command on the remote node.
No output should be printed from the remote node before the output of
the command is displayed.
When you can get this command to execute successfully by hand, LAM
will probably be able to function properly.
-----------------------------------------------------------------------------
n-1<31288> ssi:boot:base:linear: Failed to boot n1 (athena.musc.edu)
n-1<31288> ssi:boot:base:linear: aborted!
lamboot did NOT complete successfully
|
