On Feb 4, 2006, at 12:23 PM, Ross Boylan wrote:
> On Sat, Feb 04, 2006 at 10:38:23AM -0600, Brian Barrett wrote:
>> On Feb 3, 2006, at 6:47 PM, Ross Boylan wrote:
>>
>>> The man page for mpirun (7.1.1, Sept 2004) has a discussion of I/O
>>> redirection with many references to the "local node" or "the node
>>> that
>>> invoked mpirun."
>>>
>>> Should "the session of rank 0" be read in all those cases?
>>> Potentially
>>> several sessions could be running on the node from which you ran
>>> mpirun,
>>> and having them share stdin sounds impossible and sharing stdout
>>> undesirable.
>>>
>>> Then again, maybe rank 0 doesn't need to be on the same node as the
>>> one
>>> used to execute mpirun, so my alternate formulation seems to have
>>> some
>>> holes too.
>>
>> LAM/MPI uses a neat Unix trick for its standard input redirection,
>> passing the stdin file descriptor from mpirun to rank 0 of the
>> launched MPI application. This only works if mpirun and rank 0 are
>> on the same node (hence the references in the documentation). If
>
> The references in the documentation also seem to cover non-rank 0
> processes running on the node that invoked mpirun. Am I correct that
> meaning is not intended?
I actually mis-spoke on this one. The lowest ranking process (in
MPI_COMM_WORLD) on the node on which mpirun is invoked will get
mpirun's standard input. So the documentation is right and I was
wrong. Sorry about that...
> So stdin redirection only applies to a process which is both rank 0
> and on the node from which mpirun was invoked. And your description
> below implies that in OpenMPI the restriction is relaxed: rank 0
> anywhere gets the special treatment. Right?
Your description of Open MPI's treatment is correct - rank 0 always
gets standard input, regardless of where mpirun and rank 0 end up.
> And all processes everywhere share stdout on the node invoking
> mpirun. Does that apply to spawned processes as well?
Unfortunately we do not handle standard output / standard error
correctly for spawned processes in LAM/MPI. I believe we should
handle it correctly with Open MPI, but that has not been well tested
at this point. If it doesn't work, it's something we need to work on.
>> mpirun and rank 0 are on different nodes, LAM is unable to pass the
>> file descriptor for stdin and no standard input redirection is
>> available.
>>
>> Since there is essentially a one-to-one mapping between MPI
>> applications and invocations of mpirun (with the exception of
>> singleton applications), each rank 0 that meets the requirements
>> above has its mpirun's stdin file descriptor and each therefore has a
>> separate standard input stream.
>>
>> By the way, Open MPI deals with standard input the same way it treats
>> standard output (and indeed, the same way LAM deals with standard
>> output). It doesn't matter on which nodes mpirun and rank 0 reside
>> -- each job will have rank 0 wired up to mpirun's standard input
>> stream.
>>
>> Hope this helps. This is always a bit more confusing than it really
>> should be, so if I didn't answer your question, please let me know.
>
> Thanks for the info.
No problem - this time, I'm even pretty sure I got it right ;).
Brian
--
Brian Barrett
LAM/MPI developer and all around nice guy
Have a LAM/MPI day: http://www.lam-mpi.org/
|
