Hi Jeff:
I am trying to construct a very simple prototype based on your
suggestion #1.
And I have another question now. In a normal MPI run, the LAM daemon
network is started before mpirun, and mpirun will specify how many and
which
PEs to use by using, e.g., mpirun -np 3 a.out or mpirun n0-2 a.out.
In the following quoted design, how does the master spawn a bunch of
slaves on the PEs that I specify? In
MPI_COMM_SPAWN(command, argv, maxprocs, info, root, comm, intercomm,
array_of_errcodes)
there is way to specify the max number of procs, but if the master is not
started by mpirun, wouldn't all maxprocs processes be spawned to the
local PE?
Is there a way to start a LAM daemon network among a list of IPs
using MPI_Init(int *argc, char ***argv) from a sequential program like
a.out?
Hope my question makes sense. BTW, your design seems to be exactly
what I wanted.
Thanks,
-Lei
-------------------- quoted msg ------------------------
Actually, my ordering wasn't exactly right. Try this:
> - your matlab script launches
> - it calls MPI_Init
> - check for a published name
> - if the published name does not exist
> - spawn a master (i.e., a new, independant process)
> - the master spawns a bunch of slaves to do the work
> - the master publishes a name
> - if the published name does exist
> - MPI_Comm_connect to the master
> - the matlab script sends a bunch of work to the master
> - the master farms it out to all the slaves
> - the slaves do all the work and eventually send the result(s) to the
> master
> - the master sends the result(s) to the matlab script
> - the matlab script disconnects from the master
> - the matlab script finishes
--
{+} Jeff Squyres
Jeff Squyres wrote:
>On Jul 13, 2005, at 3:16 AM, Lei_at_ICS wrote:
>
>
>
>>I am given a Matlab+C program for me to parallelize. What I need to do
>>is to parallelize a C routine that is called by another C routine,
>>which in
>>turn is called from Matlab. There are large and complicated C data
>>structures that are passed in and out the C routine being parallelized.
>>
>>My question is: is there a way to spawn MPI processes from a non-MPI
>>process, and at the same time pass in and out C data structures between
>>the parent sequential and the child MPI processes.
>>
>>
>
>So that's 2 questions:
>
>1. can a non-MPI process spawn MPI processes: sort of. However, it's
>probably easiest if your non-MPI process simply calls MPI_INIT and
>becomes an MPI process (i.e., you don't have to launch it via mpirun --
>if it calls MPI_INIT, it will simply get an MPI_COMM_WORLD size of 1,
>and then call MPI_COMM_SPAWN from there to launch more MPI processes).
>
>Other people have done this with Matlab before (indeed, there is a
>package out there with MPI bindings for Matlab MEX), and there are
>definitely some issues that need to be thought out first. Search the
>LAM mailing list archives -- this stuff has been discussed extensively
>before.
>
>2. Can you pass C structures from the parent to the child: if you
>follow the suggestion in #1, there's nothing to pass -- Matlab itself
>becomes the parent MPI process; then you use standard MPI send/receive
>semantics to pass data from the Matlab script (or C routine) to the
>spawned processes. If you use a different mechanism, then you can use
>traditional Unix IPC mechanisms to pass the data (pipes, files, local
>sockets, shared memory, etc.).
>
>Hope that helps.
>
>
>
|
