I'm not sure about how relevant this is for LAM but I know mpich is not
thread safe and so you can only use MPI calls in one thread. I have
worked on a C++ app with a master/slave architecture where the master
has more than one thread but the slaves only ever have a single thread.
The master can do other work in the second thread but not call
MPI calls... I think MPI2 has more support for threads...
for what it's worth... noel.
Bailey, Richard T (US SSA) wrote:
> I developed a C++ app w/ a master/slave architecture for splitting an
> array in the master and having the slaves each process part of the
> array. The master sets up the slaves, including the communicator, when
> the master object is constructed. The Master->ProcessArray() member splits the array and, depending on input params
> either does or does not create a separate thread to send and receive the
> subarrays from the previously-established slaves. In either case, I use
> nonblocking MPI_Irecv() and MPI_Isend() calls for master/slave
> communication. When I do not use a separate thread, all is well.
> However, if I do use a separate thread (so the client code can do other
> work while the thread spawned by Master->ProcessArray() is running), then MPI_Irecv() blocks, with the effect
> that only 1 slave ever gets work and all processing is sequential. I
> put 1 second sleeps in the slaves and put fprintfs around the code to
> verify the it is the MPI_Irecv() that is causing the problem:
>
>
>
> fprintf(
>
> stderr,
>
> "V: Starting AsyncRecvSubarrayParamsOutFromSlave %d @T %8.3f\n",
>
> TileJob->SlaveStatus->Rank,
>
> GetSecs()
>
> );
>
>
>
> MPI_Irecv(
>
> & TileJob->SubarrayParamsOut,
>
> sizeof(SubarrayParamsOutType) / sizeof(int),
>
> MPI_INT,
>
> TileJob->SlaveStatus->Rank, // Source
>
> SubarrayParamsOutTag,
>
> AllComms,
>
> & TileJob->SlaveStatus->MPIRequest
>
> );
>
>
>
> fprintf(
>
> stderr,
>
> "V: Done w/ AsyncRecvSubarrayParamsOutFromSlave %d @T %8.3f\n",
>
> TileJob->SlaveStatus->Rank,
>
> GetSecs()
>
> );
>
>
>
> Here is the output:
>
> V: Starting AsyncRecvSubarrayParamsOutFromSlave 1 @T 1.755
>
> V: Done w/ AsyncRecvSubarrayParamsOutFromSlave 1 @T 2.763
>
>
>
> The same output from a non-threaded run shows that MPI_Irecv() is
> nonblocking.
>
>
>
> I am running LAM v. 7.0 compiled w/ gcc 2.95.3 on a Sun workstation OS
> 5.7. The thread is created with pthread_create(). I tried initializing
> MPI 3 different ways with the same result:
>
> MPI_Init(NULL, NULL);
>
> MPI_Init_thread(NULL, NULL, MPI_THREAD_SINGLE, &DontCare);
>
> MPI_Init_thread(NULL, NULL, MPI_THREAD_SERIALIZED, &DontCare);
>
>
>
> What do I need to do to get this to work in the multithreaded mode?
>
>
>
>
>
> R. T. Bailey
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> This list is archived at http://www.lam-mpi.org/MailArchives/lam/
|
