On Wed, 2 Jun 2004, mohammed mubeen jukaku wrote:
> I havnt used the -s option for mpirun, the executable is residing on all
> m/cs. Basically its a master slave approach wherin the master sends
> chunks of data to all slaves for processing and wait for the slaves to
> return the partial output. This process continues untill there is no
> more data to send.
Just out of curiousity:
1. Is this an encoder derived from
http://www.osl.iu.edu/~jsquyres/bladeenc/ ? I did that as a toy
and educational tool several years ago, and stopped development a
long time ago. Indeed, BladeEnc has been abondoned in favor of the
Ogg/Vorbis project. If so, note that this encode works nicely in
parallel, but heed all the warnings on that web page -- due to the
differential aspects of MP3 encoding, you will *not* get the same
result encoding in parallel as you will in serial. That being
said, I still have lots of MP3s that were encoded with the parallel
bladeenc, and I listen to them on a regular basis (i.e., I can't
tell the difference by listening to them, but I'm not a huge
audiophile).
As a further sidenote to this digression, the Ogg Vorbis encoder is
*not* parallelizable (or at least it wasn't the last time I
checked, which I think was sometime within the last year) for deep,
complex reasons.
2. The process you describe is essentially what my parallelized
BladeEnc did. It did not suffer from any problems based on
location of the exexecutable. I suggest using the timer / binary
search method to discover where the bottleneck is occuring. LAM
should not care about filesystem location with respect to the
overall speed of your application; it sounds like there is some
kind of outside influence that is slowing things down.
> On Wed, 02 Jun 2004 Jeff Squyres wrote :
>> On Tue, 1 Jun 2004, mohammed mubeen jukaku wrote:
>>
>>> I have developed a parallel MP3 encoder using BladeEncoder as the base. It runs quite fast on NFS (I have a single home dir). But its damm slow when not on NFS (for eg. when i run from /tmp, which is availble on all m/cs) - ofcourse slower than the serial encoder. Can any1 help me?
>>
>> While your results may seem counter-intuitive, there are probably a lot of factors involved here. One large one to investigate is how you're sending the data out. Another is how you're sending the executable out (are you using the -s option to mpirun, or is the executable and/or data already staged on each node).
>>
>> You might want to throw some crude timers around portions of your program and see exactly where the delay is occuring -- sometimes such timers and a binary search through your code (narrowing the timer range on smaller and smaller parts of your code to find out where the major time sink is) can be quite enlightening. :-)
>>
>> -- {+} Jeff Squyres
>> {+} jsquyres_at_[hidden]
>> {+} http://www.lam-mpi.org/
>
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
