On Mar 9, 2005, at 12:41 PM, Kumar, Ravi Ranjan wrote:
> Thank you for the reply! I'll apply non-blocking send-recv to check if
> problem still exits.
>
> I have another question on sending contiguous data. I want to know
> which index is traversed first among 3 indices (z, y & x) in
> T[Nz][Nx][Ny]. I wrote my code in C++ and MPI. Suppose, I want to send
> Nx*Ny number of data so I am simply using T[2][0][0] (if I want to
> send 3rd plane out of Nz planes of data) and I am receiving data in
> another processor at location (say) T[3][0][0]. I have doubt about x-y
> co-ordinate of data. Will it be received in the same fashion as it was
> sent?? Do I need to check which indices ( x or y ) is traversed first?
Before answering that, you need to check how you allocated this memory.
Generally, C arrays are contiguous starting at the lowest level (e.g.,
foo[a][b][c] will be adjacent to foo[a][b][c+1]). But it depends on
how you allocated it whether the last element of the one row in the 3rd
dimension is adjacent to the first element of the next row in the 3rd
dimension (e.g., whether foo[a][b][max] is adjacent to foo[a][b+1][0]).
Similarly for traversing up to the 2nd dimension.
Did you allocate the data in T with a single malloc, for example
(malloc(sizeof(double) * Nz * Nx * Ny))? And then setup pointers for
T[][]? Or did you malloc each row and/or plane separately?
In your case, it is *probably* best to use a single malloc and get the
pointer arrays to match, because then (viewing the data as T[a][b][c]),
you can send contiguous c rows, and bXc planes (which is what I'm
assuming you want). You can also setup datatypes to do more
interesting behavior if you need it (e.g., aXc planes). But you really
can only do these datatypes if you use the one-big-malloc approach; you
need to be able to guarantee repetitive absolute differences in
addresses (which you cannot if, for example, you malloc each row
separately).
BTW, I saw *probably* because I don't know your exact application and
hardware and whatnot. Your mileage may vary depending on your specific
setup.
Note that I'm not entirely sure how
double ***foo = new[a][b][c]
lays out the memory. I *think* it's just like the one-big-malloc
approach, but I'm not 100% sure of that...
All that being said, as long as you alloc the memory the same way in
both the sender and the receiver, what you send is what you'll receive.
So if you use the one-big-malloc approach on both sides and send b*c
doubles starting at &T[a1][0][0], and receive b*c doubles starting at
&T[a2][0][0], you'll be ok.
Make sense?
--
{+} Jeff Squyres
{+} jsquyres_at_[hidden]
{+} http://www.lam-mpi.org/
|
