LAM/MPI logo

LAM/MPI General User's Mailing List Archives

  |   Home   |   Download   |   Documentation   |   FAQ   |   all just in this list

From: Neil Storer (Neil.Storer_at_[hidden])
Date: 2005-02-23 04:53:48


Hi,

Similar to Brian's suggestion suggestion. You could try either:

  1) Placing your matrix in "Blank common"
     e.g.
               COMMON big_matrix(16400,16400)

     Blank common is not stored in a BSS segment.

  or

  2) Allocating your matrix dynamically (f90)
     e.g.
               REAL, DIMENSION(:), ALLOCATABLE :: big_matrix
               ALLOCATE(big_matrix(160400,16400), STAT=ierr)
               IF (ierr .NE. 0) THEN
                   STOP 'Allocation failed'
               ENDIF

These may work - no guarantees though.

Regards
         Neil
Brian Barrett wrote:

> On Feb 22, 2005, at 4:32 PM, Srinivasa Prade Patri wrote:
>
>> Iam running LU decomposition algorithm on a 23 node cluster.
>> Each node has 2GB of RAM. When iam trying to run the algorithm for
>> single precision complex matrix of size 16400*16400,iam getting the
>> following compilation errors.
>>
>> /tmp/ccmtcg1e.o(.text+0xa1): In function `MAIN__':
>> : relocation truncated to fit: R_X86_64_32S .bss
>
>
> <snip>
>
>> I know for such a big matrix the matrix
>> size is more than 2GB (16400*16400*8 Bytes = 2.15GB). IS this the
>> reason for the above compilation errors?
>>
>> If this is the case,do we have any
>> techniques where i can run the algorithm for matrix sizes more than
>> 16400*16400. Or the cluster iam using is limited for certain problem
>> size (16384*16384).
>
>
> Based on your error messages and some google searches, it looks like
> you're tripping up a limitation in at least some versions of the GCC
> suite on x86_64 machines. They don't seem to do well with large text
> sections (anywhere they have to have hard coded addresses > 2GB in
> size. I was unable to reproduce the problems on our Opteron machines,
> but they are running a very recently installed version of Gentoo, so
> maybe the bug was fixed along the way? Based on that, upgrading might
> help.. The other option is to stick the large matrix in the heap
> instead of the text or data section by making it dynamically
> allocated. That should keep you from angering the linker gods.
>
>
> Hope this helps,
>
> Brian
>

-- 
+-----------------+---------------------------------+------------------+
| Neil Storer     |    Head: Systems S/W Section    | Operations Dept. |
+-----------------+---------------------------------+------------------+
| ECMWF,          | email: neil.storer_at_[hidden]    |    //=\\  //=\\  |
| Shinfield Park, | Tel:   (+44 118) 9499353        |   //   \\//   \\ |
| Reading,        |        (+44 118) 9499000 x 2353 | ECMWF            |
| Berkshire,      | Fax:   (+44 118) 9869450        | ECMWF            |
| RG2 9AX,        |                                 |   \\   //\\   // |
| UK              | URL:   http://www.ecmwf.int/    |    \\=//  \\=//  |
+--+--------------+---------------------------------+----------------+-+
   | ECMWF is the European Centre for Medium-Range Weather Forecasts |
   +-----------------------------------------------------------------+