I don't think that this limitation is related to (LAM) MPI. I am using
LAM (either version 6 and 7) on various Linux PCs and I can create *large*
fortran files (above 100 GB with no trouble). I did not specified any
special flag for building LAM.
I guess your problem is related to kernel level, to file system level
(recent ext2 or ext3 or xfs are fine), to file access (NFS 2 enforces a 2
GB limit on file size, NFS 3 is fine), to libc level, and of course to
compiler versions.
Large files are supported by default with no special options on recent
distribution, such as mandrake 9.0 or 9.1, which include gcc/g77 3.2 or
above. The intel compiler was limited to 4 GB files in version 6, this
limitation has been removed in present version (I am using ifc 7.1).
You may have a look on http://www.suse.de/~aj/linux_lfs.html for details
about "Large File Support in Linux".
Hoping this helps.
Pierre Valiron.
PS. I am not using MPI parallel IO. This could put additional limitations
on file size. However I was able to produce large files over PVFS.
On Thu, 18 Sep 2003, Virgilio, Vincent wrote:
> Perhaps you need to build MPI with (at least) the following compiler flags:
>
> -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
>
> which should enable 'large file support' in the standard C library. (FWIW, I
> don't think the _GNU_ standard C++ library supports large files yet.)
>
> I ran into (seemingly) the same limit in my application, and broke through
> it with these flags.
>
> Caveat emptor (BEWARE): I am not using MPI facilities for I/O, so didn't
> notice the limit there, and therefore didn't compile LAM/ROMIO with these
> flags.
>
> Vince Virgilio
>
> > -----Original Message-----
> > From: Lei_at_UCI [mailto:pan_at_[hidden]]
> > Sent: Thursday, September 18, 2003 1:02 PM
> > To: LAM/MPI mailing list
> > Subject: LAM: Limitation in size of a single file
> >
> >
> > Hi,
> >
> > The Linux cluster I am using seems to have a
> > Limitation of ~2GB on the size one single
> > File can be. Both Fortran file I/O and
> > MPI parallel file I/O failed with broken
> > Pipe on outputting a file of ~2.8GB.
> >
> > Please help! Sorry I am not a system admin
> > Which means if you help me please give me
> > Detailed steps on how to resolve this.
> >
> > Thanks and I appreciate your help!
> >
> > -Lei
--
_/_/_/_/ _/ _/ Dr. Pierre VALIRON
_/ _/ _/ _/ Laboratoire d'Astrophysique (UMR 5571 CNRS)
_/ _/ _/ _/ Observatoire de Grenoble / U. Joseph Fourier
_/_/_/_/ _/ _/ BP 53 F-38041 Grenoble Cedex 9 (France)
_/ _/ _/
_/ _/ _/ http://www-laog.obs.ujf-grenoble.fr
_/ _/ _/ mailto:Pierre.Valiron_at_[hidden]
_/ _/_/ Phone / Fax: +33 (0)4 76.51.47.87 / (0)4 76.44.88.21
|
