Difference between memory usage when using standard arrays and derived types in fortran 90

Question

I observed a weird behavior regarding the memory usage of derived data types. The following fortran90 code demonstrates the issue.

  module prec
  implicit none
  integer, parameter :: d_t = selected_real_kind(15,307)
  end module
  module typdef
  use prec
  implicit none
  type level_2
  real(kind=d_t), allocatable :: lev_3(:)
  end type
  type level_1
  type(level_2),allocatable :: lev_2(:,:)
  end type
  type array
  type(level_1),allocatable :: lev_1(:,:)
  end type
  end module
  program mem_test
  use prec
  use typdef
  implicit none
  integer :: n,i,j,k,l,m,egmax,niter,ncells,namom,nsmom
  real(kind=d_t),allocatable :: simple_array(:,:,:,:,:)
  type(array)                :: fancy_array
  real(kind=d_t)             :: it
  egmax=7
  niter=2
  ncells=3000000
  namom=1
  nsmom=1
  !
  !
  !  
  allocate( simple_array(egmax,niter,ncells,namom,nsmom) )
  !
  !
  !
  allocate( fancy_array%lev_1(egmax,niter))
  do i=1,niter
   do j=1,egmax
     allocate( fancy_array%lev_1(j,i)%lev_2(ncells,namom) )
   end do
 end do
 do i=1,niter
   do j=1,egmax
     do k=1,namom
       do l=1,ncells
         allocate( fancy_array%lev_1(j,i)%lev_2(l,k)%lev_3(nsmom) )
       end do
     end do 
   end do
 end do
 !
 do n=1,100000
 it=0.0_d_T
 do i=1,100000
  it=it+1.0_d_t
 end do
 end do
 ! 
 !
 deallocate(simple_array)
 deallocate(fancy_array%lev_1)
 end program

I want to store data in a multi-dimensional array (egmax*niter*ncell*namom*nsmom double precision numbers). I did that in two different ways:

A multidimensional standard array "simple_array(egmax,niter,...,)"
A nested derived data structure "fancy_array" as defined in the piece of code that I provided.

I compiled the code using

    ifort -g -o  test.exe file.f90

I ran it in valgrind and compared the memory consumption of simple_array and fancy_array. simple_array uses about 300Mb as expected while fancy_array uses 3Gb (10 times as much) even though it stores the same number of real numbers. Therefore, it should also consume only 300Mb.

Running a simpler test case, where the derived type is only one level deep, e.g.

     type level_1
        real(kind=d_t),allocatable :: subarray(:)
     end type
     type array
        type(level_1),allocatable :: lev_1(:)
     end type

consumes exactly the amount of memory I am expecting it to. It does not consume 10x as much memory. Has anyone observed similar behavior or has any clue why this would occur? My only idea for the reason of the described behavior is that fancy_array alocated non-contiguous memory and fortran somehow needs to keep track of it, hence the increase in memory consumption. I would appreciate any input or similar observations.

Thanks for your help.

Sebastian

IanH · Accepted Answer · 2013-01-28 00:11:38Z

6

(Allocatable components are a Fortran 2003 feature.)

The typical means by which Fortran processors (including Intel Fortran) implementation of allocatable array objects is to use a descriptor - a data structure that contains information such as the location of the array data in memory and the bounds and stride of each dimension of the array, amongst other things.

For Intel Fortran on a x64 platform that descriptor takes 72 bytes for a one dimensional allocatable array. In your derived type case you have about 42 million such arrays - one for every lev_3 component that you bring into existence, plus a much smaller number for the parent allocatable components. 72 bytes by 42 million gives about 3 GB. There may be further overhead associated with the underlying memory allocator.

On the same platform the descriptor for a rank five array takes 168 bytes and there is only one memory allocation

The data storage requirements for the two approaches will be about the same.

Note the capability offered by the two approaches is significantly different (hence the difference in overhead) - in the derived type case you can change the allocation status, bounds and extent of each lev_3 component. You do not have anywhere near that flexibility in in the single array case - if allocated that array must be rectangular.

(In Fortran 90 your component's dimensions in their declarations would need to be constant expressions (fixed at compile time). No descriptors would be used and the memory requirements of the two approaches would converge.)

edited Jan 28, 2013 at 0:11

answered Jan 27, 2013 at 20:27

IanH

21.6k2 gold badges41 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user2014624 Over a year ago

Thanks for the answer! I think that settles it. And of course, you are right, allocatable components in derived types are a 2003 feature. Still think of my code as fortran90 somehow even if it uses elements from 2003.

IanH Over a year ago

Well, it would settle it if I could multiply. The number of allocatable objects is/was out by an order of magnitude.

Collectives™ on Stack Overflow

Difference between memory usage when using standard arrays and derived types in fortran 90

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related