Jagged array OutOfMemoryException

Question

I'm trying to initialize a jagged array of 120 000 x 4600 for a data mining algorithm but I have an OutOfMemoryException exception

double[][] array = new double[120000][];
for (int i = 0; i < array.Length; i++)
{
    array[i] = new double[4600];
}

it throws when i is around 49 000

I know .Net is limited to 2GB per object, but I thought that the array here would be a list of addresses to an array of double. So it wouldn't be a big single object.

I thought that's why Jon Skeet suggests an jagged array for this question OutOfMemoryException on declaration of Large Array

I don't think that I understand his answer.

Is it one big object and if it's not why does it throw an exception.

Thank you

Did you compile as 64 bits? Also make sure that "Prefer 32-Bit" isn't set in the Build options. — Matthew Watson
– Matthew Watson, Commented Nov 4, 2013 at 16:30
4,600 * 49,000 (the iteration at which it throws) is 225,400,000 elements. That multiplied by 8 bytes (the size of a double) is 1,803,200,000 bytes. And you don't know why it throws OutOfMemoryException? — Daniel Daranas
– Daniel Daranas, Commented Nov 4, 2013 at 16:31
To all the people saying he will have a 4GB data structute, that is only true if he declared the array [,], by making a jagged array with [][] it is one array of 120000 references and 120000 arrays that are 4600 * 8 bytes large. Marc, you are correct, the commenters are wrong. — Scott Chamberlain
– Scott Chamberlain, Commented Nov 4, 2013 at 16:36
@Scott The commenters saying that you have one single 4 GB data structure are wrong, yes - only that there are no such commenters. — Daniel Daranas
– Daniel Daranas, Commented Nov 4, 2013 at 16:38
@DanielDaranas The people I was addressing have deleted their comments. — Scott Chamberlain
– Scott Chamberlain, Commented Nov 4, 2013 at 16:41

Tilak · Accepted Answer · 2013-11-04 16:38:16Z

2

If it is 32-bit application, You are rightly getting OutOfMemoryException. For this size requirements you need to target X64.

At i = 49000, Total memory = 49000*4600*8 = 1803200000 bytes = ~1.68GB.

Now For 32-bit applications (targeted X86), Total User Memory avaiable to an application is 2GB (unless the application is Large address aware, .NET application - Large Address Aware and OS is also enabled for this. Ex: (for)Vista. Then there is some CLR overhead, then application overhead.

At i = 120000, You need total memory as Total memory = 120000*4600*8 = 1803200000 bytes = ~4.11GB. (Platform target should be X64)

answered Nov 4, 2013 at 16:38

Tilak

30.8k19 gold badges85 silver badges133 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Brian Rasmussen Over a year ago

Also, the address space is usually pretty fragmented in 32 bit processes which limits the size of the biggest contiguous block.

Collectives™ on Stack Overflow

Jagged array OutOfMemoryException

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related