StringBuilder Class OutOfMemoryException

Question

I have written following function

public void TestSB()
{
  string str = "The quick brown fox jumps over the lazy dog.";
  StringBuilder sb = new StringBuilder();
  int j = 0;
  int len = 0;

  try
  {
     for (int i = 0; i < (10000000 * 2); i++)
     {
        j = i;
        len = sb.Length;
        sb.Append(str);
     }

    Console.WriteLine("Success ::" + sb.Length.ToString());
  }
  catch (Exception ex)
  {
      Console.WriteLine(
          ex.Message + " :: " + j.ToString() + " :: " + len.ToString());
  }
}

Now I suppose, that StringBuilder has the capacity to take over 2 billion character (2,147,483,647 to be precise).

But when I ran the above function it gave System.OutOfMemoryException just on reaching the capacity of about 800 million. Moreover, I am seeing widely different result on different PC having same memory and similar amount of load.

Can anyone please provide or explain me the reason for this?

I would take a look at stackoverflow.com/questions/363680/… and stackoverflow.com/questions/1733667/… — Baz1nga
– Baz1nga, Commented Sep 24, 2011 at 8:07
You will be able to approach the maimum better with StringBuilder sb = new StringBuilder(10000000 * 1); Using a(n initial) capacity is always a good idea with big collections. — Henk Holterman
– Henk Holterman, Commented Sep 24, 2011 at 8:25

Jon Skeet · Accepted Answer · 2011-09-24 08:06:04Z

40

Each character requires 2 bytes (as a char in .NET is a UTF-16 code unit). So by the time you've reached 800 million characters, that's 1.6GB of contiguous memory required¹. Now when the StringBuilder needs to resize itself, it has to create another array of the new size (which I believe tries to double the capacity) - which means trying to allocate a 3.2GB array.

I believe that the CLR (even on 64-bit systems) can't allocate a single object of more than 2GB in size. (That certainly used to be the case.) My guess is that your StringBuilder is trying to double in size, and blowing that limit. You may be able to get a little higher by constructing the StringBuilder with a specific capacity - a capacity of around a billion may be feasible.

In the normal course of things this isn't a problem, of course - even strings requiring hundreds of megs are rare.

¹ I believe the implementation of StringBuilder actually changed in .NET 4 to use fragments in some situations - but I don't know the details. So it may not always need contiguous memory while still in builder form... but it would if you ever called ToString.

answered Sep 24, 2011 at 8:06

Jon Skeet

1.5m893 gold badges9.3k silver badges9.3k bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Tigran Over a year ago

Whell, but why this behaviour can vary between different machines considering that allocation limit is handled by VM and not system itself?

Jon Skeet Over a year ago

@Tigran: It can vary based on two things: the VM implementation (different major versions, different variations based on CPU architecture) and the implementation details of StringBuilder itself. Oh, and how much memory is available of course...

Neil Fenwick Over a year ago

@Tigran I think because the StringBuilder effectively needs contiguous memory to allocate its contents and memory can be fragmented in different ways based on what the machine has been doing beforehand. You can get still possibly get OutOfMemory exceptions when there may be still be lots of physical RAM still free, because there is not enough contiguous memory.

Tigran Over a year ago

@jon agree, but what sounds strange to me that the guy saying to have very different results on machines with apparently same config. But you confirm actually my doubts on machines equality.

Jon Skeet Over a year ago

@atur: Right - that corresponds with my footnote - the implementation of StringBuilder has changed in .NET 4.

|

Collectives™ on Stack Overflow

StringBuilder Class OutOfMemoryException

1 Answer 1

9 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related