StackOverflow exception when initializing array

Question

I call the method and get StackOverflowException. It is not recursive call and it just contains array initialization. I need an array of BigIntegers, the code works fine with int array even of much bigger size. I show simplified example and in real code I can't use loop to fill the array as I can't generate numbers I need, so I have to hard code them all.

Setup: x64 mode, .Net Core

From error details we can see that:

1) Stack trace is null

2) Error presumably originated in System.Collections.ListDictionaryInternal

    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Before"); // <--- This is displayed

            var a = GetBigIntegers(); // <--- Method is called

            Console.WriteLine("After"); // <--- We will never get there
        }


        static BigInteger[] GetBigIntegers()
        {
            // <--- Crash here
            return new BigInteger[]
            {
                1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
                // Many more lines (850-900) and they are 2-3 times longer than here
                1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
            };
        }
    }

I've checked IL code, it looks correct and it takes close to 400 000 lines.

.method private hidebysig static 
    valuetype [System.Runtime.Numerics]System.Numerics.BigInteger[] GetBigIntegers () cil managed 
{
    // Method begins at RVA 0x207c
    // Code size 1130123 (0x113e8b)
    .maxstack 4
    .locals init (
        [0] valuetype [System.Runtime.Numerics]System.Numerics.BigInteger[]
    )

    // (no C# code)
    IL_0000: nop
    IL_0001: ldc.i4 66500
    IL_0006: newarr [System.Runtime.Numerics]System.Numerics.BigInteger
    IL_000b: dup
    IL_000c: ldc.i4.0
    //  return new BigInteger[66500]IL_000d: ldc.i4.1
    IL_000e: call valuetype [System.Runtime.Numerics]System.Numerics.BigInteger [System.Runtime.Numerics]System.Numerics.BigInteger::op_Implicit(int32)
    // (no C# code)
    IL_0013: stelem [System.Runtime.Numerics]System.Numerics.BigInteger
    IL_0018: dup
    IL_0019: ldc.i4.1
    IL_001a: ldc.i4.1
    IL_001b: call valuetype [System.Runtime.Numerics]System.Numerics.BigInteger [System.Runtime.Numerics]System.Numerics.BigInteger::op_Implicit(int32)
    IL_0020: stelem [System.Runtime.Numerics]System.Numerics.BigInteger
.....
    IL_113e75: dup
    IL_113e76: ldc.i4 66499
    IL_113e7b: ldc.i4.1
    IL_113e7c: call valuetype [System.Runtime.Numerics]System.Numerics.BigInteger [System.Runtime.Numerics]System.Numerics.BigInteger::op_Implicit(int32)
    IL_113e81: stelem [System.Runtime.Numerics]System.Numerics.BigInteger
    IL_113e86: stloc.0
    IL_113e87: br.s IL_113e89

    IL_113e89: ldloc.0
    IL_113e8a: ret
} // end of method Program::GetBigIntegers

I expected that array will be initialized and returned, but actually I got StackOverflow error.

I know that I can use different approaches to do the same stuff, but I want to know WHY it doesn't work this way. Hope it is interesting to everyone reading this question too.

What happens when you remove a couple of these lines? remove like 90% of those 900 lines and see if the error still happens. — LoukMouk
– LoukMouk, Commented Jun 25, 2019 at 12:12

Artyom Ignatovich · Accepted Answer · 2019-06-26 00:59:34Z

3

The actual reason is that evaluation stack frame size is not big enough to fit in everything pushed into.

The reason for that is hiding behind JIT-compiler optimizations which are not performed for struct initialization inside big methods (which leads to poor-perfomance machine code being generated).

Source.

answered Jun 26, 2019 at 0:59

Artyom Ignatovich

6011 gold badge4 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Andrii Siriak Over a year ago

I've read your reference and it looks like my issue. But I don't understand where JIT compiler puts this objects onto stack. From IL I see that there is not more than 4 items on the stack at any time. Reference to array (1), reference to array (2), index (3), value to be stored (4). Could you please explain why I don't see allocations that cause overflow?

Andy Ayers Over a year ago

Right, this is a limitation on how the jit handles structures returned from calls. Large structures are returned implicity by reference. So internally when the jit sees a method that returns a large structure it allocates an “jit temporary” to handle the return value. If you have a large initializer list like this, then each initializer turns into a call and each call gets it s own jit temporary struct. The jit is not clever about re-using this space so each temporary struct gets its own slot on the stack. And the jit may also make additional copies to other temporaries.

Andy Ayers Over a year ago

The method needs to allocate space for all these temporaries in the prolog. So at the start of the method there will be a very large adjustment to the stack pointer, and this adjustment triggers a stack overflow.

Andy Ayers Over a year ago

Detecting when temporaries can be reused to avoid blowing up the stack like this is not as simple as one would hope. But it is something we should fix. As github.com/dotnet/coreclr/issues/14103 says, the workaround is to declare the initializer as an array of int[] say, and then loop over that filling in your array of BigInteger.

Collectives™ on Stack Overflow

StackOverflow exception when initializing array

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related