Json.Net throws OutOfMemoryException only in Visual Studio

Question

I have some strange behavior which I can't figure out.

I'm using a WCF service to save files to a certain DB table. The WCF service has a single method which takes a JSON string as parameter. The JSON in this case is a serialized command which contains a List<FileData> amongst other properties. The WCF service deserializes the JSON and runs the CommandHandler for this specific command.

An end-user ran into a bug when he tried to upload a file with a size of 52 MB. The WCF service returned a 404 error.

I was able to reproduce this in Visual Studio. After changing the config file according to this article, the 404 disappeared.

But now a new exception appeared: While the command is successfully serialized client side, successfully processed by WCF, the deserialization throws an OutOfMemoryException. This is the top of the stacktrace:

at Newtonsoft.Json.JsonTextReader.ReadData(Boolean append, Int32 charsRequired) at Newtonsoft.Json.JsonTextReader.ReadData(Boolean append) at Newtonsoft.Json.JsonTextReader.ReadStringIntoBuffer(Char quote) at Newtonsoft.Json.JsonTextReader.ParseString(Char quote) at Newtonsoft.Json.JsonTextReader.ParseValue() at Newtonsoft.Json.JsonTextReader.ReadInternal() at Newtonsoft.Json.JsonReader.ReadAsBytesInternal() at Newtonsoft.Json.JsonTextReader.ReadAsBytes() at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ReadForType(JsonReader reader, JsonContract contract, Boolean hasConverter)

I wrote a unittest to prove the bug. But against all odds this test passes, in other words, no OutOfMemoryException is thrown.

The test for the sake of completeness:

    [TestMethod]
    public void LoadBigFile_SerializeDeserialize_DoesntThrowOutOfMemoryException()
    {
        // Arrange
        byte[] bytes = new byte[80000000];
        Random r = new Random(23);
        r.NextBytes(bytes);

        var command = new SomeCommand(new List<FileData>
        {
            new FileData(
                fileFullName: @"D:\SomePdfFile.pdf",
                modifyDate: DateTime.MaxValue,
                data: bytes
                )
        });

        var data = JsonConvert.SerializeObject(command);

        // Act
        var deserializedCommand = 
              JsonConvert.DeserializeObject<SomeCommand>(data);

        // Assert
        Assert.AreEqual(bytes.Length, deserializedCommand.Files.First().Data.Length);
    }

So, I took my chances and changed the config file in production and tried to upload the same file. And that just works!!! No OutOfMemoryException!

Now my question is, why does the OutOfMemoryException only happen in Visual Studio, while a unittest in the same instance of VS doesn't? It feels a bit strange that I can't test uploading big files in Visual Studio, while it works in production. Notice that I also tried to run in Debug as in Release Mode.

Some details:

Using Json.Net 7.0.1
Visual Studio 2015, update 2
WCF hosted in IIS Express locally, IIS in production
Windows 10 latest build 64 bit
Production server Windows server 2008 R2 64 bit
.Net Framework 4.5.2

Is your local dev machine 32-bit or 64-bit? Is that different than the deployment server? Is your configuration "Any CPU"? — mellamokb
– mellamokb, Commented Jun 5, 2016 at 13:26

Aleksey L. · Accepted Answer · 2016-06-05 14:54:22Z

6

I've reproduced OutOfMemoryException in unit test by changing byte[] bytes = new byte[80000000]; to byte[] bytes = new byte[52000000]; and running in loop (2 times). Test runner is 32 bit.

So back to IIS Express - I think you're using 32 bit version. You can change this at

Tools | Options | Projects and Solutions | Web Projects | Use the 64 bit version of IIS Express

answered Jun 5, 2016 at 14:54

Aleksey L.

38.4k11 gold badges86 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Ric .Net Over a year ago

Works like a charm! Thanx, never knew IIS ran on 32 bit by default, always thought it followed the built configuration.

Royi Namir Over a year ago

BTW - what's wrong with 80MB ? it's not even close to 2GB limit

Aleksey L. Over a year ago

@Royi, it turns out that when such an object is serialized/deserialized, memory consumption is much over than 80MB

Aleksey L. Over a year ago

@Royi, yeah, it still reproducible with Json.net 10. It must be also somehow related to test runner/visual studio. But regardless of unit test, it can also be reproduced with console app(32bit) with array of 220MB

Suicide Platypus Over a year ago

3 and a half years later, you saved me with this answer. Thanks.

Collectives™ on Stack Overflow

Json.Net throws OutOfMemoryException only in Visual Studio

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related