out of memory exception on StringWriter using MVC, can memory be increased?

Question

I have to create a large XML Spreadsheet and it now seems to have hit the threshold on what it can cope with in one go.

I'm not sure on how to split it out so wanted to know if there is a way to increase the memory allocation for internal purposes?

Code that is causing the error is below:

private string RenderViewToString()
{
    using (var writer = new StringWriter())
    {
        var view = ViewEngines.Engines.FindView(_context, _viewName, null).View as RazorView;
        var viewDataDictionary = new ViewDataDictionary<TModel>(_model);
        var viewCtx = new ViewContext(_context, view, viewDataDictionary, new TempDataDictionary(), writer);
        viewCtx.View.Render(viewCtx, writer);
        return writer.ToString();
    }
}

It falls on the Render() when a user decides to download ALL data from the possible combinations of data, before the enrichment of data this file would be circa 55mb converted into an Excel document. Now that the columns have been doubled, the line of code rendering the writer is falling over.

viewCtx.View.Render(viewCtx, writer);

Has anyone got a workaround or know a way to split the relevant data into manageable chunks without recoding it all?

It obtains the data from a view model and that itself takes from 1 to tens of thousands of rows across multiple tables and all is needed by the users in a singular Excel download which is causing issues.

The string is going out to this code:

    protected override void WriteFile(HttpResponseBase response)
    {
        response.Write(RenderViewToString());

    }

But the whole issue of the of whether it needs to be returned string is irrelevant as it never gets that far due to the ViewContext.View.Render(ViewContext, TextWriter) falling over, so is there a way to render the ViewContext to something without it falling over due to a relatively small file (60+ MB) causing it issues?

I have tried to allow the large file use in web.config (A colleague suggested it) but that didnt work.

At the moment the ViewContext is too large to render so I just need to find a way to render it without using the in built one, any ideas?

----- Edit -----

Big thanks to Joe, I modified your code as I dont have IIS Set up to use the HttpContext headers, but the following code did the trick:

        HttpContext.Current.Response.Clear();
        using (var writer = new StreamWriter(HttpContext.Current.Response.OutputStream))
        {
            var view = ViewEngines.Engines.FindView(_context, _viewName, null).View
                as RazorView;
            var viewDataDictionary = new ViewDataDictionary<TModel>(_model);
            var viewCtx = new ViewContext(_context,
                view, viewDataDictionary, new TempDataDictionary(), writer);
            viewCtx.View.Render(viewCtx, writer);
        }
        return null;

Now produces a 70MB file no problems.

More RAM isn't going to help (unless you can put 1TB in a 64bit PC maybe:) ). A strategy for paging/splitting the data is required here. — Jason Evans
– Jason Evans, Commented Oct 17, 2013 at 15:13
Do you really need the string itself, or are you writing to an output of some kind, like a file or the HTTP response? — Joe Enos
– Joe Enos, Commented Oct 17, 2013 at 15:19
@Joe - The output is an Excel file that allows the user to save or open it. Nothing more than that. — Simon Osbon
– Simon Osbon, Commented Oct 17, 2013 at 16:05
Checkout stackoverflow.com/questions/1569532/…. It shows how to return plain-text files, but you can definitely tweak it to fit your needs. — Jim D'Angelo
– Jim D'Angelo, Commented Oct 17, 2013 at 16:13

Joe Enos · Accepted Answer · 2013-10-17 16:32:13Z

1

You're writing the view output to a string, then writing it to a response. What you may want to do is skip the middle-man, and write the view directly to the response. Since the Render method accepts any TextWriter, you can pass in Response.OutputStream to your StreamWriter instead of just writing to a plain string.

So something like:

System.Web.HttpContext.Current.Response.Clear();
System.Web.HttpContext.Current.Response.Headers["content-disposition"] = 
    "attachment;filename=somefile.txt"; // or whatever
System.Web.HttpContext.Current.Response.Headers["content-type"] =
    "text/plain"; // or whatever
using (var writer = new StreamWriter(System.Web.HttpContext.Current.Response.OutputStream))
{
    var view = ViewEngines.Engines.FindView(_context, _viewName, null).View
        as RazorView;
    var viewDataDictionary = new ViewDataDictionary<Foo>(_model);
    var viewCtx = new ViewContext(_context, 
        view, viewDataDictionary, new TempDataDictionary(), writer);
    viewCtx.View.Render(viewCtx, writer);
}
return null;

answered Oct 17, 2013 at 16:32

Joe Enos

40.6k11 gold badges86 silver badges144 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Simon Osbon Over a year ago

Cheers I will have a look at that and report back.

Simon Osbon Over a year ago

now its failing again after working for one day... any other ideas?

Joe Enos Over a year ago

Not really - this should take care of the problem you had from a large string. But if you've got a different large item in memory, like your model or dictionary, I don't know if you can easily fix that. Resolving this type of problem usually means streaming data instead of building it all upfront, but I don't know if it's something you'd be able to do given this type of data. I'd probably recommend exploring splitting your data into smaller chunks, so you can keep less in memory at a time.

Collectives™ on Stack Overflow

out of memory exception on StringWriter using MVC, can memory be increased?

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related