OutOfMemoryException trying to write Xml from DataSet

Question

In C# i'm trying to get the xml string from a big DataSet:

private string GetXmlFromDecomposedPortfolio(string dataSetName, DecomposedPortfolio ptf)
{
     StringWriter writer = new StringWriter();

     System.Data.DataSet ds = new System.Data.DataSet(dataSetName);

     ds.Tables.Add(ptf.Security.Copy());   
     ds.WriteXml((TextWriter)writer, XmlWriteMode.IgnoreSchema);

     return writer.ToString();
}

but i've got the exception:

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.     
    at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)    
    at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)    
    at System.Text.StringBuilder.Append(Char value)     at System.IO.StringWriter.Write(Char value)  
    at System.Xml.XmlTextWriter.WriteStartElement(String prefix, String localName, String ns)   
    at System.Data.DataTextWriter.WriteStartElement(String prefix, String localName, String ns)  
    at System.Data.XmlDataTreeWriter.XmlDataRowWriter(DataRow row, String encodedTableName)  
    at System.Data.XmlDataTreeWriter.Save(XmlWriter xw, Boolean writeSchema)    
    at System.Data.DataSet.WriteXml(XmlWriter writer, XmlWriteMode mode)   
    at System.Data.DataSet.WriteXml(TextWriter writer, XmlWriteMode mode)    
    at Decompose.Library.Render.GetXmlFromDecomposedPortfolio(String dataSetName, DecomposedPortfolio ptf)   
    at Decompose.Library.Render.SavePE()  
    at Decompose.Library.WorkFlow.ProcessBatch()

Any suggestion?

Managed code solutions that use classes from the System.Data or System.Xml name may encounter a System.OutOfMemoryException when working with large datasets. — Tim Schmelter
– Tim Schmelter, Commented Dec 12, 2012 at 16:14

Tigran · Accepted Answer · 2012-12-12 16:17:42Z

1

First you create huge (apparently) xml data

ds.WriteXml((TextWriter)writer, XmlWriteMode.IgnoreSchema);

After clone it, into the string object writer.ToString();, so you almost double the memory you need.

What you can do, is to create XML row-per-row, so create kind of XmlDataSetRowEnumerator, that retrieves XML-per-row and yield returns the resulting XML.

answered Dec 12, 2012 at 16:17

Tigran

62.3k8 gold badges90 silver badges124 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

L.Grillo Over a year ago

And what about splitting the dataset in different ones?

Collectives™ on Stack Overflow

OutOfMemoryException trying to write Xml from DataSet

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related