Dealing with OutOfMemory Exception

Question

Is there a proper way to deal with Out Of Memory Exceptions while looping through a large list and adding objects from that list? What is the proper way to go about doing this? I have a large Linq query that returns around 600K items. I then go through each item and add it to an object. The code is below.

  static void Main(string[] args)
    {
        GrabData();
    }

    public static void GrabData()
    {
        decimal? TotalNetClaim = 0;
        using (var Context = new VSCMeSnapEntities())
        {              
            List<DriveTimeObject> DataFile = new List<DriveTimeObject>(); 
            DriveTimeObject DT = new DriveTimeObject();
            DateTime ParamDate = new DateTime(2015, 05, 30);
            List<viewDriveTimeFileDump> DataQuery = new List<viewDriveTimeFileDump>();

            DataQuery = (from z in Context.viewDriveTimeFileDumps select z).ToList();

            foreach (var item in DataQuery)
            {
                decimal? AmountChargedParts = DT.GetAmountChargedParts(item.chrComponSts.Trim(), item.mnsTotalParts);
                decimal? AmountChargedPartsTax = DT.GetAmountChargedPartsTax(item.chrComponSts.Trim(), item.mnsTotalPartTax);
                decimal? AmountChargedLabor = DT.GetAmountChargedLabor(item.chrComponSts.Trim(), item.mnsTotalLabor);
                decimal? AmountChargedLaborTax = DT.GetAmountChargedLaborTax(item.chrComponSts.Trim(), item.mnsTotalLaborTax);
                int? DaysOut = DT.GetDaysOutClaim(item.intRepairFacilCode, item.dtmContPurchDate, item.dtmReported);
                long? MilesOut = DT.GetMilesOutClaim(item.intRepairFacilCode, item.inbIncurMiles, item.inbOrigMiles);
                decimal? deductible = DT.GetDeductible(item.chrContSts, item.mnsDeduct);
                decimal? netClaim = DT.GetNetClaim(item.chrComponSts.Trim(), item.mnsTotalParts, item.mnsTotalPartTax, item.mnsTotalLabor, item.mnsTotalLaborTax, item.mnsDeduct);

                DataFile.Add(new DriveTimeObject
                {
                    DealerNumber = item.chrDlrNum,
                    VSCName = item.chvVSCName,
                    IcLocationNumber = item.IcLocationNumber,
                    IcRegion = item.IcRegion,
                    Identifier = item.chrIdentifier,
                    ContractNumber = item.chrContNum,
                    VIN = item.chrVIN,
                    CoverageCode = item.CvgCode,
                    ClaimNum = item.intClaimNum,
                    OriginalMiles = item.inbOrigMiles,
                    ContractPurchaseDate = item.dtmContPurchDate,
                    IncurMiles = item.inbIncurMiles,
                    DateReported = item.dtmReported,
                    DaysOutClaim = DaysOut,
                    MilesOut = MilesOut,
                    RepairFacilityNumber = item.intRepairFacilCode,
                    FacilityName = item.chvFacilityName,
                    ZipFive = item.chrZipFive,
                    FacilityAdvisor = item.chrFacilAdvisor,
                    ComponentStatus = item.chrComponSts,
                    ComponentStatusWord = item.ComponDesc,
                    ComponentCode = item.chrComponCode,
                    StatusMasterDescription = item.MasterDesc,
                    ComponentDescription = item.chvComponDesc,
                    Parts = AmountChargedParts,
                    PartsTax = AmountChargedPartsTax,
                    Labor = AmountChargedLabor,
                    LaborTax = AmountChargedLaborTax,
                    Deductible = deductible,
                    NetClaim = netClaim,
                    CarrierCode = item.intCarrierCode,
                    NetworkStatus = item.NetworkStatus,
                    AddOn = item.chrAddOn,
                    ETCDate = item.ETC,
                    ATCDate = item.ATC,
                    LaborTime = item.reaLaborTime,
                    PaidDate = item.dtmPdDate,
                    PaymentID = item.intPaymentID,
                    BatchNumber = item.intBatchNum

                });

                TotalNetClaim += netClaim;
            }
            Context.Dispose();
        }

        Console.WriteLine(TotalNetClaim);
        Console.ReadKey();


    }

I run out of memory during the foreach loop and I was wondering how I should go about adjusting my code to make this work.

What is the point of the DataFile List? You keep lots of objects in it but never use it... — DavidG
– DavidG, Commented Jun 2, 2015 at 22:02

harsimranb · Accepted Answer · 2015-06-02 22:30:10Z

1

The way to prevent out of memory is to not run out of memory. Which means you need to get rid of objects you don't need.

Without learning more of your use case, it is hard to suggest the fix. Regardless of that, it's bad practice to have so many objects in memory that you run out and crash. Only keep in memory what you need.

One fix is to not use RAM memory, and instead use hard drive memory. Ex: You can write those objects to a database and get rid of them, so you don't keep them around. Considering you have 600k objects, you could do these in batches of 10k/25k records. Then when you need the objects, you can query them. If you need to do calculations with all the objects, I would recommend doing those operations using SQL queries.

edited Jun 2, 2015 at 22:30

answered Jun 2, 2015 at 22:08

harsimranb

2,3031 gold badge35 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

DavidG · Accepted Answer · 2015-06-02 22:07:23Z

0

You are creating a new DriveTimeObject and storing it in a List<DriveTimeObject> called DataFile. Assuming there are 600k items in your DataQuery, this means that your list also contains 600k items.

However, you are not using that list at all so it's just chewing up all of your memory for no reason. Remove that and save yourself a whole tonne of memory and it should also run a lot faster.

answered Jun 2, 2015 at 22:07

DavidG

120k13 gold badges232 silver badges239 bronze badges

Collectives™ on Stack Overflow

Dealing with OutOfMemory Exception

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related