System.OutOfMemoryException on savechange() in c# using nopCommerce [duplicate]

Question

I around have 400,000 records which is coming from file. But after inserting 30000 to 33000 of records it is throwing error 'System.OutOfMemoryException'

In my system i have 8gb of ram i think it is enough for this. And i inserting bunch of records like 500 records at a time using this code.

this._context.AutoDetectChangesEnabled = false;

if(Counter % 500 == 0)
    _context.SaveChanges();

i tried all possible changes and do lots of r & d and also find all possible solution on this stack overflow site but it can't help.

I am using nopcommerce basically for this context using

private readonly IDbContext context;

And also let me know if any confusion!

public partial class EfRepository<T> : IRepository<T> where T : BaseEntity
{
      private IDbContext _context;
      private IDbSet<T> _entities;

      protected virtual IDbSet<T> Entities
      {
          get
          {
              if (_entities == null)
                  _entities = _context.Set<T>();
              return _entities;
          }
      }

      public virtual void Insert(IEnumerable<T> entities, bool enableTrackChanges = true)
        {
            try
            {
                if (entities == null)
                    throw new ArgumentNullException("entities");

                if(!enableTrackChanges)
                {
                    this._context.AutoDetectChangesEnabled = false;
                }

                foreach (var entity in entities)
                    this.Entities.Add(entity);

                this._context.SaveChanges();
            }
            catch (DbEntityValidationException dbEx)
            {
                throw new Exception(GetFullErrorText(dbEx), dbEx);
            }
            finally
            {
                if (!this._context.AutoDetectChangesEnabled)
                    this._context.AutoDetectChangesEnabled = true;
            }
        }
}

Your description of the issue is a bit light. From which line of code is the exception coming from? Is it the SaveChange? Could you maybe show the loop in your question? — Gimly
– Gimly, Commented Aug 23, 2017 at 7:32
on context.SaveChanges(); when loop is coming on this after saving 30000 of records then i get error — PixelDev
– PixelDev, Commented Aug 23, 2017 at 7:35
How much ram you have is irrelevant. Read It's the address space stupid (which may sound rude if you're not aware that it's wordplay based on a famous quote) — Damien_The_Unbeliever
– Damien_The_Unbeliever, Commented Aug 23, 2017 at 7:44
Where did you kept your savechanges()...is it in loop or outside loop? — LifeOfPi
– LifeOfPi, Commented Aug 23, 2017 at 7:45

m4ttsson · Accepted Answer · 2017-08-23 09:34:17Z

4

You need to recreate the context in order to avoid outofmemoryexception. The memory will increase until entity framework can't handle it and then the exception is thrown. It is a memory limitation in entity framework so it won't help to run on an even better computer.

This should work:

if(Counter % 500 == 0)
{
    _context.SaveChanges();
    _context = new ....();
}

Also see this for some more help on inserting many rows with EF. Fastest Way of Inserting in Entity Framework

Edit: I apologize for being unclear and not citing sources. The limitation of an object in .NET is 2GB, arrays can be bigger on 64bit machines but an EF context cannot be bigger than 2GB. And when using an EF context continuously the memory usage may increase rapidly until the limit is reached. To free memory it is therefore best to renew the context after a while when inserting many rows of data. So the limit is on .NET, not on EF as I mistakenly said before.

About EF context: https://msdn.microsoft.com/en-us/library/jj729737(v=vs.113).aspx

How to get >2gb arrays (also says that other objects are unchanged) https://learn.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element

Old longer source about 2GB limit and larger arrays: https://blogs.msdn.microsoft.com/joshwil/2005/08/10/bigarrayt-getting-around-the-2gb-array-size-limit/

edited Aug 23, 2017 at 9:34

answered Aug 23, 2017 at 7:41

m4ttsson

2032 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

14 Comments

PixelDev Over a year ago

Thanks for your answer i will try this and let you know because it's take time.

PixelDev Over a year ago

in this here my context is this private readonly IDbContext _context; In IDbContext it's an interface

PixelDev Over a year ago

Can you help me to using this i am using nopcommerse for this

m4ttsson Over a year ago

How is the IDbContext created/inserted into the class that has the _context?

CodeCaster Over a year ago

"It is a memory limitation in entity framework" - that sounds like voodoo programming. Don't suggest fixes if you don't understand the problem.

|

Collectives™ on Stack Overflow

System.OutOfMemoryException on savechange() in c# using nopCommerce [duplicate]

1 Answer 1

14 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

14 Comments

Linked

Related