How to handle out of memory exception while working with HashMap

Question

I am using a HashMap which stores 1000 entries at a time. After processing the records I call clear() method and loads 1000 fresh entries for same HashMap reference. I have to do this same process for 2,000,0000 records. Buyt after processing only 750,000 its gives me Exception in thread "ThreadedStreamConsumer" java.lang.OutOfMemoryError: Java heap space.

I have also tried WeekHashMap but it also didn't work out for me. What should be the effective method to handle this?

Either you're not really clearing the map, or you're holding onto the data elsewhere. Or maybe, given the "ThreadedStreamConsumer," you're spinning up too many threads and their total consumption exceeds what you have available. But without you showing actual code, it's impossible to say what the real problem might be. — kdgregory
– kdgregory, Commented May 30, 2014 at 20:50
You have to somehow eliminate all references to the objects which you want to "forget". Eg, if you create a new HashMap for the new ones, you must null all references to the old HashMap (or overwrite them with references to the new one). And any other array containing lists of the objects must be similarly disposed of. Somewhere you're hanging on to lists of old objects. — Hot Licks
– Hot Licks, Commented May 30, 2014 at 21:11
are the records the same size. Is it possible that the error occur in a specific record which is causing the error? Have you noticed if it occurs in a specific record? Can you increase the available memory of your project? — Eypros
– Eypros, Commented May 30, 2014 at 21:56

j3ny4 · Accepted Answer · 2014-05-31 11:49:48Z

0

I guess the memory leak isn't related to HashMap, but somewhere else. You would find where the leak is by analyzing the heap dump.

In order to instruct JVM to produce the heap dump upon OutOfMemoryError add -XX:+HeapDumpOnOutOfMemoryError parameter to the java command line.

Once you have the heap dump, load it using VisualVM or any other profiler of your choice. That will allow you to find what's the object holds references to the processed records and amend you program to prevent that.

answered May 31, 2014 at 11:49

j3ny4

4412 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to handle out of memory exception while working with HashMap

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related