OutOfMemoryError: Java heap space error when start solr

Question

I start indexing DB articles with solr, but after add about 58 million article (and about 113 GB size of disk) , i get below error message on tomcat log error

Note1: i already set Init memory pool to 256MB, and Max memory pool:1400MB to tomcat server.
Note2: I can post or search article but must wait over 3 min for get response.

8-apr-2010 14:27:07 org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: Java heap space
    at org.apache.lucene.util.PriorityQueue.initialize(PriorityQueue.java:89)
    at org.apache.lucene.search.HitQueue.<init>(HitQueue.java:67)
    at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:113)
    at org.apache.lucene.search.TopScoreDocCollector.<init>(TopScoreDocCollector.java:37)
    at org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:42)
    at org.apache.lucene.search.TopScoreDocCollector$InOrderTopScoreDocCollector.<init>(TopScoreDocCollector.java:40)
    at org.apache.lucene.search.TopScoreDocCollector.create(TopScoreDocCollector.java:100)
    at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:979)
    at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
    at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
    at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
    at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
    at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:859)
    at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:574)
    at org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1527)
    at java.lang.Thread.run(Unknown Source)

What's problem ?

Have any suggestion ?

Important Question: Why solr using heap memory ???

Have you tried to attach jconsole to get a better look what is going on? — Patrick Cornelissen
– Patrick Cornelissen, Commented Apr 8, 2010 at 13:21
Is it ok for the memory footprint to increase? are you storing some part of the index in memory? If the answer is no, I suppose this is a memory leak. Try using: jmap -histo <process id> to see what kind of objects occupy a larger portion of the heap than expected. — Eyal Schneider
– Eyal Schneider, Commented Apr 8, 2010 at 13:35
thanks for reply. i don't know many about java, also i can'f find jmap. only install YourKit java profiler on server. how to find what's problem ? — Hamid
– Hamid, Commented Apr 8, 2010 at 14:19
what are you using to index your documents? DataImportHandler? — Mauricio Scheffer
– Mauricio Scheffer, Commented Apr 8, 2010 at 19:10

Alex Nauda · Accepted Answer · 2010-04-16 22:47:43Z

5

Upgrade to the latest jdk 6 if you haven't already. I hit a similar OOME on jdk 5 that went away with 6. I suspect an nio thing.
Try lowering maxPendingDeletes in your solrconfig.xml.
To find out what component of solr is using up all of the memory, start solr like this:

java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=some_directory_of_your_choice -jar start.jar

Then parse the heap dump using the stand-alone MAT from http://www.eclipse.org/mat/ and lower the corresponding cache that is eating up all your memory.

answered Apr 16, 2010 at 22:47

Alex Nauda

4,5461 gold badge27 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Hamid Over a year ago

I just install windows 64 bit for set bigger memory for tomcat (solr) and my problem solve, because segment very large and can't open with low memory.

Jé Queue · Accepted Answer · 2010-04-16 22:51:18Z

1

Enable GC logging and graph the timeseries, this can tell you velocity of JVM memory allocation and how much is being collected if at all so you can approximate a high water mark.

answered Apr 16, 2010 at 22:51

Jé Queue

10.7k14 gold badges59 silver badges62 bronze badges

Collectives™ on Stack Overflow

OutOfMemoryError: Java heap space error when start solr

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related