Java heap space error while parsing large xml file

Question

I want to parse a large xml file(785mb) and write the data to csv. I am getting java heapspace error(out of memory) when I try to parse the file. I tried increasing the heap size to 1024mb but the code could handle a file of 50mb maximum.

Please let me know a solution for parsing large xml file in java.

One thing that I've learned when parsing data from a file is not to cache it. Ensure that you're not pulling the entire file into an object. — generalcrispy
– generalcrispy, Commented Oct 16, 2014 at 14:08
If you use 32-bit java you won't go over cca 1.5GB of heap space — Miki
– Miki, Commented Oct 16, 2014 at 14:09
Use a SAX parser. It takes stream approach to loading and parsing of the xml file. Whichever parser you are using seem to be loading the entire xml file into memory and that could be/likely causing the issue. Google for SAX parsers, you will find one. — KiranCK
– KiranCK, Commented Oct 16, 2014 at 14:16

Romain · Accepted Answer · 2014-10-16 14:13:09Z

1

You should use a SAXParser instead of a DOMParser The difference is that it doesn't load the complete XML data in memory.

Look at this tutorial : http://www.mkyong.com/java/how-to-read-xml-file-in-java-sax-parser/

Regards,

Romain.

answered Oct 16, 2014 at 14:13

Romain

1,3532 gold badges9 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

amrita jathkar Over a year ago

I cannot use a SAX parser as the tags in the xml file are unknown. So I won't be able to create a error handler for SAX parser.

Romain Over a year ago

Ok so the StAX parser might be a better choice for you because it uses an iterator approach.

pbespechnyi · Accepted Answer · 2014-10-16 14:10:43Z

0

The solution here is to use Streaming Api for XML (StAX). Here is good tutorial.

answered Oct 16, 2014 at 14:10

pbespechnyi

2,2931 gold badge19 silver badges29 bronze badges

Collectives™ on Stack Overflow

Java heap space error while parsing large xml file

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related