Hibernate - Searching - Java heap space

Question

I have a java heap space problem with hibernate since I add a modification in mi code. The program load information from a text file (2GB)

BufferedReader input = new BufferedReader(new FileReader(file));

while (line.compareTo(EOF) != 0) {
    //Load of infoObject, lots of parse information from the text file
    //Read lines of text using: line = input.readLine();

    getSession().save(infoObject);

    counter++;

    if (counter % 100 == 0) {
        getSession().flush();
        System.gc();
    }
}

This work great, now in case that the infoObject already exists in my db, I need to update the record, I use this:

BufferedReader input = new BufferedReader(new FileReader(file));

while (line.compareTo(EOF) != 0) {
    //Load of infoObject

    iObject infoObject_tmp = new iObject();
    infoObject_tmp.setNumAccount(numAccount);
    infoObject_tmp.setCloseDate(new Date("02/24/2011"));
    iObject infoObject_search = (iObject) getSession().load(iObject.class, infoObject_tmp);

    if (infoObject_search !=null){
        getSession().update(infoObject);
    }else{
        getSession().save(infoObject);
    }
    counter++;

    if (counter % 100 == 0) {
        getSession().flush();
        System.gc();
    }
}

FreeMemory:

Registry 1: 750283136.
Registry 10000: 648229608.
Registry 50000: 411171048.
Registry 100000: Java Heap Space.

How can i fix the java heap space problem? I know that the problem is when i check if the iObject exists or not.

So, what is your question, then?

buruzaemon
– buruzaemon

2011-05-16 14:18:26 +00:00
Commented May 16, 2011 at 14:18 — buruzaemon
– buruzaemon, Commented May 16, 2011 at 14:18
How can i fix the java heap space problem?

JMira
– JMira

2011-05-16 14:28:13 +00:00
Commented May 16, 2011 at 14:28 — JMira
– JMira, Commented May 16, 2011 at 14:28

jtahlborn · Accepted Answer · 2011-05-16 15:11:39Z

3

you need a flush() followed by a clear(). flush() only pushes any pending updates to the database. clear() removes the objects from the session. you need to be careful with clear() though, because any other code which has a reference to an object which was in the session now has a "detached" object.

answered May 16, 2011 at 15:11

jtahlborn

53.8k5 gold badges80 silver badges122 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

JMira Over a year ago

This improve the performance a lot, i'm running some test but i think that this solve my problem.

Michael McGowan · Accepted Answer · 2011-05-16 15:18:46Z

1

It's tough to tell what exactly is going on without seeing more of your code. It's also a bit confusing since your BufferedReader does not appear to be used.

That said, one possibility, given what it looks like you are doing, is that somewhere you are calling substring on some text input and keeping a reference to the String object that is returned as a result. For instance, you might be doing something like this:

List<String> allMySubstrings = new ArrayList<String>();
while((line = input.readLine()) != null){
  String mySubstring = line.subString(40, 42);
  // mySubstring still has a reference to the whole character array from line
  allMySubstrings.add(mySubstring);
}

You might think then that you will only have references to allMySubstrings and not every line that was read. However, because of the way subString is implemented, the resulting String object will have a reference to the entire original character array from line, not just the relevant substring. Garbage collection won't occur as you might expect as a result

If you are doing this, you can get around it by creating a new String object:

List<String> allMySubstrings = new ArrayList<String>();
while((line = input.readLine()) != null){
  String mySubstring = new String(line.subString(40, 42));
  allMySubstrings.add(mySubstring);
}

Note that this might be happening even if you do not explicitly call subString as common methods like split might be responsbile.

edited May 16, 2011 at 15:18

answered May 16, 2011 at 15:11

Michael McGowan

6,6168 gold badges45 silver badges70 bronze badges

2 Comments

JMira Over a year ago

The problem is not reading the file, I take out the code that check if the account exists, and the memory performance is great.Any way i will check the strings, thank you!

Michael McGowan Over a year ago

@JMira Does the code checking if the account exists call split or subString on line and hold on to the results in some collection by any chance?

Harry Lime · Accepted Answer · 2011-05-16 14:56:53Z

0

You declare BufferedReader input, but you never seem to use it. Could it be that the line Object that you refer to in the while loop is accessing the underlying File in a non-buffered way?

answered May 16, 2011 at 14:56

Harry Lime

29.6k4 gold badges34 silver badges38 bronze badges

3 Comments

JMira Over a year ago

Yes, I don't put the whole code because is really big, but I use: line = input.readLine(); To access the text file.

Harry Lime Over a year ago

Does the file contain line terminating characters (\n or \r)?

JMira Over a year ago

Yes, the problem is not reading the file, I take out the code that check if the account exists, and the memory performance is great.

Collectives™ on Stack Overflow

Hibernate - Searching - Java heap space

3 Answers 3

1 Comment

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related