Java OutOfMemoryError with ArrayList<List<Integer>>

Question

I want to create a very large graph (with ~10 million edges) in Java. I plan to List<List<Integer>> to describe the edges, with the inside List<Integer> describing the two vertices of each edge (and the vertices are Integer type).

The following code throws the OutOfMemoryError after about 1 million edges are added to the graph. (I simplified how the edge is generated for the sake of discussion.)

public static void main(String[] args) {
  List<List<Integer>> graph = new ArrayList<List<Integer>>();
  for (int i = 0; i < 10000000; i++) {
    List<Integer> edge = new ArrayList<Integer>();
    // the real edges are more complicated (than from vertex i to vertex i+1)
    // this is simplified for the sake of the discussion here
    edge.add(i);
    edge.add(i+1);
    graph.add(edge);
  }
}

I have searched for OutOfMemoryError, and I have increased the initial heap size to 2G for Eclipse: -Xms2g -Xmx4g -Xss2m (which get passed to JVM). But that did not solve the problem.

Then I thought maybe I should garbage collect the List<Integer> edge variable, by calling System.gc(), in case its memory does not get cleared. That did not work either.

I was thinking maybe the problem is with the List<List<Integer>> data structure. I tried List<int[]>, which lasted a bit longer: more edges are added before OutOfMemoryError happens. I do not have a better idea right now.

I have searched around for similar problems, but have not find much help. I wonder if anyone has experience with this kind of situation.

No the parameters you set for eclipse does not get passed to the program inside — SwiftMango
– SwiftMango, Commented Feb 16, 2015 at 1:24
Check the Eclipse launch configuration, Arguments tab, VM arguments text box; use an appropriate -Xmx argument there. Probably your jvm is being launched with the default limit, which is often 256MB. Also please look what's going on in your post with less-than and greater-than characters because it sure doesn't look right. — chrisinmtown
– chrisinmtown, Commented Feb 16, 2015 at 1:29
@chrislott, thank you for your comment about the VM argument in run config. I think by "less-than and greater-than characters" you are referring to the angle brackets for the generics? — Ethan
– Ethan, Commented Feb 16, 2015 at 2:47

Jeffrey · Accepted Answer · 2015-02-16 01:31:05Z

5

To let your program use more memory from Eclipse:

Go to Run -> Run Configurations. You will see this window

Click on Arguments Run Configurations/Arguments

Enter your arguments to the VM Run Configurations/Arguments/VM Arguments

answered Feb 16, 2015 at 1:31

Jeffrey

44.9k8 gold badges93 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

dimm · Accepted Answer · 2015-02-16 01:46:52Z

0

Since you are using a lot of RAM besides setting the max heap parameter, make sure you use 64-bit Java. The 32-bit is limited to 2 Gigs or something like that.

Also, for large graphs you should consider using a database.

And last but not least, maybe you can rethink your algorithm, sometimes you just don't need all the nodes and edges.

answered Feb 16, 2015 at 1:46

dimm

1,80813 silver badges16 bronze badges

Collectives™ on Stack Overflow

Java OutOfMemoryError with ArrayList<List<Integer>>

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related