Java needed memory for new thread

Question

I have an application which massively creates threads. As result I get an OutOfMemoryError. My idea is to wait until there is enough free space to create the next Thread. But therefore I need to know how many memory I need to create a thread and if this amount of memory is available. Is there a way to get the amount of memory a thread needs? And how can I determine if this amount of memory is available?

What I've already tried:

for (int i = 0; i < links.size(); i++) {
    while(Runtime.getRuntime().maxMemory() - Runtime.getRuntime().totalMemory() +
        Runtime.getRuntime().freeMemory() < MEMORY_THRESHOLD) {
        synchronized (lock) {
            lock.wait(10);
        }
    }
    Thread t = new Thread(new Task(links.get(i)));
    t.setDaemon(true);
    t.start();
}

But even when I use 100MB as threshold I get an OutOfMemoryError.

I'm not an expert here ,but I feel like you're comparing apples to oranges with the line : .maxMemory() - untime.getRuntime().totalMemory() , maxMemory and totalMemory sound different to me — Caffeinated
– Caffeinated, Commented Oct 31, 2014 at 15:23
@Coffee this should be the memory the JVM can allocate, when I just use freeMemory(), it blocks before the first thread is created. — kaetzacoatl
– kaetzacoatl, Commented Oct 31, 2014 at 15:27
An application that "massively creates threads" sounds a bit scary to me. Creating threads is expensive as you've found, which is why thread pools exist. How many threads are you creating? If you are creating so many threads that you're causing memory pressure that is a serious problem. I would suggest that you take a step back and reconsider your solution. The book "Java Concurrency in Practice" is a good resource here. — helmy
– helmy, Commented Oct 31, 2014 at 15:35
@DavidSchwartz, Back when threads first became popular, multiprocessor computers did not exist outside of University research departments. Threads were invented as a way to organize a program (especially a real-time, embedded program) that had to wait for many different asynchronous inputs. Instead of having one big event loop that polled for each of the different possible inputs, the program would have one thread to wait on each different input source. — Solomon Slow
– Solomon Slow, Commented Oct 31, 2014 at 16:07

Sleiman Jneidi · Accepted Answer · 2015-02-13 11:24:07Z

6

No, you go and delete this code and use a ThreadPool instead. What you trying to do is quite hard and you should be using the higher abstraction you have to do multithreading or you don't do it at all.

Check out Executors, currently they are the higher abstraction. If your tasks are compute bound then ForkJoin in Java 7 is your friend and you don't need to bother about the pool size as long as you know how to recursively divide your problem into subproblems.

edited Feb 13, 2015 at 11:24

answered Oct 31, 2014 at 15:24

Sleiman Jneidi

23.4k14 gold badges60 silver badges80 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

kaetzacoatl Over a year ago

Your solution works, but I still don't know how mny threads I can create. Executors.newFixedThreadPool() asks me for the maximum amount of threads, but thats exactly what I don't know.

Voo Over a year ago

@kaetzacoatl somewhere between the number of cpu cores and twice that, more and performance degrades needlessly. If you have tons of threads waiting around all day long use an async framework.

Ioannis Deligiannis · Accepted Answer · 2014-10-31 16:15:10Z

There is not enough information on the question in order to suggest an optimal solution.

Typically you may need to create many more threads than your actual CPU cores when you have expensive I/O operations. Judging from the links variable, this might be one of these cases, but as others suggested creating too many threads usually suggests a bad design. It might even get your system to run out of available resources.

Even when going over the network, your bandwidth is fixed, so unless you have direct access to an internet backbone it is highly unlikely that you'll get any performance benefit as most websites should respond with high speed.

Having said that, even if you slowed down thread creation you cannot control how much memory a thread will consume and at what time. i.e. All of your threads might start downloading data at the same time and even if you don't create any new ones, you will still run out of memory.

In conclusion, follow what others users suggested:

Use an ExecutorService
Read a bit more about Threads (Java Concurrency in practice, is a very good source)
Think why you are creating so many threads? Why do you expect that whatever you do will work faster?

PS. It would also help if you included why you need to create so many threads in your question

Collectives™ on Stack Overflow

Java needed memory for new thread

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related