Java cannot create new process: too many open file error

Question

I am starting too many processes in Java using:

       Runtime.getRuntime().exec("java -ja myJar.jar")

When it reaches around 350 processes, I get an IO exception:

 Cannot run program "java": java.io.IOException: error=24, Too many open files
 Exception in creating process
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
    at java.lang.Runtime.exec(Runtime.java:610)
    at java.lang.Runtime.exec(Runtime.java:448)
    at java.lang.Runtime.exec(Runtime.java:345)

In each process I am using one database connection. I am running ubuntu 32 bit OS. But when I run:

   ulimit -u

I can see that process limit is unlimited. What could be the problem?

the world is limited, even if you set ulimit -u unlimited. it would be better if you change your program in order to execute more threads, instead of launching the same program more than 350 times! — vulkanino
– vulkanino, Commented Feb 14, 2012 at 8:55
Note that there are two separate 'too many open files' file limits, with separate error numbers: ENFILE (Too many files open in system) and EMFILE (Too many files open). Usually, but by no means guaranteed, EMFILE is errno=24. That's fixed by your process using fewer file descriptors. The system problem is unusual these days, but can occasionally happen; it is often indicative that either a reboot is needed or some processes need cleaning up. — Jonathan Leffler
– Jonathan Leffler, Commented Apr 18, 2012 at 13:57

user1208576 · Accepted Answer · 2012-02-14 08:54:17Z

1

All systems have their limits - sounds like you've hit your system's limit.

In linux, creating new processes consumes lots of inodes (like windows handles), which is a lot like a file handle. The only way around it is to allocate more via kernel settings (I don't know how offhand).

Have you considered starting lots of java Threads instead? They would consume a lot less system resources.

answered Feb 14, 2012 at 8:54

user1208576

Sign up to request clarification or add additional context in comments.

1 Comment

user207421 Over a year ago

inodes are not handles. inodes are identifiers of files in the file system. Handles, or fds (file descriptors) belong to open files. OP is running out of handles. The way around it is to call Process.waitFor() in this instance.

Jonathan Leffler · Accepted Answer · 2012-04-18 13:52:22Z

1

The problem is that you have too many files open, not too many processes in operation. To check the file limit do:

ulimit -n

It will commonly be 1024.

Check http://www.puschitz.com/TuningLinuxForOracle.shtml and search for ulimit for good instructions on changing this limit.

edited Apr 18, 2012 at 13:52

Jonathan Leffler

759k145 gold badges961 silver badges1.3k bronze badges

answered Apr 18, 2012 at 5:09

Brett Dalton

111 bronze badge

Comments

PeterMmm · Accepted Answer · 2012-02-14 08:55:21Z

0

Running 350 JVM instances is not the normal way. Can you redesign and run 350 "main threads" within the same JVM. That is how servlet container works. All web applications are running in the same JVM.

PD. ulimit man pages says to see max. open files is ulimit -n.

answered Feb 14, 2012 at 8:55

PeterMmm

24.7k15 gold badges77 silver badges115 bronze badges

1 Comment

Muneeb Over a year ago

I am using this design for a bench-marking. ulimit -n gives 1024

Collectives™ on Stack Overflow

Java cannot create new process: too many open file error

3 Answers 3

1 Comment

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related