0

I want to start a wildfly on my virtual server. Every time I run ./wildfly/bin/standalone.sh I get the following error while bootstrapping:

MSC000001: Failed to start service jboss.ironjacamar.idle-remover: org.jboss.msc.service.StartException in service jboss.ironjacamar.idle-remover: java.lang.OutOfMemoryError: unable to create new native thread
    at org.jboss.as.connector.subsystems.jca.IdleRemoverService.start(IdleRemoverService.java:53)
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
    at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881) [jboss-msc-1.2.2.Final.jar:1.2.2.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_45]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_45]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_45]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
    at java.lang.Thread.start0(Native Method) [rt.jar:1.8.0_45]
    at java.lang.Thread.start(Thread.java:714) [rt.jar:1.8.0_45]
    at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950) [rt.jar:1.8.0_45]
    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1357) [rt.jar:1.8.0_45]
    at java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:668) [rt.jar:1.8.0_45]
    at org.jboss.jca.core.connectionmanager.pool.idle.IdleRemover.start(IdleRemover.java:137)
    at org.jboss.as.connector.subsystems.jca.IdleRemoverService.start(IdleRemoverService.java:51)
    ... 5 more

A ulimit -a gets me the following: 

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 385262
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 385262
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

As you can see, virtual memory is unlimited and max user processes is pretty hugh.
I also increased/decreased the stack size with -Xss256kb when starting the jvm as described in this post but without any success.
Also increasing max-threads by 50 in standalone.xml for every subsystems did not work (described in this post).
Do you have any other idea how to solve this problem?

Java version is

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
Improve this question
6
  • 2
    Try increasing open files, see stackoverflow.com/a/15736766/3080094 Commented Jun 21, 2015 at 9:47
  • how much physical memory is on the box? Commented Jun 21, 2015 at 15:52
  • @vanOekel I set it now to 65536 but the error still occurs. Commented Jun 21, 2015 at 16:14
  • @John Ament the server has 2gb Commented Jun 21, 2015 at 16:14
  • What settings are you passing in when you start wildfly? Commented Jun 21, 2015 at 23:37

4 Answers 4

Reset to default
1

To fix OutOfMemoryError, launch your java process using -Xmx2048m switch.

Hope this helps.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This does not work as the error message does not point to heap space but to "unable to create new native thread". I also tried this but the error still occurs.
Did you try increasing the open files handles as suggested by @vanOkeal? That looks like the last option. You have tried pretty much everything.
I did. In theory I understand what's happening but practically I have no idea why. On my local dev machine everything works fine with the very same settings for the wildfly.
0

How many CPUs does your box have? IIRC, some thread pools in wildfly are creating threads in relation to the CPU count, so if you have unusually many CPUs, that might be a problem. Although it says it cannot create new threads it might still be a problem with memory (and not with max user processes) because every thread need memory as well.

Otherwise you might try to create a heap dump to find out how many thread are created by wildfly.

Maybe a complete startup log could also provide more information

Improve this answer

2 Comments

Thanks for your reply. The server has just one CPU.I will try to create a heap dump.
added hint on startup log
0

You need to increase the number of open files allowed. The key to the error is unable to create new native thread. On Linux if you execute ulimit -n 8192 that should fix it.

Improve this answer

Comments

0

On Linux systems, try increasing vm.max_map_count sysctl value if your system supports a higher value. Default OS limits on Memory Map counts are generally low - 65530.

On a CentOS or RHEL machine,

sysctl -w vm.max_map_count=131072

To make this change permanent, echo "vm.max_map_count = 131072" >> /etc/sysctl.conf

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.