62

I profile running Java applications often with VisualVM but it needs X to run on the machine.

I know I can connect through management port but that will be an offline sampled profiling which is not enough for me.

So I'm looking for a solution with which I can profile the CPU usage of the methods of a running Java application from command-line. It's enough for me to collect data on the server and then the collected data can be analyzed on a different machine.

Update:

It seems I need to be more specific. I want to profile a running Java application from command line, I don't want to stop it and rerun it.

Improve this question
6
  • 3
    Can you collect 10 or 20 stack samples with jstack? Then if Foo is a method, its overall time usage is the fraction of samples containing it. Its CPU usage is the fraction of those samples that don't terminate in I/O or a system call. Its "self time" is the fraction of samples in which it itself is the terminus. Commented Jul 27, 2011 at 18:30
  • that would be the same as the VisualVM offline profiling, won't be? Commented Jul 28, 2011 at 13:56
  • 1
    Check the doc. It doesn't tell you, by line (not function) the percent of inclusive time (not exclusive) that line is responsible for, and overall time (not just CPU). It suffers from these problems. Stack sampling is not pretty, but it finds the performance problems very quickly. Other tools are pretty, and they give you lots of numbers to puzzle over, but they don't take you straight to the problem, whatever it is. Commented Jul 28, 2011 at 17:12
  • I tried to get the stack traces but it refuses to run without the -F flag, with which it freezes my app. Commented Jul 28, 2011 at 17:48
  • Hey, I don't need anything pretty. I either run it under the IDE and collect them that way, or use something like jstack that snapshots the stack of a running app. Commented Jul 29, 2011 at 1:35

7 Answers 7

Reset to default
32

The jvmtop application is a convenient tool for profiling from the commandline. No need to stop the jvm. Usage:

jvmtop.sh --profile <PID>

Will give you output like this which will be updating while the app runs:

  Profiling PID 24015: org.apache.catalina.startup.Bootstrap
  36.16% (    57.57s) hudson.model.AbstractBuild.calcChangeSet()
  30.36% (    48.33s) hudson.scm.SubversionChangeLogParser.parse()
   7.14% (    11.37s) org.kohsuke.stapler.jelly.JellyClassTearOff.parseScript()
  ...

The advantage is that it does not take the use of instrumentation. The classes of the to-be-profiled jvm will not be altered.

If you are looking for something more visual then have a look at jvm-mon which is based on jvmtop

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

I've been messing witth and cursing visualvm for years. Thanks for pointing me to this handy tool. Works without JVM agents, server restarts, etc. Simply untar on the box you want to profile, point JAVA_HOME to the right place and run.
This is an excellent tool but the project looks like it is dead. no release since 2015
jvm-mon is a ...sub-optimal tool, profiling should mean also stack trace.
Looks like jvmtop doesn't work any more since JDK 9.
Someone updated for Java 11 but it still doesn't work with Java 17
28

Looks like the "built-in" way to profile a java app from the command line is to start it with profiling command line parameters, like this

$ java -Xrunhprof:cpu=samples,file=myprogram.hprof ...

Then examine the file "myprogram.hprof" with some GUI tool (or web server tool like jhat) or command line tool after the process exits (and the file is created at that time).

If you use the "QUIT" signal trick, mentioned https://stackoverflow.com/a/2344436/32453 then you can generate a file at will without exiting the JVM (it appears to append to the previous output file). Or wait until the process exits and it will generate the file.

This (built-in) profiler does a sample infrequently so typically low slowdown/impact overall.

ref: http://web.archive.org/web/20160623224137/https://thunderguy.com/semicolon/2004/04/18/profiling-a-java-program-easily/

You could also just do the "poor man's profiler" by collecting lots of jstacks and dumping them into ex: a flamegraph or some other analyzer/conglomerator...

Improve this answer

1 Comment

Thanks for the pointer! Note for future readers: hprof is sampling threads in the Java RUNNABLE state (so it's not equivalent of aggregating multiple jstack dumps, like I expected, but neither does it measure CPU consumption). It has other gotchas - see brendangregg.com/blog/2014-06-09/…
16

Can you collect 10 or 20 stack samples with jstack? Then if Foo is a method, its overall time usage is the fraction of samples containing it. Its CPU usage is the fraction of those samples that don't terminate in I/O or a system call. Its "self time" is the fraction of samples in which it itself is the terminus.

I don't need anything pretty. I either run it under the IDE and collect them that way, or use something like jstack that snapshots the stack of a running app.

That's the random-pause technique.

Improve this answer

3 Comments

I selected this idea: svn.apache.org/repos/asf/jackrabbit/sandbox/jackrabbit-j3/src/…
This solution helped me to find a bottleneck with DNS resolution on the server. Thank you again!
@KARASZI István: Glad it helped.
7

The most precise profiling can be achieved with https://github.com/jvm-profiling-tools/async-profiler.

This project is a low overhead sampling profiler for Java that does not suffer from Safepoint bias problem. It features HotSpot-specific APIs to collect stack traces and to track memory allocations. The profiler works with OpenJDK, Oracle JDK and other Java runtimes based on HotSpot JVM.

Here is my script to install and run it from command-line:

async-profiler.sh

if [ ! -d profiler ]; then
  mkdir profiler && cd profiler && curl -L https://github.com/jvm-profiling-tools/async-profiler/releases/download/v1.6-ea/async-profiler-1.6-ea-linux-x64.tar.gz | tar xvz
  echo 1 > /proc/sys/kernel/perf_event_paranoid
  echo 0 > /proc/sys/kernel/kptr_restrict
  #apt install openjdk-8-dbg
else
  cd profiler
fi

#jps

./profiler.sh -d 60 -f dump_`date +%Y-%m-%d_%H-%M-%S`.jfr `jps -q`

It assumes that app is run under same user and there is a single java process PID to be listed by jps. Profiling duration is 60 seconds.

No modification of app's startup options or app restart is needed.

GUI for examining dumps is built-in into IntelliJ IDEA Ultimate: https://www.jetbrains.com/help/idea/cpu-profiler.html.

Improve this answer

2 Comments

Nice illustrated article on async-profiler: hackernoon.com/…
Another intro: baeldung.com/java-async-profiler
6

We have used hprof on our servers and it definitely is better than sysouts in case you can't run a full fledged VisualVM session.

Examples of using hprof are plenty out there:

Improve this answer

6 Comments

but I don't really get how can I attach it to a pid
Sorry, AFAIK, hprof doesn't allow for attaching to a process. Your best bet would be to look into tools like jprofile and jtop. java.sun.com/developer/technicalArticles/J2SE/monitoring
@SanjayT.Sharma i am using HPROF for CPU profiling by using cpu=times,and generating the data after each specific run of the application,but i am not able to analyze the data like which process for a specific run had max cpu usage.Using jhat didn't helped much,plz enlighten.
@blong: Good spot, fixed the broken links.
@SanjayT.Sharma , great, thanks so much! Downvote converted to upvote :)
|
3

One way to profile an "already started" JVM is to aggregate multiple jstacks taken over time.

You can for instance parse and display them as a FlameGraph (see details at the various answers for that link, I won't redundantly include them here).

Improve this answer

Comments

1

You can run most commercial profilers remotely so an agent is run on the server then connect to that agent through a client on your dev machine. My absolute favorite profiler is JProfiler. It's fairly reasonable purchase, and it's very stable (which not all commercial profilers that's true).

http://www.ej-technologies.com/products/jprofiler/overview.html

Other commercial profilers that are stable, but not my favorite are YourKIT.

http://www.yourkit.com/

Those smaller vendors make good tools. These tools will provide you tons of information about method timings, memory use, GC, etc. Much more than jconsole.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.