Everyone knows that it happens many times that a JVM runs into an “Out of Memory” condition. If it happens one time, no problem. Usually a simple restart of the application should solve the problem.
But if you continuesly spending time in restarting, it is time to analyze why the application is running out of memory.
Then it is time to take a heapdump.
What is a heapdump?
A heapdump is a memory screenshot of a java process. The snapshot contains information about the Java objects and classes in the heap at the moment the snapshot is triggered. Read more about it on that IBM Knowledge Page.
The classic way of capturing a heapdump
Everyone who is working in the middleware area or in the area as I do, should know about the classic way to capture a heapdump. Jmap is the tool which gives you the possibility with one command to take the dump.
jmap -dump:format=b,file=/tmp/heapdump.bin $PID
The problem with that way is that it can take several minutes, even hours, to finish. It depends of the size of the configured java heap size.
I had to use jmap for one of my last captures. The JVM had a heap size configured of 10G. The process took me 7 hours. 7 hours!!!
The real problem is not the 7 hours. The problem is that in that number of hours the JVM cannot be restarted nor can be used by the users of the application. It is blocked until the heapdump has been captured.
But there is a better way!
Capture a heapdump without downtime
Next to the jmap tool, we can use the GNU Debugger (gdb) to get our heapdump. The GNU Project debugger, allows you to see what is going on `inside’ another program while it executes — or what another program was doing at the moment it crashed. Check out more about it on their page.
With the gdb tool, the heapdump will be taken in 2 steps:
- Create a core file of the Java Process
- Convert the core file to a heapdump file
Here comes the good part. The 1st step takes about 20-30 seconds. The 2nd part can be done outside of the machine and you can restart your JVM after the 1st step is done.
How to take the first step:
- Start the GNU Debugger with your java process id:
gdb –pid=$PID - Then execute the following 3 commands:
(gdb) gcore /tmp/jvm.core(gdb) detach(gdb) quit
With the gcore command you created the java core file.
The process in summary as mentioned takes about 20-30 seconds. I recommend you to do it as fast as you can.
As soon as you are done with the first step you can restart your application to get it up and running again.
In the second step we are going to convert the java core file to a java heap dump file.
For the second step there are 3 important things to mention. First to know is that the converting will be done with the classic jmap tool.
Second, you need to pay attention that you use the jmap tool which runs on the same java version as the JVM, you took the java core file.
Third, the convertion process can be done also on another machine (e.g. on test environment) to not affect the production machine in any way (cpu, etc.).
Convert the java core file to a heapdump with following command:
jmap -dump:format=b,file=jvm.hprof /usr/bin/java /tmp/jvm.core
The heapdump at the end has the name jvm.hprof. /usr/bin/java need to be replaced with your java version.
The process can take also several time like the classic way, but your application is already up and running.
Test this process few times in your test environment to get a feeling how it works.
Summary
There are probably other better ways to take a heapdump, but the way with the gnu debugger helped me to save some time.
If you have questions, don’t hesitate to contact me.
If you want to know more about it, join my Slack Workspace or send me an email.
Stay up-to-date and subscribe to my Newsletter!
Thank you for this. Quickest and least disruptive method of getting a heap dump