JVM Profilers

Identify and resolve performance bottlenecks, memory leaks, and threading issues

Overview

This tutorial aims to enable you to identify and resolve problematic patterns in memory and CPU usage in Java and Scala code using JProfiler and JVisualVM.

Profilers in the Market

One survey published 2015-11 as pdf presented these market share numbers:

Comparison for Mac (named license with two years of support):

Product	Date	File Name	MB	$USD
JProfiler	2017-01-21	jprofiler_macos_10_0.dmg	148 MB	$698
JProfiler	2016-06-21	jprofiler_macos_9_2.dmg	126 MB	$698
JVisualVM	2016	GitHub 1.3.9	13 MB	free
Yourkit	2016.02	yjp-2016.02-b43-mac	8.7 MB	$649
jClarity Censum	-	-	- MB	$?

JVisualVM

VisualVM has the largest market share because perhaps it is free and is bundled in the JDK since JDK 6 update 7.

http://docs.oracle.com/javase/7/docs/technotes/guides/visualvm/profiler.html

The bleeding-edge version at
https://visualvm.github.io/download.html
is actively maintained by two developers in Prague, the Czech Republic.

VIDEO:

Introduction to Java Visual VM from 2013.

Philip Starritt has created videos on VisualVM:

Launcher Setup in Eclipse. 9 July 2017.
https://visualvm.github.io/idesupport.html
Java Heap Dump Analysis of live running app. 9 July 2017
OutOfMemoryError Java Heap Space Fix - Heap Dump Analysis 12 July 2017
Out of Memory - Head Dump Analysis
- https://www.youtube.com/watch?v=dUQqmnmCBbg Using jvisualvm to find Heap Problems
- https://www.youtube.com/watch?v=pRwvx4QpUfo Java VisualVM - Introduction
- https://www.youtube.com/watch?v=z8n7Bg7-A4I
- https://www.youtube.com/watch?v=1kGt1ctY-a0 Introduction to Java VisualVM Java GC Monitoring with JVisualVM by Rohit Dhall

JProfiler Install & License

Download from
https://www.ej-technologies.com/download/jprofiler/files
for your platform.
PROTIP: While you wait for the download, identify the version of IDE you’ll be using with JProfiler. Most developers use integrations with their IDE IntelliJ IDEA, Eclipse, NetBeans, Oracle JDeveloper.

Also, get a license at
https://www.ej-technologies.com/buy/jprofiler/select

$698 with a year of support.

After download
Open the JProfiler installer.
On a Mac, drag the JProfiler icon to drop on the Applications folder.
Exit the installer dialog.
Delete (Move to Trash) the installer file to save disk space.
Use Finder to initiate JProfiler.
The first time it opens, JProfiler enters Setup. Click Next.
Click Next to Evaluate for 10 days.

IDE
Optionally: Click “[Select an IDE]” if you have one and click Integrate.

See http://resources.ej-technologies.com/jprofiler/help/doc/ide/$folder$.html

An example of IDE configuration is http://www.andrejkoelewijn.com/blog/
Click Next, Finish.

JProfiler is started up automatically.
Check “Don’t show this dialog again” and Cancel.

See Sample App Leak Memory

Just to walk through JProfiler’s UI, let’s look at a program provided by JProfiler to behave badly.

Click to select the “Animated Bezier Curve demo” session which JProfile provides to intentionally leak memory not garbage collected: Its code contains a “LeakMap” object.
Click Start.
If an Evaluation version dialog appears, click “Evaluate”.

The Session Starup settings dialog appears.

Help and Docs
Click the Help button for context-sensitive help.

PROTIP: This is the best way to access specific documentation. So if you want to review documentation on another process, open the 303 page pdf JProfiler Manual at
http://resources.ej-technologies.com/jprofiler/help/doc/help.pdf. It presents same content as Online Help.
To continue working, click on the app dialog partially hidden by the Help dialog.

Videos

Among videos created by Ingo Kegel (CTO of the company) is this from 2012 for an older version of the program.

TODO: Recreate video using new verion of JProfiler, and run Scala.

Startup Settings

A terminal window is opened for the demo process and the main window of JProfiler displays profiling metrics.

PROTIP: Probes collect measurements.
Click OK to accept the Startup Settings.

A run is begun automatically.

Start and Stop GUI

PROTIP: In the Terminal window opened automatically, notice “JVMTI version 1.1 detected” refers to the technology the JVM provides to enable JProfiler to obtain instrumentation data.
Click on Session Settings at the top menu:

Notice we are launching a new JVM instance and using the JVM indicated referencing the relative CLASSPATH indicated.

Session with Attach

Alternately, blog entry How to monitor Kafka apps with JProfiler shows this setting screen for attaching a running program:

The app is started with this command
```
java -cp `mapr classpath`:nyse/nyse-taq-streaming-1.0-jar-with-dependencies.jar -agentpath:/home/mapr/jprofiler9/bin/linux-x64/libjprofilerti.so=port=11002 com.example.Run consumer /user/iandow/mystream:mytopic
```
Remote access

It’s “jpenable” which loads the profiling agent and makes it possible to connect with a remote session from another computer.

PROTIP: The profiling agent and JProfiler GUI communicate with each other through a socket. By default, the profiling agent listens on port 8849. Many networks block traffic from such a port.

This video shows how to setup a SSH Tunnel.
Click OK to dismiss the dialog.

Watch it run
Click on Telemetries category Overview.
Among processes, click on the Demo to expose the app under test.
Check “Leak Memory” on the BezierAnim app window.

Programs that leak memory will show an upward trend in thread memory usage.

PROTIP: Many memory leaks accumulate gradually over time, so may require a long run to identify conclusively.
Click the zoom in and out icons at the lower right corner.

Each GC Activity spike reduces the amount of Memory used (shown in blue) as a portion of memory allocated (shown as the top of the green block).
Click on Live memory category All Objects view to sort objects by the amount each uses memory.

Diffs in memory usage

PROTIP: Most items are generic objects such as “java.lang.Long” which uses a lot of memory because many objects use it. We want to see the objects creating additional memory usage over time.
Click the Mark Current button at the right of the top ribbon to set the baseline for differences to be displayed.
Righ-Click on the high-level object defined by a custom name (“java.awt.geom.GeneralPath”) to select “Show selection in Heap Walker”.

“Take Heap Snapshot for Selection”.
Click OK to the Heap Snapshot Options with “Select recorded objects”.
Click “References” to activate the reference graph view.
Select “Cumulated outgoing references”.
Click the gray arrows to expand items at each level.
Right-click on object to select “Show paths to GC Root”.
Click OK to see Single Root.
Scroll horizontally to the left in the chain.

Biggest Objects
Click on “Biggest Objects” icon.
Expand the tree by clicking gray arrows under the app “bezier.BezierAdmin”

Show source code
Right-click to select “Use Retain Items”.
Check the “Classes” radio button, then OK.

BLAH: Where is this in v9.2?
Check the “Allocations” radio button.

QUESTION: How do we know leakMap is within this class?
Select “Cumulated allocation tree”.
Click OK.
Right-click to select “Show Source”.

Save Snapshot

PROTIP: Snapshots can be taken only while the program is running.
Click the Take Snapshot camera icon on the ribbon.

Alternately,
click the HPROF heap snapshot.
Click Save Snapshot.

Offline

Use the integration wizard to activate this by appending to the -agentpath VM parameter
```
offline,config=[config file],id=[id]
```

Root Causes

Memory leaks was a small part of all issues according to this report in 2015

JProfiler Origin Tracker

Although Chrome announced it is doing away with Extensions, JProfiler Origin Tracker Chrome Browser Extension can still be useful.

Strategy

JProfiler has several “modes” it can run.

PROTIP: Take a “top-down” approach to analyzing a Java program.

Ideally, you can attach JProfiler to a running server.
It can be time consuming to create load scripts that provide the coverage of code needed. So it is often useful to see running production servers.
Begin using JProfiler by using the “Profile at startup” mode.
Understand a programs’s major dependencies.
Identify the biggest objects.
Identify the rate of memory use.
Instrument classes by having JProfiler retransform the program.
Run JProfile in offline mode to save snapshots when you need to analyze several JVMs running at the same time. Then have an Ant program task to compare the snapshots.

The difference run-time data areas are illustrated here: jvm architecture-650 From Java8 virtual machine architecture by Ranjith ramachandran

To manage the Method Area holding classes defined in source code,
use Memory Profiling

NOTE: In Java8, metaspace setting takes the place of the MaxPermSize parameter that specifies space of the method area holding classes.

To manage the “thrashing” of PC (Program Counters) pointing to the next instruction to be executed in each thread,
use CPU profiling.
To manage the Java Stacks containing stack frames for each chain of methods in each thread “Stack Overflow” errors refer to this area,
use Thread Profiling.
Native Method Stacks per thread
To manage the Heap holding objects instantiated for use in JVM (garbage collected),
use the Heap Walker.

ReTransform to Instrument Code

Other Sample Programs

http://www.javamonamour.org/2014/01/how-to-demo-garbage-collection-jconsole.html discussed in https://www.youtube.com/watch?v=Bjv_9pKiALQ

The company behind JProfile, ej-technologies, is based in Munich, Germany.

There has been no activity on the @JProfiler Twitter account.

There is no JProfiler group on LinkedIn, Facebook, etc.

JMC (Java Mission Control)

Java Mission Control monitors the JVM in real-time.

Java Mission Control requires a commercial license for use in production.

JMC has an Experimental Plug-in for Eclipse.

PROTIP: JMC comes bundled with the HotSpot JVM since Oracle JDK 7 Update 40 (7u40). So in a Terminal:

Use the abbreviation for Java Mission Control to load it:

jmc

A welcome screen titled “Mission Control 5.5” appears.

Java Mission Control User’s Guide 5.5
Optional: Click the “A” at the upper-right corner to change font size.
Click the program icon

A JMX console appears.
Open another terminal window to bring up the app under observation:
```
java -XX:+UnlockCommercialFeatures -XX:+FlightRecorder LoadAndDeadlock
```
NOTE: Although Oracle has built in the recording engine into the JVM run-time, it needs to be enabled in JVM start-up switches.

https://docs.oracle.com/javacomponents/index.html

Oracle also provides optional tools for heap dump analysis and DTrace recording.

Java SE Suite provides Deterministic GC.

YourKit

YourKit from Germany has profilers for both JVM and .NET.

Download from https://www.yourkit.com/java/profiler/download/
In https://www.yourkit.com/purchase/ license is $649 with a year of support.

Censum

Censum from JClarity.com (by Martijn Verburg based in London) is available as both a local VM log reader (for $749/year) and as a SaaS web app.

Friends of jClarity mailing list / forum!

https://blog.codecentric.de/en/2017/09/jvm-fire-using-flame-graphs-analyse-performance/

CTO/CEO Martijn Verburg (@karianna) speaks at VIDEO: The Diabolical Developer’s Guide to Performance Tuning introduces the Performance Diagnostic Methodology (PDM).

Tapiki

Takipi is focused on Scala performance tuning. It is the only tool I know of that shows numbers associated with Scala code. It’s free up to 2 servers for Scala.

What to Look For

https://shipilev.net/blog/2016/close-encounters-of-jmm-kind/

As a comparison, take a look at the profiler in Visual Studio for C# Xamarin code.

Wait, there’s more. Click one of these …

This article is one of a series about tuning and performance:

Wilson Mar

JVM Profilers

Profilers in the Market

JVisualVM

JProfiler Install & License

After download

IDE

See Sample App Leak Memory

Help and Docs

Videos

Startup Settings

Start and Stop GUI

Session with Attach

Remote access

Watch it run

Diffs in memory usage

Biggest Objects

Show source code

Save Snapshot

Offline