TLDR;

OpenJ9 and IBM J9 are a different JVM implementation from the default Oracle HotSpot JVM. With the modern adoptopenjdk pre-made Docker images it is easy to swap and test different combinations and pick the right JVM for you.

The rumours seem to be true, OpenJ9 seems to blow HotSpot away on memory usage. HotSpot seems to have the edge CPU-wise.

OpenJ9

In the Java world most people are familiar with OpenJDK. This is a complete JDK implementation including the HotSpot JVM engine. Not a lot of developers know or try alternatives to HotSpot. Asking around some colleagues remembered the name JRockit, nobody mentioned IBM J9 and/or Eclipse OpenJ9.

I’ve read that OpenJ9 is very good with memory management and is tailered for usage in the cloud/in containers. OpenJ9 is an independent implementation of the JVM. It’s origins are IBM’s Java SDK/IBM J9 which can trace its history back to OTI Technologies Envy Smalltalk (thanks Dan Heidinga!).

With the current rise in microservice usage (and most services are not so micro in Java). I recon this could become a hot topic again!

Testing

Before the Docker-era it was relatively hard to compare different JVMs, versions. You needed to download, install, script and run everything. But now a lot of pre-made images are available online.

Here is my idea on how to test the JVMs:

1. Create a simple Spring Boot application
2. Start the application in various Docker Images
3. Measure memory usage after startup and GC
4. Measure the time it takes to run a small CPU-intensive test

This is by no means a thorough test or benchmark, but it should give us a basic idea of what we can expect from the virtual machines.

Spring Boot application

The Spring Boot application I created contains the following endpoints:

1. A REST endpoint that calls the GC (trying to make it fair)
2. A REST endpoint that creates 1000 large random arrays and sorts them, returns the runtime (in ms)

Here is the listing of the CPU-test:

@RestController
public class LoadTestController {

    @RequestMapping("/loadtest")
    public LoadTestResult loadtest() {

        long before = System.currentTimeMillis();

        Random random = new Random();

        for(int i = 0; i < 1000; i++) {
            long[] data = new long[1000000];
            for(int l = 0; l < data.length; l++) {
                data[l] = random.nextLong();
            }
            Arrays.sort(data);
        }

        return new LoadTestResult(System.currentTimeMillis() - before);
    }
}

Again, we can argue endlessly about if this test makes sense and is even remotely relevant… but still it should give us some basic idea of what kind of performance we can expect. If the rumoured memory improvements are true, might there be a performance hit? Is there a performance trade-off?

JVM images

I’ve decided to test the following images.

First we have the (slim) openjdk images for 8/9/10/11:

• openjdk:8-slim
• openjdk:9-slim
• openjdk:10-slim
• openjdk:11-slim

Next there are the adoptopenjdk images for 8/9/10:

• adoptopenjdk/openjdk8
• adoptopenjdk/openjdk9
• adoptopenjdk/openjdk10

Then we have OpenJ9, again provided by adoptopenjdk for 8, 9 and a nightly build of 9 (see my previous blogpost):

• adoptopenjdk/openjdk8-openj9
• adoptopenjdk/openjdk9-openj9
• adoptopenjdk/openjdk9-openj9:nightly

And I decided to include IBM’s own J9 image as well:

• ibmcom/ibmjava:8-jre

Testing with Docker

After building my Spring Boot application I launched each Docker image using the following command:

docker run -it -v /Projects/temp/spring-boot-example:/app/spring-boot-example -p 8080:8080 IMAGE_NAME /bin/bash

I’m mapping my “spring-boot-example” project folder to “/apps/spring-boot-example” so I can start the JAR file inside the container. Also I’m forwarding port 8080 back to my host so I can call the endpoints.

Next, inside the container, I launch the Spring Boot application:

java -jar /app/spring-boot-example/target/spring-boot-example-0.0.1-SNAPSHOT.jar

After waiting a bit, calling the endpoints a couple of times and performing a GC I measured the memory usage.

After that I called the “/loadtest” endpoint containing the array-sorting test and waited for the results.

Memory benchmark

Here are the results of the memory used by the simple Spring Boot application:

At first you can see that the memory usage for Java 8 is much higher than for Java 9 and 10, good!

But the biggest shock is how much less memory OpenJ9 and J9 are using, almost 4x less memory if you compare Java 8 with OpenJ9. I’m amazed, how does this even work? Now we can almost call our Spring Boot service micro!

I’ve also experimented with running some production Spring Boot code (not just simple examples) and here I’ve seen improvements up to 40-50% decrease in memory usage.

CPU benchmark

Online I’ve read that OpenJ9 isn’t as good as HotSpot if you look at CPU intensive tasks. That is why I created a small test for this as well.

1000 arrays with 1000000 random long values being sorted. This takes around 100 seconds, this should give the JVM enough time to adjust and optimize. I’ve called the benchmark twice for each tested image. I’ve recorded the second time trying to eliminate warmup times.

In the chart we can see that indeed the J9 and OpenJ9 images are slower, not by much max 18%. It seems for this particular testcase Java 8 beats most Java 9 implementations (except coupled with OpenJ9).

What’s next?

My current project has a lot more memory issues than CPU issues on production (frequently running out of memory while having 1-2% CPU usage). We are definitely thinking about switching to OpenJ9 in the near future!

We did already encounter some issues during testing:

1. Hessian: (binary protocol) has a build-in assumption that System.identityHashCode always returns a positive number. For HotSpot this is true but OpenJ9/J9 can also return negative numbers. This is an open issue and the Hessian project hasn’t fixed this in a couple of years, seems to be dead? Our solution is to move away from Hessian altogether
2. Instana: We love our monitoring tool Instana, but it had some problems connecting their agent to OpenJ9/J9. Luckily the people at Instana helped us identify a bug and a fix should be published today (and is automatically updated, w00t!)

Open questions I haven’t looked in to:

1. Can you still get/use jmap/hprof information etc in OpenJ9?
2. How will it hold up during longer production runs?
3. Will we find other weird bugs? It feels tricky…

Have you tried OpenJ9/J9? Let me know in the comments.

Is there anything else you’d love to see tested? The best way to contact me is to send me a tweet royvanrijn.