Caffeine Cache: A High Performance Caching Library

The goal of this posting is to learn about caffeine cache and how to use it.

The previous post covered Spring Cache. (It was written only in Korean so far, but I am going to translate it into English.) In the content, the cache manager is required to use cache in spring.

In this post, we’re going to explore the Caffeine cache, which is performance-friendly among local caches. In the Github repository Caffeine Cache, they describe the Caffeine as “a high performance, near-optimal caching library”. It also presents performance differences from various caches, backed by the terms high performance and optimal caching.

So, let’s compare a variety of caches including Ehcache, Guava, and ConcurrentHashMap.

Caffeine versus …

Caffeine VS EhCache

Ehcache supports many features such as multi-level cache, distributed cache, and cache listener. Caffeine cache has higher cache performance than Ehcache and uses a better cache removal strategy.

The caffeine cache uses “The Window TinyLfu” removal strategy, which provides an almost optimal hit ratio.

Benchmarks

Caffeine cache performance was proved by benchmark tests as shown in the pictures below.

⚠️ The test assumes that there is no limit on cache capacity and cached values always calculate the same way.

As you can see in the above pictures, throughput per Ops/s(Operations per seconds) shows a very high performance compared to other caches.

Functions

The actual code that composes a caffeine cache can be found in the next post, but we can now explore what its function is.

You can refer to Caffeine Wiki which describes most of the functions and the following is organizing the content.

📌 Population Strategy

Caffeine provides four types of population strategies: manual, loading synchronously, and asynchronous variants.

#1. Manual

The manual cache is a very simple and basic way to use caffeine cache as follows.

Cache<K, V> cache = Caffeine.newBuilder().<setSomethingUp>().build();

The Cache interface allows for explicit control of retrieving, updating, and invalidating entries.

#2. Loading

A LoadingCache is a Cache built with an attached CacheLoader.

LoadingCache<K, V> cache = Caffeine.newBuilder().<setSomethingUp>().build(CacheLoader<> loader);

It needs a definition of a way to load Caches in param at the last chaining build method.

#3. Asynchronous (Manual)

An AsyncCache is a Cache variant that computes entries on an Executor and returns a CompletableFuture.

AsyncLoadingCache<K, V> cache = Caffeine.newBuilder().buildAsync();

The default Executor is ForkJoinPool.commonPool() and can be overridden via Caffeine.executor(Executor).

#4. Asynchronous Loading

An AsyncLoadingCache is a Cache characterized above LoadingCache and AsyncCache. It is a AsyncCache built with AsyncCacheLoader.

AsyncLoadingCache<K, V> cache = Caffeine.newBuilder().buildAsync(AsyncCacheLoader);

A CacheLoader should be supplied when the computation is best expressed in a synchronous fashion. Alternatively, a AsyncCacheLoader should be supplied when the computation is expressed asynchronously and returns a CompletableFuture.

📌 Eviction

There are three types of evicted rules: based on size, time, and reference. The following refers to this page.

The caffeine eviction policy uses “Window TinyLfu” because it shows a high hit ratio as shown above and a rather small installation space.

#1. Size-based

You can use size-based eviction to set maximumSize() method.

Caffeine.newBuilder().maximumSize(long).build();

A Size-based eviction will try to evict entries that have not been used recently, entries that are frequently unused when the entries surpass a certain size.

#2. Time-based

A Time-based eviction is supplied in three types.

✔ expireAfterAccess(long, TimeUnit)

Expire entries after the specified duration has passed since the last read or written.

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterAccess(5, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

✔ expireAfterWrite(long, TimeUnit)

Expire entries after the specified duration has passed since created or the last replacement.

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

✔ expireAfter(Expiry)

Expires entries after the variable duration has passed.

#3. Reference-based

Caffeine allows you to access garbage collection of entries. This allows entries to be garbage collected if there are no other strong or soft references.

Note that weak and soft value references are not supported by AsyncCache.

✔️ using Weak References

Caffeine.weakKeys() and Caffeine.weakValues() store each keys and values respectively using weak references.

// Evict when neither the key nor value are strongly reachable
LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .weakKeys()
    .weakValues()
    .build(key -> createExpensiveGraph(key));

This allows entries to be garbage-collected if there are no other strong references to the keys or values.

Since garbage collection depends only on identity equality, this causes the whole cache to use identity ( ==) equality to compare keys or values, instead of equals().

✔️ using Soft References

Caffeine.softValues() stores values using soft references. It is recommended to use the more predictable maximum cache size instead because of the performance implications of using soft references.

// Evict when the garbage collector needs to free memoryLoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .softValues()
    .build(key -> createExpensiveGraph(key));

Use of softValues() will cause values to be compared using identity (==) equality instead of equals(). Softly referenced objects are garbage-collected in a globally least-recently-used manner, in response to memory demand.