Always measure performance in the context of your application. Measuring gives you a leg up over people who are too smart to measure. This page is intended to help you select the right tool for the job, and fine tune to get the best possible results.

Cache performance can be measured using 3 key metrics:

• Hit Rate
• Throughput
• Latency

As BenchmarkDotNet gains popularity, a lot of emphasis is placed on microbenchmark latency. Microbenchmarks can be a good predictor of overall application performance for a narrow set of use cases (like if a cache is invoked in at very high frequency and the hit rate is high, or the value factory executes very quickly). However, in most cases hit rate is the critical metric to optimize. In a multithreaded application throughput is the next most important concern.

Caches trade an increase in memory for a decrease in computation or latency. The larger the cache, the more items can be stored. When the cache is full, the cache replacement policy must decide which entry to discard when admitting a new entry. The two factors that determine hit rate are cache size and replacement policy.

Both LRU and LFU policies both provide excellent hit rate across different workloads. Choose a policy based on the characteristics of your workload and then tune cache size to get the best tradeoff between memory consumption and hit rate.

The LRU and LFU policies each make different algorithmic tradeoffs, throughput depends on:

• Workload, which can be read (hit), write (miss) or update heavy
• Number of concurrent threads
• CPU architecture

Throughput measurements are provided here. Read throughput is in most cases the most important metric, because cache misses and updates will typically be bounded by fetching or computing the data that is being cached.

Pay attention to extension points and avoid the introduction of bottlenecks that might limit throughput. For example, a global lock inside the cache value factory, expiry calculator or event handler logic. LRU metrics slightly reduce concurrent throughput.

Below are a few simple details that can affect cache lookup latency.

The type of the cache key, and its associated Equals and GetHashCode methods can influence lookup cache speed, and the distribution of the GetHashCode method will influence the number of collisions in the underlying hash table. You can specify an implementation of IEqualityComparer<K> at cache construction.

For example, if the cache key is a string, prefer ordinal to culture sensitive string comparison. The HashCode.Combine method used when autogenerating equality methods produces a high quality hash, but can be much slower than handwriting a simpler equivalent.

All cache features cost performance. This library has been designed such that disabling features eliminates cost. Therefore, try to use only the required cache features via the cache builder methods. Time-based expiry, atomic and scoped values incur a slight penalty for lookup latency. Events incur an event args heap allocation when an event handler is registered.

Lookup latency is measured for different features here.

IExpiryCalculator methods can be called at very high frequency. To get the lowest latency and highest throughput, avoid heap allocations and minimize computation within the GetExpireAfter* methods.

An efficient expire after write calculator is show below. Since expiry time is fixed it can be calculated once up front at initialization instead of per create/read/update call, avoiding floating point multiplication on the hot path.

public class ExpireAfterWrite : IExpiryCalculator<string, int>
{
    private readonly Duration timeToExpire = Duration.FromMinutes(5);

    public Duration GetExpireAfterCreate(string key, int value) => timeToExpire;
    public Duration GetExpireAfterRead(string key, int value, Duration current) => current;
    public Duration GetExpireAfterUpdate(string key, int value, Duration current) => timeToExpire;
}