Quick Cache

Lightweight and high performance concurrent cache optimized for low cache overhead.

• Small overhead compared to a concurrent hash table
• Scan resistant and high hit rate caching policy (S3-FIFO)
• User defined weight per item
• Scales well with the number of threads
• Atomic operations with get_or_insert and get_value_or_guard functions
• Atomic async operations with get_or_insert_async and get_value_or_guard_async functions
• Supports item pinning
• Handles zero weight items efficiently
• Allows for customizable lifecycle hooks (e.g. can be used to implement eviction listeners)
• Doesn't use background threads
• Only trivially verifiable usages of unsafe
• Small dependency tree

The implementation is optimized for use cases where the cache access times and overhead can add up to be a significant cost. Features like: time to live, iterators, event listeners and others; are partially or not implemented in Quick Cache. If you need these features you may want to take a look at the Moka crate.

Examples

Basic usage

use quick_cache::unsync::Cache;

fn main() {
    let mut cache = Cache::new(5);
    cache.insert("square", "blue");
    cache.insert("circle", "black");
    assert_eq!(*cache.get(&"square").unwrap(), "blue");
    assert_eq!(*cache.get(&"circle").unwrap(), "black");
}

A cache with custom item weights. In this case according to the string length of the value.

use quick_cache::{Weighter, sync::Cache};

#[derive(Clone)]
struct StringWeighter;

impl Weighter<u64, String> for StringWeighter {
    fn weight(&self, _key: &u64, val: &String) -> u64 {
        // Be cautions out about zero weights!
        val.len() as u64
    }
}

fn main() {
    let cache = Cache::with_weighter(100, 100_000, StringWeighter);
    cache.insert(1, "1".to_string());
    cache.insert(54, "54".to_string());
    cache.insert(1000, "1000".to_string());
    assert_eq!(cache.get(&1000).unwrap(), "1000");
}

Using the Equivalent trait for complex keys

use quick_cache::{sync::Cache, Equivalent};

#[derive(Debug, Hash)]
pub struct Pair<A, B>(pub A, pub B);

impl<A, B, C, D> Equivalent<(C, D)> for Pair<A, B>
where
    A: PartialEq<C>,
    B: PartialEq<D>,
{
    fn equivalent(&self, rhs: &(C, D)) -> bool {
        self.0 == rhs.0 && self.1 == rhs.1
    }
}

fn main() {
    let cache: Cache<(String, i32), String> = Cache::new(5);
    cache.insert(("square".to_string(), 2022), "blue".to_string());
    cache.insert(("square".to_string(), 2023), "black".to_string());
    assert_eq!(cache.get(&Pair("square", 2022)).unwrap(), "blue");
}

Benchmarks

Since this crate is performance oriented it needs some comparisons. That said, benchmarks can be misleading so take everything with a pinch of salt.

Benchmarks performed with mokabench in a x64 Linux OS + Intel i9-12900H CPU.

Trace 1 (S3 from the Arc paper)

Trace 2 (DS1 from the Arc paper)

Trace 3 (SPC1 from the Arc paper)

Notes:

• LRU+Mutex: hashlink crate + std::sync::Mutex
• SPC1 only includes 1 client to save space, it follows the same trend as other traces.
• Other Moka variants not included to save space, they perform similarly or worse than Mini Moka. Full results

References

• CLOCK-Pro: An Effective Improvement of the CLOCK Replacement (pdf)
• FIFO queues are all you need for cache eviction (pdf)
• Caffeine (source code)
• Cache2k (source code)

License

This project is licensed under the MIT license.