#lru-cache #cache #lru

quick_cache

Lightweight and high performance concurrent cache

13 unstable releases (4 breaking)

0.5.1 Apr 7, 2024
0.4.2 Mar 24, 2024
0.4.0 Aug 29, 2023
0.3.0 Apr 29, 2023
0.1.1 Aug 16, 2022

#7 in Caching

Download history 6745/week @ 2024-01-03 7509/week @ 2024-01-10 12120/week @ 2024-01-17 14799/week @ 2024-01-24 11543/week @ 2024-01-31 13273/week @ 2024-02-07 14514/week @ 2024-02-14 13010/week @ 2024-02-21 10145/week @ 2024-02-28 13478/week @ 2024-03-06 13051/week @ 2024-03-13 14069/week @ 2024-03-20 12609/week @ 2024-03-27 13472/week @ 2024-04-03 18117/week @ 2024-04-10 15400/week @ 2024-04-17

62,505 downloads per month
Used in 72 crates (18 directly)

MIT license

135KB
3K SLoC

Quick Cache

Crates.io Docs CI

Lightweight and high performance concurrent cache optimized for low cache overhead.

  • Small overhead compared to a concurrent hash table
  • Scan resistent and high hit rate caching policy
  • User defined weight per item
  • Scales well with the number of threads
  • Atomic operations with get_or_insert and get_value_or_guard functions
  • Atomic async operations with get_or_insert_async and get_value_or_guard_async functions
  • Doesn't use background threads
  • Only trivially verifiable usages of unsafe
  • Small dependency tree

The implementation is optimized for use cases where the cache access times and overhead can add up to be a significant cost. Features like: time to live, iterators, event listeners and others; are not currently implemented in Quick Cache. If you need these features you may want to take a look at the Moka crate.

Examples

Basic usage

use quick_cache::unsync::Cache;

fn main() {
    let cache = Cache::new(5);
    cache.insert("square", "blue");
    cache.insert("circle", "black");
    assert_eq!(*cache.get(&"square").unwrap(), "blue");
    assert_eq!(*cache.get(&"circle").unwrap(), "black");
}

A cache with custom item weights. In this case according to the string length of the value.

use quick_cache::{Weighter, sync::Cache};

#[derive(Clone)]
struct StringWeighter;

impl Weighter<u64, String> for StringWeighter {
    fn weight(&self, _key: &u64, val: &String) -> u32 {
        // Be cautions out about zero weights!
        val.len().clamp(1, u32::MAX as usize) as u32
    }
}

fn main() {
    let cache = Cache::with_weighter(100, 100_000, StringWeighter);
    cache.insert(1, "1".to_string());
    cache.insert(54, "54".to_string());
    cache.insert(1000, "1000".to_string());
    assert_eq!(cache.get(&1000).unwrap(), "1000");
}

Using the Equivalent trait for complex keys

use quick_cache::{sync::Cache, Equivalent};

#[derive(Debug, Hash)]
pub struct Pair<A, B>(pub A, pub B);

impl<A, B, C, D> Equivalent<(C, D)> for Pair<A, B>
where
    A: PartialEq<C>,
    B: PartialEq<D>,
{
    fn equivalent(&self, rhs: &(C, D)) -> bool {
        self.0 == rhs.0 && self.1 == rhs.1
    }
}

fn main() {
    let cache: Cache<(String, i32), String> = Cache::new(5);
    cache.insert(("square".to_string(), 2022), "blue".to_string());
    cache.insert(("square".to_string(), 2023), "black".to_string());
    assert_eq!(cache.get(&Pair("square", 2022)).unwrap(), "blue");
}

Benchmarks

Since this crate is performance oriented it needs some comparisons. That said, benchmarks can be misleading so take everything with a pinch of salt.

Benchmarks performed with mokabench in a x64 Linux OS + Intel i9-12900H CPU.

Trace 1 (S3 from the Arc paper)

Cache Max Capacity Clients Hit Ratio(%) Elapsed time(s)
HashLink (LRU w/ Mutex) 100000 1 2.327 2.479
HashLink (LRU w/ Mutex) 100000 3 2.327 6.171
HashLink (LRU w/ Mutex) 100000 6 2.328 12.467
QuickCache Sync Cache 100000 1 12.764 2.465
QuickCache Sync Cache 100000 3 12.765 1.387
QuickCache Sync Cache 100000 6 12.764 0.835
Mini Moka Sync Cache 100000 1 10.436 10.473
Mini Moka Sync Cache 100000 3 10.541 8.439
Mini Moka Sync Cache 100000 6 10.366 8.008
HashLink (LRU w/ Mutex) 400000 1 12.039 2.956
HashLink (LRU w/ Mutex) 400000 3 12.041 6.939
HashLink (LRU w/ Mutex) 400000 6 12.043 13.503
QuickCache Sync Cache 400000 1 42.044 3.250
QuickCache Sync Cache 400000 3 42.044 1.299
QuickCache Sync Cache 400000 6 42.044 0.777
Mini Moka Sync Cache 400000 1 42.544 9.671
Mini Moka Sync Cache 400000 3 41.525 7.052
Mini Moka Sync Cache 400000 6 41.007 6.618
HashLink (LRU w/ Mutex) 800000 1 56.600 3.221
HashLink (LRU w/ Mutex) 800000 3 56.603 5.921
HashLink (LRU w/ Mutex) 800000 6 56.605 12.768
QuickCache Sync Cache 800000 1 66.801 3.980
QuickCache Sync Cache 800000 3 66.802 1.418
QuickCache Sync Cache 800000 6 66.803 0.798
Mini Moka Sync Cache 800000 1 70.304 10.613
Mini Moka Sync Cache 800000 3 68.054 4.754
Mini Moka Sync Cache 800000 6 66.288 4.137

Trace 2 (DS1 from the Arc paper)

Cache Max Capacity Clients Hit Ratio(%) Elapsed time(s)
HashLink (LRU w/ Mutex) 1000000 1 3.086 10.097
HashLink (LRU w/ Mutex) 1000000 3 3.085 22.443
HashLink (LRU w/ Mutex) 1000000 6 3.082 41.351
QuickCache Sync Cache 1000000 1 15.052 9.426
QuickCache Sync Cache 1000000 3 15.059 4.139
QuickCache Sync Cache 1000000 6 15.078 2.467
Mini Moka Sync Cache 1000000 1 14.910 28.974
Mini Moka Sync Cache 1000000 3 13.257 24.681
Mini Moka Sync Cache 1000000 6 13.518 22.759
HashLink (LRU w/ Mutex) 4000000 1 20.245 9.863
HashLink (LRU w/ Mutex) 4000000 3 20.250 19.418
HashLink (LRU w/ Mutex) 4000000 6 20.259 36.932
QuickCache Sync Cache 4000000 1 44.565 9.083
QuickCache Sync Cache 4000000 3 44.574 4.272
QuickCache Sync Cache 4000000 6 44.569 2.449
Mini Moka Sync Cache 4000000 1 45.398 31.108
Mini Moka Sync Cache 4000000 3 44.236 21.381
Mini Moka Sync Cache 4000000 6 41.723 18.719
HashLink (LRU w/ Mutex) 8000000 1 43.035 9.751
HashLink (LRU w/ Mutex) 8000000 3 43.034 16.554
HashLink (LRU w/ Mutex) 8000000 6 43.031 34.978
QuickCache Sync Cache 8000000 1 71.124 7.675
QuickCache Sync Cache 8000000 3 71.139 3.198
QuickCache Sync Cache 8000000 6 71.143 2.348
Mini Moka Sync Cache 8000000 1 67.447 28.462
Mini Moka Sync Cache 8000000 3 67.995 17.054
Mini Moka Sync Cache 8000000 6 64.738 12.929

References

License

This project is licensed under the MIT license.

Dependencies

~2–29MB
~377K SLoC