6 releases
0.2.3 | Mar 10, 2023 |
---|---|
0.2.2 | Feb 18, 2023 |
0.2.0 | Dec 11, 2022 |
0.1.1 | Aug 16, 2022 |
#21 in Caching
3,342 downloads per month
Used in 2 crates
69KB
1.5K
SLoC
Quick Cache
Lightweight and high performance concurrent cache optimized for low cache overhead.
- Small overhead compared to a concurrent hash table
- Scan resistent and high hit rate caching policy
- User defined weight per item
- Scales well with the number of threads
- Doesn't use background threads
- One usage of unsafe, trivially verifiable
- Small dependency tree
The implementation is optimized for use cases where the cache access times and overhead can add up to be a significant cost. Features like: time to live, event listeners and others; are not current implemented in Quick Cache. If you need these features you may want to take a look at the Moka crate.
Examples
Basic usage
use quick_cache::unsync::Cache;
fn main() {
let cache = Cache::new(5);
cache.insert("square", "blue");
cache.insert("circle", "black");
assert_eq!(*cache.get(&"square").unwrap(), "blue");
assert_eq!(*cache.get(&"circle").unwrap(), "black");
}
A cache with custom item weights. In this case according to the string length of the value.
use quick_cache::{Weighter, sync::Cache};
#[derive(Clone)]
struct StringWeighter;
impl Weighter<u64, (), String> for StringWeighter {
fn weight(&self, _key: &u64, _qey: &(), val: &String) -> NonZeroU32 {
NonZeroU32::new(val.len().clamp(1, u32::MAX as usize) as u32).unwrap()
}
}
fn main() {
let cache = Cache::with_weighter(100, 100_000, StringWeighter);
cache.insert(1, "1".to_string());
cache.insert(54, "54".to_string());
cache.insert(1000, "1000".to_string());
assert_eq!(*cache.get(&"1000").unwrap(), "1000");
}
Two keys variant (KQCache), which allows the caller to avoid making expensive keys on gets.
use quick_cache::sync::KQCache;
fn main() {
let cache = KQCache::new(5);
// Normally the cache key would be the tuple (String, u32), which could force
// the caller to clone/allocate the string in order to form the tuple key.
// That is not the case with the KQCache.
cache.insert("square".to_string(), 2022, "blue");
cache.insert("square".to_string(), 2023, "black");
assert_eq!(cache.get("square", &2022).unwrap(), "blue");
}
Benchmarks
Since this crate is performance oriented it needs some comparisons. That said, benchmarks can be misleading so take everything with a pinch of salt.
Benchmarks performed with mokabench in a x64 Linux OS + Intel i9-12900H CPU.
Trace 1 (S3 from the Arc paper)
Cache | Max Capacity | Clients | Hit Ratio(%) | Elapsed time(s) |
---|---|---|---|---|
HashLink (LRU w/ Mutex) | 100000 | 1 | 2.327 | 2.479 |
HashLink (LRU w/ Mutex) | 100000 | 3 | 2.327 | 6.171 |
HashLink (LRU w/ Mutex) | 100000 | 6 | 2.328 | 12.467 |
QuickCache Sync Cache | 100000 | 1 | 12.764 | 2.465 |
QuickCache Sync Cache | 100000 | 3 | 12.765 | 1.387 |
QuickCache Sync Cache | 100000 | 6 | 12.764 | 0.835 |
Mini Moka Sync Cache | 100000 | 1 | 10.436 | 10.473 |
Mini Moka Sync Cache | 100000 | 3 | 10.541 | 8.439 |
Mini Moka Sync Cache | 100000 | 6 | 10.366 | 8.008 |
HashLink (LRU w/ Mutex) | 400000 | 1 | 12.039 | 2.956 |
HashLink (LRU w/ Mutex) | 400000 | 3 | 12.041 | 6.939 |
HashLink (LRU w/ Mutex) | 400000 | 6 | 12.043 | 13.503 |
QuickCache Sync Cache | 400000 | 1 | 42.044 | 3.250 |
QuickCache Sync Cache | 400000 | 3 | 42.044 | 1.299 |
QuickCache Sync Cache | 400000 | 6 | 42.044 | 0.777 |
Mini Moka Sync Cache | 400000 | 1 | 42.544 | 9.671 |
Mini Moka Sync Cache | 400000 | 3 | 41.525 | 7.052 |
Mini Moka Sync Cache | 400000 | 6 | 41.007 | 6.618 |
HashLink (LRU w/ Mutex) | 800000 | 1 | 56.600 | 3.221 |
HashLink (LRU w/ Mutex) | 800000 | 3 | 56.603 | 5.921 |
HashLink (LRU w/ Mutex) | 800000 | 6 | 56.605 | 12.768 |
QuickCache Sync Cache | 800000 | 1 | 66.801 | 3.980 |
QuickCache Sync Cache | 800000 | 3 | 66.802 | 1.418 |
QuickCache Sync Cache | 800000 | 6 | 66.803 | 0.798 |
Mini Moka Sync Cache | 800000 | 1 | 70.304 | 10.613 |
Mini Moka Sync Cache | 800000 | 3 | 68.054 | 4.754 |
Mini Moka Sync Cache | 800000 | 6 | 66.288 | 4.137 |
Trace 2 (DS1 from the Arc paper)
Cache | Max Capacity | Clients | Hit Ratio(%) | Elapsed time(s) |
---|---|---|---|---|
HashLink (LRU w/ Mutex) | 1000000 | 1 | 3.086 | 10.097 |
HashLink (LRU w/ Mutex) | 1000000 | 3 | 3.085 | 22.443 |
HashLink (LRU w/ Mutex) | 1000000 | 6 | 3.082 | 41.351 |
QuickCache Sync Cache | 1000000 | 1 | 15.052 | 9.426 |
QuickCache Sync Cache | 1000000 | 3 | 15.059 | 4.139 |
QuickCache Sync Cache | 1000000 | 6 | 15.078 | 2.467 |
Mini Moka Sync Cache | 1000000 | 1 | 14.910 | 28.974 |
Mini Moka Sync Cache | 1000000 | 3 | 13.257 | 24.681 |
Mini Moka Sync Cache | 1000000 | 6 | 13.518 | 22.759 |
HashLink (LRU w/ Mutex) | 4000000 | 1 | 20.245 | 9.863 |
HashLink (LRU w/ Mutex) | 4000000 | 3 | 20.250 | 19.418 |
HashLink (LRU w/ Mutex) | 4000000 | 6 | 20.259 | 36.932 |
QuickCache Sync Cache | 4000000 | 1 | 44.565 | 9.083 |
QuickCache Sync Cache | 4000000 | 3 | 44.574 | 4.272 |
QuickCache Sync Cache | 4000000 | 6 | 44.569 | 2.449 |
Mini Moka Sync Cache | 4000000 | 1 | 45.398 | 31.108 |
Mini Moka Sync Cache | 4000000 | 3 | 44.236 | 21.381 |
Mini Moka Sync Cache | 4000000 | 6 | 41.723 | 18.719 |
HashLink (LRU w/ Mutex) | 8000000 | 1 | 43.035 | 9.751 |
HashLink (LRU w/ Mutex) | 8000000 | 3 | 43.034 | 16.554 |
HashLink (LRU w/ Mutex) | 8000000 | 6 | 43.031 | 34.978 |
QuickCache Sync Cache | 8000000 | 1 | 71.124 | 7.675 |
QuickCache Sync Cache | 8000000 | 3 | 71.139 | 3.198 |
QuickCache Sync Cache | 8000000 | 6 | 71.143 | 2.348 |
Mini Moka Sync Cache | 8000000 | 1 | 67.447 | 28.462 |
Mini Moka Sync Cache | 8000000 | 3 | 67.995 | 17.054 |
Mini Moka Sync Cache | 8000000 | 6 | 64.738 | 12.929 |
References
- CLOCK-Pro: An Effective Improvement of the CLOCK Replacement. (PDF)
- Caffeine (source code)
- Cache2k (source code)
License
This project is licensed under the MIT license.
Dependencies
~1–5MB
~78K SLoC