#file #io #cache

freqfs

An in-memory filesystem cache layer over tokio::fs, with LFU eviction

15 unstable releases (6 breaking)

Uses new Rust 2021

new 0.7.0 Jan 26, 2023
0.5.6 Jan 6, 2023
0.5.4 Dec 29, 2022
0.5.3 Sep 27, 2022
0.4.3 Sep 28, 2021

#50 in Caching

Download history 8/week @ 2022-10-07 5/week @ 2022-10-14 7/week @ 2022-10-21 10/week @ 2022-10-28 10/week @ 2022-11-04 8/week @ 2022-11-11 10/week @ 2022-11-18 4/week @ 2022-11-25 3/week @ 2022-12-02 10/week @ 2022-12-09 13/week @ 2022-12-16 21/week @ 2022-12-23 45/week @ 2022-12-30 51/week @ 2023-01-06 17/week @ 2023-01-13 43/week @ 2023-01-20

169 downloads per month
Used in tinychain

Apache-2.0

39KB
931 lines

freqfs

An in-memory filesystem cache layer based on tokio::fs, with least-frequently-used eviction


lib.rs:

freqfs is an in-memory cache layer over [tokio::fs] with least-frequently-used eviction.

freqfs automatically caches the most frequently-used files and backs up the others to disk. This allows the developer to create and update large collections of data purely in-memory without explicitly sync'ing to disk, while still retaining the flexibility to run on a host with extremely limited memory. This is especially useful for web serving, database, and data science applications.

See the examples directory for detailed usage examples.

This crate assumes that file paths are valid Unicode and may panic if it encounters a file path which is not valid Unicode.

It also assumes that all file I/O under the cache root directory (the one whose path is passed to [load]) is routed through the cache (not e.g. via [tokio::fs] or [std::fs] elsewhere). It may raise a [std::io::Error] or panic if this assumption is not valid.

In the case that your program may not have permission to write to a filesystem entry, be sure to check the permissions before modifying it. The background cleanup thread will panic if it attempts an impermissible write operation.

Dependencies

~3–7.5MB
~122K SLoC