#file #io #cache

freqfs

An in-memory filesystem cache layer over tokio::fs, with LFU eviction

7 releases (4 breaking)

new 0.5.0 Jan 18, 2022
0.4.4 Dec 9, 2021
0.4.3 Sep 28, 2021
0.3.1 Sep 23, 2021
0.1.0 Sep 21, 2021

#51 in Caching

Download history 14/week @ 2021-10-02 7/week @ 2021-10-09 9/week @ 2021-10-16 3/week @ 2021-10-23 16/week @ 2021-10-30 7/week @ 2021-11-06 1/week @ 2021-11-13 18/week @ 2021-11-20 11/week @ 2021-11-27 35/week @ 2021-12-04 9/week @ 2021-12-11 7/week @ 2021-12-18 10/week @ 2021-12-25 3/week @ 2022-01-01 12/week @ 2022-01-08 35/week @ 2022-01-15

62 downloads per month
Used in tinychain

Apache-2.0

42KB
960 lines

freqfs

An in-memory filesystem cache layer based on tokio::fs, with least-frequently-used eviction


lib.rs:

freqfs is an in-memory cache layer over [tokio::fs] with least-frequently-used eviction.

freqfs automatically caches the most frequently-used files and backs up the others to disk. This allows the developer to create and update large collections of data purely in-memory without explicitly sync'ing to disk, while still retaining the flexibility to run on a host with extremely limited memory. This is especially useful for web serving, database, and data science applications.

See the examples directory for detailed usage examples.

This crate assumes that file paths are valid Unicode and may panic if it encounters a file path which is not valid Unicode.

It also assumes that all file I/O under the cache root directory (the one whose path is passed to [load]) is routed through the cache (not e.g. via [tokio::fs] or [std::fs] elsewhere). It may raise a [std::io::Error] or panic if this assumption is not valid.

In the case that your program may not have permission to write to a directory or file in the cache, be sure to check the permissions before modifying any directory or file. The background cleanup thread will panic if it attempts an impermissible write operation.

Dependencies

~4MB
~70K SLoC