8 releases (4 breaking)

0.5.0 Aug 25, 2020
0.4.2 Aug 14, 2020
0.4.1 Jul 13, 2020
0.3.2 Sep 10, 2017
0.1.1 Jul 20, 2015

#293 in Data structures

Download history 10718/week @ 2024-03-14 15264/week @ 2024-03-21 12450/week @ 2024-03-28 12296/week @ 2024-04-04 13332/week @ 2024-04-11 15200/week @ 2024-04-18 14355/week @ 2024-04-25 15303/week @ 2024-05-02 13826/week @ 2024-05-09 12428/week @ 2024-05-16 13804/week @ 2024-05-23 13780/week @ 2024-05-30 20212/week @ 2024-06-06 14480/week @ 2024-06-13 13519/week @ 2024-06-20 9885/week @ 2024-06-27

60,363 downloads per month
Used in 40 crates (12 directly)

MIT license

22KB
318 lines

Cuckoo Filter

Crates.io

Documentation

Cuckoo filter is a Bloom filter replacement for approximated set-membership queries. While Bloom filters are well-known space-efficient data structures to serve queries like "if item x is in a set?", they do not support deletion. Their variances to enable deletion (like counting Bloom filters) usually require much more space.

Cuckoo filters provide the flexibility to add and remove items dynamically. A cuckoo filter is based on cuckoo hashing (and therefore named as cuckoo filter). It is essentially a cuckoo hash table storing each key's fingerprint. Cuckoo hash tables can be highly compact, thus a cuckoo filter could use less space than conventional Bloom filters, for applications that require low false positive rates (< 3%).

For details about the algorithm and citations please use this article for now

"Cuckoo Filter: Better Than Bloom" by Bin Fan, Dave Andersen and Michael Kaminsky

Example usage

extern crate cuckoofilter;

...

let value: &str = "hello world";

// Create cuckoo filter with default max capacity of 1000000 items
let mut cf = cuckoofilter::new();

// Add data to the filter
let success = cf.add(value).unwrap();
// success ==> Ok(())

// Lookup if data is in the filter
let success = cf.contains(value);
// success ==> true

// Test and add to the filter (if data does not exists then add)
let success = cf.test_and_add(value).unwrap();
// success ==> Ok(false)

// Remove data from the filter.
let success = cf.delete(value);
// success ==> true

C Interface

This crate has a C interface for embedding it into other languages than Rust. See the C Interface Documentation for more details.

Notes & TODOs

  • This implementation uses a a static bucket size of 4 fingerprints and a fingerprint size of 1 byte based on my understanding of an optimal bucket/fingerprint/size ratio from the aforementioned paper.
  • When the filter returns NotEnoughSpace, the element given is actually added to the filter, but some random other element gets removed. This could be improved by implementing a single-item eviction cache for that removed item.
  • There are no high-level bindings for other languages than C. One could add them e.g. for python using milksnake.

Dependencies

~0.6–1.3MB
~17K SLoC