3 releases (breaking)
0.3.0 | Oct 25, 2020 |
---|---|
0.2.0 | Oct 25, 2020 |
0.1.0 | Oct 25, 2020 |
#269 in Database implementations
235 downloads per month
Used in 5 crates
(4 directly)
20KB
382 lines
simple_wal
A simple Rust write-ahead-logging crate.
Features
- Optimized for sequential reads & writes
- Easy atomic log compaction
- Advisory locking
- CRC32 checksums
- Range scans
- Persistent log entry index
lib.rs
:
A simple write-ahead-logging crate.
Features
- Optimized for sequential reads & writes
- Easy atomic log compaction
- Advisory locking
- CRC32 checksums
- Range scans
- Persistent log entry index
The entire log is scanned through on startup in order to detect & clean interrupted writes and determine the length of the log. It's recommended to compact the log when old entries are no longer likely to be used.
Usage:
use simple_wal::LogFile;
let path = std::path::Path::new("./wal-log");
{
let mut log = LogFile::open(path).unwrap();
// write to log
log.write(&mut b"log entry".to_vec()).unwrap();
log.write(&mut b"foobar".to_vec()).unwrap();
log.write(&mut b"123".to_vec()).unwrap();
// flush to disk
log.flush().unwrap();
}
{
let mut log = LogFile::open(path).unwrap();
// Iterate through the log
let mut iter = log.iter(..).unwrap();
assert_eq!(iter.next().unwrap().unwrap(), b"log entry".to_vec());
assert_eq!(iter.next().unwrap().unwrap(), b"foobar".to_vec());
assert_eq!(iter.next().unwrap().unwrap(), b"123".to_vec());
assert!(iter.next().is_none());
}
{
let mut log = LogFile::open(path).unwrap();
// Compact the log
log.compact(1).unwrap();
// Iterate through the log
let mut iter = log.iter(..).unwrap();
assert_eq!(iter.next().unwrap().unwrap(), b"foobar".to_vec());
assert_eq!(iter.next().unwrap().unwrap(), b"123".to_vec());
assert!(iter.next().is_none());
}
Log Format:
00 01 02 03 04 05 06 07|08 09 10 11 12 13 14 15|.......|-4 -3 -2 -1|
-----------------------|-----------------------|-------|-----------|
starting index |entry length | entry | crc32 |
unsigned 64 bit int le |unsigned 64 bit int le | data | 32bit, le |
Numbers are stored in little-endian format.
The first 8 bytes in the WAL is the starting index.
Each entry follows the following format:
- A 64 bit unsigned int for the entry size.
- The entry data
- A 32 bit crc32 checksum.
Dependencies
~1.5–2.4MB
~40K SLoC