8 releases
0.4.2 | Feb 5, 2021 |
---|---|
0.4.1 |
|
0.4.0 | Dec 23, 2020 |
0.3.0 | Feb 8, 2020 |
0.1.3 | Dec 8, 2018 |
#778 in Algorithms
663 downloads per month
Used in 9 crates
(4 directly)
17KB
260 lines
extsort

Exposes external sorting (i.e. on disk sorting) capability on arbitrarily sized iterator, even if the generated content of the iterator doesn't fit in memory. Once sorted, it returns a new sorted iterator.
In order to remain efficient for all implementations, the crate doesn't handle serialization, but leaves that to the user.
The sorter can optionally use rayon
to sort the in-memory buffer. It is generally
faster when the buffer size is big enough for parallelism to have an impact over its overhead.
Example
extern crate extsort;
extern crate byteorder;
use extsort::*;
use byteorder::{ReadBytesExt, WriteBytesExt};
use std::io::{Read, Write};
#[derive(Debug, Eq, PartialEq, Ord, PartialOrd)]
struct MyStruct(u32);
impl Sortable for MyStruct {
fn encode<W: Write>(&self, write: &mut W) {
write.write_u32::<byteorder::LittleEndian>(self.0).unwrap();
}
fn decode<R: Read>(read: &mut R) -> Option<MyStruct> {
read.read_u32::<byteorder::LittleEndian>()
.ok()
.map(MyStruct)
}
}
fn main() {
let sorter = ExternalSorter::new();
let reversed_data = (0..1000).rev().map(MyStruct).into_iter();
let sorted_iter = sorter.sort(reversed_data).unwrap();
let sorted_data: Vec<MyStruct> = sorted_iter.collect();
let expected_data = (0..1000).map(MyStruct).collect::<Vec<MyStruct>>();
assert_eq!(sorted_data, expected_data);
}
Dependencies
~1.3–6.5MB
~115K SLoC