file-chunker

This crate provides the FileChunker type, which is useful for efficiently reading a file in (approximately) equally-sized parts.

The original use case was to process a log file in chunks, one thread per chunk, and to guarantee that each chunk ended with a full line of text.

Example

use file_chunker::FileChunker;
let file = std::fs::File::open("/path/to/file").unwrap();
let chunker = FileChunker::new(&file).unwrap();
chunker.chunks(1024, Some('\n'))
    .unwrap()
    .iter()
    .for_each(|chunk| {
        println!("{:?}", chunk);
    });

lib.rs:

This crate provides the FileChunker type, which is useful for efficiently reading a file in (approximately) equally-sized parts.

The original use case was to process a log file in chunks, one thread per chunk, and to guarantee that each chunk ended with a full line of text.

Example

use file_chunker::FileChunker;

let file = std::fs::File::open("/path/to/file").unwrap();
let chunker = FileChunker::new(&file).unwrap();
chunker.chunks(1024, Some('\n'))
    .unwrap()
    .iter()
    .for_each(|chunk| {
        println!("{:?}", chunk);
    });