fixed-buffer

This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.

Features

• forbid(unsafe_code)
• Depends only on std
• Write bytes to the buffer and read them back
• Lives on the stack
• Does not allocate
• Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
• No macros
• Good test coverage (100%)
• Async support by enabling cargo features async-std-feature, futures-io, smol-feature, or tokio.

Limitations

• Not a circular buffer. You can call shift() periodically to move unread bytes to the front of the buffer.

Examples

Read and handle requests from a remote client:

use fixed_buffer::{deframe_line, FixedBuf};
use std::io::Error;
use std::net::TcpStream;

fn handle_conn(mut tcp_stream: TcpStream) -> Result<(), Error> {
    let mut buf: FixedBuf<4096> = FixedBuf::new();
    loop {
        // Read a line
        // and leave leftover bytes in `buf`.
        let line_bytes = match buf.read_frame(
            &mut tcp_stream, deframe_line)? {
                Some(line_bytes) => line_bytes,
                None => return Ok(()),
            };
        let request = Request::parse(line_bytes)?;
        handle_request(request)?;
    }
}

For a complete example, see tests/server.rs.

Read and process records:

use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;

fn try_process_record(b: &[u8]) -> Result<usize, Error> {
    if b.len() < 2 {
        return Ok(0);
    }
    if b.starts_with("ab".as_bytes()) {
        println!("found record");
        Ok(2)
    } else {
        Err(Error::new(ErrorKind::InvalidData, "bad record"))
    }
}

fn read_and_process<R: Read>(mut input: R) -> Result<(), Error> {
    let mut buf: FixedBuf<1024> = FixedBuf::new();
    loop {
        // Read a chunk into the buffer.
        if buf.copy_once_from(&mut input)? == 0 {
            return if buf.len() == 0 {
                // EOF at record boundary
                Ok(())
            } else {
                // EOF in the middle of a record
                Err(Error::from(
                    ErrorKind::UnexpectedEof))
            };
        }
        // Process records in the buffer.
        loop {
            let num_to_consume =
                try_process_record(buf.readable())?;
            if num_to_consume == 0 {
                break;
            }
            buf.try_read_exact(num_to_consume).unwrap();
        }
        // Shift data in the buffer to free up
        // space at the end for writing.
        buf.shift();
    }
}
#

The From<&[u8; SIZE]> implementation is useful in tests. Example:

use core::convert::From;
assert_eq!(3, FixedBuf::from(b"abc").len());

Cargo Geiger Safety Report

Metric output format: x/y
    x = unsafe code used by the build
    y = total unsafe code found in the crate

Symbols: 
    🔒  = No `unsafe` usage found, declares #![forbid(unsafe_code)]
    ❓  = No `unsafe` usage found, missing #![forbid(unsafe_code)]
    ☢️  = `unsafe` usage found

Functions  Expressions  Impls  Traits  Methods  Dependency

0/0        0/0          0/0    0/0     0/0      🔒  fixed-buffer 1.0.0
0/0        0/0          0/0    0/0     0/0      ❓  ├── futures-io 0.3.31
0/25       0/1782       0/102  0/1     0/69     ❓  └── tokio 1.12.0
0/17       0/607        0/13   0/1     0/19     ❓      ├── bytes 1.1.0
0/90       0/596        0/2    0/0     0/69     ❓      ├── libc 0.2.161
0/37       0/2025       0/0    0/0     0/16     ❓      ├── memchr 2.4.1
0/90       0/596        0/2    0/0     0/69     ❓      │   └── libc 0.2.161
0/0        0/72         0/0    0/0     0/0      ❓      ├── num_cpus 1.13.1
0/90       0/596        0/2    0/0     0/69     ❓      │   └── libc 0.2.161
0/1        0/85         0/6    0/0     0/3      ❓      ├── once_cell 1.8.0
0/0        0/191        0/0    0/0     0/2      ❓      ├── pin-project-lite 0.2.14
0/0        0/12         0/0    0/0     0/0      🔒      ├── signal-hook-registry 1.4.0
0/90       0/596        0/2    0/0     0/69     ❓      │   └── libc 0.2.161
0/0        0/0          0/0    0/0     0/0      ❓      └── tokio-macros 1.7.0
0/0        0/12         0/0    0/0     0/3      ❓          ├── proc-macro2 1.0.36
0/0        0/0          0/0    0/0     0/0      🔒          │   └── unicode-xid 0.2.2
0/0        0/0          0/0    0/0     0/0      ❓          ├── quote 1.0.16
0/0        0/12         0/0    0/0     0/3      ❓          │   └── proc-macro2 1.0.36
0/0        0/47         0/3    0/0     0/2      ❓          └── syn 1.0.89
0/0        0/12         0/0    0/0     0/3      ❓              ├── proc-macro2 1.0.36
0/0        0/0          0/0    0/0     0/0      ❓              ├── quote 1.0.16
0/0        0/0          0/0    0/0     0/0      🔒              └── unicode-xid 0.2.2

0/170      0/5429       0/126  0/2     0/183  

Alternatives

• bytes, lots of unsafe
• buf_redux, circular buffer support, updated in 2019
• std::io::BufReader
• std::io::BufWriter
• static-buffer, updated in 2016
• block-buffer, for processing fixed-length blocks of data, some unsafe
• arrayvec, vector with fixed capacity, some unsafe

Changelog

• v1.0.0 2024-10-20 - From<&[u8; SIZE]>, FixedBuffer::from(b"x")
• v0.5.0 2022-03-21 - Move ReadWriteChain and ReadWriteTake to new read-write-ext crate.
• v0.4.0 2022-03-21
  • From<[u8; SIZE]>, FixedBuffer::from([0])
  • write_bytes to take AsRef<[u8]>
  • Rename try_read_exact to read_and_copy_exact.
  • Rename try_read_bytes to try_read_exact.
  • Remove empty, filled, read_byte, read_bytes, try_parse, and write_str.
  • deframe to allow consuming bytes without returning a frame
  • write_bytes to write as many bytes as it can, and return new NoWritableSpace error only when it cannot write any bytes. Remove NotEnoughSpaceError. The method now behaves like std::io::Write::write.

TO DO

• Change links in docs to standard style. Don't link to docs.rs.
• Idea: buf.slice(buf.read_frame(&mut reader, deframe_crlf))
• Add an frame_copy_iter function. Because of borrowing rules, this function must return non-borrowed (allocated and copied) data.
• Set up CI on:
  • DONE - Linux x86 64-bit
  • macOS
  • Windows
  • https://crate-ci.github.io/pr/testing.html#travisci
  • Linux ARM 64-bit (Raspberry Pi 3 and newer)
  • Linux ARM 32-bit (Raspberry Pi 2)
  • RISCV & ESP32 firmware?
• DONE - Try to make this crate comply with the Rust API Guidelines.
• DONE - Find out how to include Readme.md info in the crate's docs.
• DONE - Make the repo public
• DONE - Set up continuous integration tests and banner.
  • https://github.com/actions-rs/example
  • https://alican.codes/rust-github-actions/
• DONE - Add some documentation tests
  • https://doc.rust-lang.org/rustdoc/documentation-tests.html
  • https://doc.rust-lang.org/stable/rust-by-example/testing/doc_testing.html
• DONE - Set up public repository on Gitlab.com
  • https://gitlab.com/mattdark/firebase-example/blob/master/.gitlab-ci.yml
  • https://medium.com/astraol/optimizing-ci-cd-pipeline-for-rust-projects-gitlab-docker-98df64ae3bc4
  • https://hub.docker.com/_/rust
• DONE - Publish to crates.io
• DONE - Read through https://crate-ci.github.io/index.html
• DONE - Get a code review from an experienced rustacean
• DONE - Add and update a changelog
  • Update it manually
  • https://crate-ci.github.io/release/changelog.html

Release Process

1. Edit Cargo.toml and bump version number.
2. Run ../release.sh

License: Apache-2.0