#io #network #tokio #protocol #async

fixed-buffer

Fixed-size buffers for network protocol parsers

12 releases

0.3.0 Oct 9, 2021
0.2.3 Jan 14, 2021
0.2.2 Dec 9, 2020
0.1.7 Nov 20, 2020
0.1.3 Oct 30, 2020

#113 in Network programming

35 downloads per month
Used in fixed-buffer-tokio

Apache-2.0

92KB
1.5K SLoC

fixed-buffer

crates.io version license: Apache 2.0 unsafe forbidden pipeline status

This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.

Features

  • No unsafe
  • Depends only on std
  • Write bytes to the buffer and read them back
  • Lives on the stack
  • Does not allocate memory
  • Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
  • Easy to learn & use. Easy to maintain code that uses it.
  • Works with Rust latest, beta, and nightly
  • No macros
  • Good test coverage (99%)
  • fixed_buffer_tokio adds async functions

Limitations

  • Not a circular buffer. You can call shift() periodically to move unread bytes to the front of the buffer.

Documentation

https://docs.rs/fixed-buffer

Examples

Read and handle requests from a remote client:

use fixed_buffer::{
    deframe_line, FixedBuf, ReadWriteChain};
use std::io::{Error, Read, Write};
use std::net::TcpStream;

fn handle_request<RW: Read + Write>(
    reader_writer: &mut RW,
    request: Request,
) -> Result<(), Error> {
    // ...
    Ok(())
}

fn handle_conn(mut tcp_stream: TcpStream
) -> Result<(), Error> {
    let mut buf: FixedBuf<4096> = FixedBuf::new();
    loop {
        // Read a line
        // and leave leftover bytes in `buf`.
        let line_bytes = match buf.read_frame(
            &mut tcp_stream, deframe_line)? {
                Some(line_bytes) => line_bytes,
                None => return Ok(()),
            };
        let request = Request::parse(line_bytes)?;
        // Read any request payload
        // from `buf` + `TcpStream`.
        let mut reader_writer = ReadWriteChain::new(
            &mut buf, &mut tcp_stream);
        handle_request(&mut reader_writer, request)?;
    }
}

For a complete example, see tests/server.rs.

Read and process records:

use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;

fn try_process_record(b: &[u8]) -> Result<usize, Error> {
    if b.len() < 2 {
        return Ok(0);
    }
    if b.starts_with("ab".as_bytes()) {
        println!("found record");
        Ok(2)
    } else {
        Err(Error::new(ErrorKind::InvalidData, "bad record"))
    }
}

fn read_and_process<R: Read>(mut input: R)
    -> Result<(), Error> {
    let mut buf: FixedBuf<1024> = FixedBuf::new();
    loop {
        // Read a chunk into the buffer.
        if buf.copy_once_from(&mut input)? == 0 {
            return if buf.len() == 0 {
                // EOF at record boundary
                Ok(())
            } else {
                // EOF in the middle of a record
                Err(Error::from(
                    ErrorKind::UnexpectedEof))
            };
        }
        // Process records in the buffer.
        loop {
            let bytes_read =
                try_process_record(buf.readable())?;
            if bytes_read == 0 {
                break;
            }
            buf.read_bytes(bytes_read);
        }
        // Shift data in the buffer to free up
        // space at the end for writing.
        buf.shift();
    }
}
#

The filled constructor is useful in tests.

Alternatives

Changelog

TO DO

Release Process

  1. Edit Cargo.toml and bump version number.
  2. Run ../release.sh

License: Apache-2.0

No runtime deps