16 releases (1 stable)
1.0.0 | Oct 21, 2024 |
---|---|
0.5.0 | Mar 21, 2022 |
0.4.0 | Mar 21, 2022 |
0.3.1 | Dec 30, 2021 |
0.1.7 | Nov 20, 2020 |
#146 in Parser implementations
7,917 downloads per month
Used in 7 crates
(6 directly)
52KB
402 lines
fixed-buffer
This is a Rust library with fixed-size buffers, useful for network protocol parsers and file parsers.
Features
forbid(unsafe_code)
- Depends only on
std
- Write bytes to the buffer and read them back
- Lives on the stack
- Does not allocate
- Use it to read a stream, search for a delimiter, and save leftover bytes for the next read.
- No macros
- Good test coverage (100%)
- Async support by enabling cargo features
async-std-feature
,futures-io
,smol-feature
, ortokio
.
Limitations
- Not a circular buffer.
You can call
shift()
periodically to move unread bytes to the front of the buffer.
Examples
Read and handle requests from a remote client:
use fixed_buffer::{deframe_line, FixedBuf};
use std::io::Error;
use std::net::TcpStream;
fn handle_conn(mut tcp_stream: TcpStream) -> Result<(), Error> {
let mut buf: FixedBuf<4096> = FixedBuf::new();
loop {
// Read a line
// and leave leftover bytes in `buf`.
let line_bytes = match buf.read_frame(
&mut tcp_stream, deframe_line)? {
Some(line_bytes) => line_bytes,
None => return Ok(()),
};
let request = Request::parse(line_bytes)?;
handle_request(request)?;
}
}
For a complete example, see
tests/server.rs
.
Read and process records:
use fixed_buffer::FixedBuf;
use std::io::{Error, ErrorKind, Read};
use std::net::TcpStream;
fn try_process_record(b: &[u8]) -> Result<usize, Error> {
if b.len() < 2 {
return Ok(0);
}
if b.starts_with("ab".as_bytes()) {
println!("found record");
Ok(2)
} else {
Err(Error::new(ErrorKind::InvalidData, "bad record"))
}
}
fn read_and_process<R: Read>(mut input: R) -> Result<(), Error> {
let mut buf: FixedBuf<1024> = FixedBuf::new();
loop {
// Read a chunk into the buffer.
if buf.copy_once_from(&mut input)? == 0 {
return if buf.len() == 0 {
// EOF at record boundary
Ok(())
} else {
// EOF in the middle of a record
Err(Error::from(
ErrorKind::UnexpectedEof))
};
}
// Process records in the buffer.
loop {
let num_to_consume =
try_process_record(buf.readable())?;
if num_to_consume == 0 {
break;
}
buf.try_read_exact(num_to_consume).unwrap();
}
// Shift data in the buffer to free up
// space at the end for writing.
buf.shift();
}
}
#
The From<&[u8; SIZE]>
implementation is useful in tests. Example:
use core::convert::From;
assert_eq!(3, FixedBuf::from(b"abc").len());
Cargo Geiger Safety Report
Metric output format: x/y
x = unsafe code used by the build
y = total unsafe code found in the crate
Symbols:
🔒 = No `unsafe` usage found, declares #![forbid(unsafe_code)]
❓ = No `unsafe` usage found, missing #![forbid(unsafe_code)]
☢️ = `unsafe` usage found
Functions Expressions Impls Traits Methods Dependency
0/0 0/0 0/0 0/0 0/0 🔒 fixed-buffer 1.0.0
0/0 0/0 0/0 0/0 0/0 ❓ ├── futures-io 0.3.31
0/25 0/1782 0/102 0/1 0/69 ❓ └── tokio 1.12.0
0/17 0/607 0/13 0/1 0/19 ❓ ├── bytes 1.1.0
0/90 0/596 0/2 0/0 0/69 ❓ ├── libc 0.2.161
0/37 0/2025 0/0 0/0 0/16 ❓ ├── memchr 2.4.1
0/90 0/596 0/2 0/0 0/69 ❓ │ └── libc 0.2.161
0/0 0/72 0/0 0/0 0/0 ❓ ├── num_cpus 1.13.1
0/90 0/596 0/2 0/0 0/69 ❓ │ └── libc 0.2.161
0/1 0/85 0/6 0/0 0/3 ❓ ├── once_cell 1.8.0
0/0 0/191 0/0 0/0 0/2 ❓ ├── pin-project-lite 0.2.14
0/0 0/12 0/0 0/0 0/0 🔒 ├── signal-hook-registry 1.4.0
0/90 0/596 0/2 0/0 0/69 ❓ │ └── libc 0.2.161
0/0 0/0 0/0 0/0 0/0 ❓ └── tokio-macros 1.7.0
0/0 0/12 0/0 0/0 0/3 ❓ ├── proc-macro2 1.0.36
0/0 0/0 0/0 0/0 0/0 🔒 │ └── unicode-xid 0.2.2
0/0 0/0 0/0 0/0 0/0 ❓ ├── quote 1.0.16
0/0 0/12 0/0 0/0 0/3 ❓ │ └── proc-macro2 1.0.36
0/0 0/47 0/3 0/0 0/2 ❓ └── syn 1.0.89
0/0 0/12 0/0 0/0 0/3 ❓ ├── proc-macro2 1.0.36
0/0 0/0 0/0 0/0 0/0 ❓ ├── quote 1.0.16
0/0 0/0 0/0 0/0 0/0 🔒 └── unicode-xid 0.2.2
0/170 0/5429 0/126 0/2 0/183
Alternatives
bytes
, lots ofunsafe
buf_redux
, circular buffer support, updated in 2019std::io::BufReader
std::io::BufWriter
static-buffer
, updated in 2016block-buffer
, for processing fixed-length blocks of data, someunsafe
arrayvec
, vector with fixed capacity, someunsafe
Changelog
- v1.0.0 2024-10-20 -
From<&[u8; SIZE]>
,FixedBuffer::from(b"x")
- v0.5.0 2022-03-21 - Move
ReadWriteChain
andReadWriteTake
to newread-write-ext
crate. - v0.4.0 2022-03-21
From<[u8; SIZE]>
,FixedBuffer::from([0])
write_bytes
to takeAsRef<[u8]>
- Rename
try_read_exact
toread_and_copy_exact
. - Rename
try_read_bytes
totry_read_exact
. - Remove
empty
,filled
,read_byte
,read_bytes
,try_parse
, andwrite_str
. deframe
to allow consuming bytes without returning a framewrite_bytes
to write as many bytes as it can, and return newNoWritableSpace
error only when it cannot write any bytes. RemoveNotEnoughSpaceError
. The method now behaves likestd::io::Write::write
.
Older changelog entries
- v0.3.1 - Implement
From<NotEnoughSpaceError>
andFrom<MalformedInputError>
forString
. - v0.3.0 - Breaking API changes:
- Change type parameter to const buffer size. Example:
FixedBuf<1024>
. - Remove
new
arg. - Remove
capacity
. - Remove
Copy
impl. - Change
writable
return type to&mut [u8]
.
- Change type parameter to const buffer size. Example:
- v0.2.3
- Add
read_byte
,try_read_byte
,try_read_bytes
,try_read_exact
,try_parse
. - Implement
UnwindSafe
- Add
- v0.2.2 - Add badges to readme
- v0.2.1 - Add
deframe
andmem
, needed byAsyncFixedBuf::read_frame
. - v0.2.0
- Move tokio support to
fixed_buffer_tokio
. - Add
copy_once_from
,read_block
,ReadWriteChain
, andReadWriteTake
.
- Move tokio support to
- v0.1.7 - Add
FixedBuf::escape_ascii
. - v0.1.6 - Add
filled
constructor. - v0.1.5 - Change
read_delimited
to returnOption<&[u8]>
, for clean EOF handling. - v0.1.4 - Add
clear()
. - v0.1.3
- Thanks to freax13 for these changes:
- Support any buffer size. Now you can make
FixedBuf<[u8; 42]>
. - Support any
AsRef<[u8]> + AsMut<[u8]>
value for internal memory:[u8; N]
Box<[u8; N]>
&mut [u8]
Vec<u8>
- Support any buffer size. Now you can make
- Renamed
new_with_mem
tonew
. UseFixedBuf::default()
to construct anyFixedBuf<T: Default>
, which includes arrays of sizes up to 32.
- Thanks to freax13 for these changes:
- v0.1.2 - Updated documentation.
- v0.1.1 - First published version
TO DO
- Change links in docs to standard style. Don't link to
docs.rs
. - Idea:
buf.slice(buf.read_frame(&mut reader, deframe_crlf))
- Add an
frame_copy_iter
function. Because of borrowing rules, this function must return non-borrowed (allocated and copied) data. - Set up CI on:
- DONE - Linux x86 64-bit
- macOS
- Windows
- https://crate-ci.github.io/pr/testing.html#travisci
- Linux ARM 64-bit (Raspberry Pi 3 and newer)
- Linux ARM 32-bit (Raspberry Pi 2)
- RISCV & ESP32 firmware?
- DONE - Try to make this crate comply with the Rust API Guidelines.
- DONE - Find out how to include Readme.md info in the crate's docs.
- DONE - Make the repo public
- DONE - Set up continuous integration tests and banner.
- DONE - Add some documentation tests
- DONE - Set up public repository on Gitlab.com
- DONE - Publish to crates.io
- DONE - Read through https://crate-ci.github.io/index.html
- DONE - Get a code review from an experienced rustacean
- DONE - Add and update a changelog
- Update it manually
- https://crate-ci.github.io/release/changelog.html
Release Process
- Edit
Cargo.toml
and bump version number. - Run
../release.sh
License: Apache-2.0
Dependencies
~0–5.5MB
~19K SLoC