35 releases (21 breaking)
new 0.22.1 | Jan 21, 2025 |
---|---|
0.21.1 | Dec 16, 2024 |
0.21.0 |
|
0.1.0 | May 20, 2024 |
#200 in Database implementations
684 downloads per month
Used in 26 crates
(24 directly)
87KB
1.5K
SLoC
Vortex Buffer
For now, a Vortex buffer is implemented as a very thin wrapper around the Tokio bytes crate. In the future, we may re-implement this ourselves to have more control over alignment (see https://github.com/tokio-rs/bytes/issues/437)
lib.rs
:
A library for working with custom aligned buffers of sized values.
The vortex-buffer
crate is built around bytes::Bytes
and therefore supports zero-copy
cloning and slicing, but differs in that it can define and maintain a custom alignment.
Buffer<T>
andBufferMut<T>
provide immutable and mutable wrappers aroundbytes::Bytes
andbytes::BytesMut
respectively.ByteBuffer
andByteBufferMut
are type aliases foru8
buffers.BufferString
is a wrapper around aByteBuffer
that enforces utf-8 encoding.ConstBuffer<T, const A: usize>
provides similar functionality toBuffer<T>
except with a compile-time alignment ofA
.buffer!
andbuffer_mut!
macros with the same syntax as the builtinvec!
macro for inline construction of buffers.
You can think of BufferMut<T>
as similar to a Vec<T>
, except that any operation that may
cause a re-allocation, e.g. extend, will ensure the new allocation maintains the buffer's
defined alignment.
For example, it's possible to incrementally build a Buffer<T>
with a 4KB alignment.
use vortex_buffer::{Alignment, BufferMut};
let mut buf = BufferMut::<i32>::empty_aligned(Alignment::new(4096));
buf.extend(0i32..1_000);
assert_eq!(buf.as_ptr().align_offset(4096), 0)
Comparison
Implementation | Zero-copy | Custom Alignment | Typed |
---|---|---|---|
vortex_buffer::Buffer<T> |
✔️ | ✔️ | ✔️ |
arrow_buffer::ScalarBuffer<T> |
✔️ | ❌️️️ | ✔️ |
bytes::Bytes |
✔️ | ❌️️️ | ❌️️️ |
Vec<T> |
❌️ | ❌️️ | ✔️ |
Features
The arrow
feature can be enabled to provide conversion functions to/from Arrow Rust buffers,
including arrow_buffer::Buffer
, arrow_buffer::ScalarBuffer<T>
, and
arrow_buffer::OffsetBuffer
.
Dependencies
~7MB
~111K SLoC