#async-io #io-uring #iocp #async #fs #net

completeio

Completion based IO drivers and async runtime

1 unstable release

0.1.0 Sep 26, 2023

#1353 in Asynchronous

MIT license

240KB
4.5K SLoC

CompleteIo

MIT licensed crates.io docs.rs

A thread-per-core Rust IO drivers and async runtime backed by IOCP/io_uring/mio. The name comes from "completion-based IO".

This repository is a fork of compio.

Why not Compio?

The project has different goals:

  • provide low overhead IO drivers that preallocate memory on initialization

  • drivers API accepts non-'static IO buffers

    • drivers don't own buffers
    • buffers are required to be Unpin
  • bias towards IoUring API to achieve zero-cost abstraction on Linux:

    • fixed size submission queue for operations
    • external runtime could submit an external queue of not yet queued operations as a single batch
    • timers are exposed as Timeout operation and use suspend-aware CLOCK_BOOTTIME clock source when it's available
  • Async runtime is an example runtime to test implementation of drivers

Quick start

With runtime feature enabled, we can use the high level APIs to perform fs & net IO.

use completeio::{fs::File, task::block_on};

let buffer = block_on(async {
    let file = File::open("Cargo.toml").unwrap();
    let (read, buffer) = file.read_to_end_at(Vec::with_capacity(1024), 0).await;
    let read = read.unwrap();
    assert_eq!(read, buffer.len());
    String::from_utf8(buffer).unwrap()
});
println!("{}", buffer);

While you can also control the low-level driver manually:

use arrayvec::ArrayVec;
use std::collections::VecDeque;
use completeio::{
    buf::IntoInner,
    driver::{AsRawFd, Driver, Entry, CompleteIo},
    fs::File,
    op::ReadAt,
};

let mut driver = Driver::new().unwrap();
let file = File::open("Cargo.toml").unwrap();
// Attach the `RawFd` to driver first.
driver.attach(file.as_raw_fd()).unwrap();

// Create operation and push it to the driver.
let mut op = ReadAt::new(file.as_raw_fd(), 0, Vec::with_capacity(4096));
let mut ops = VecDeque::from([(&mut op, 0).into()]);
driver.push_queue(&mut ops);

// Poll the driver and wait for IO completed.
let mut entries = ArrayVec::<Entry, 1>::new();
unsafe {
    driver
        .submit_and_wait_completed(None, &mut entries)
        .unwrap();
}
let entry = entries.drain(..).next().unwrap();
assert_eq!(entry.user_data(), 0);

// Resize the buffer by return value.
let n = entry.into_result().unwrap();
let mut buffer = op.into_inner();
unsafe {
    buffer.set_len(n);
}

println!("{}", String::from_utf8(buffer).unwrap());

Dependencies

~0.2–13MB
~102K SLoC