#api #io

io-lifetimes

A low-level I/O ownership and borrowing library

10 unstable releases (3 breaking)

0.3.1 Sep 16, 2021
0.3.0 Aug 21, 2021
0.2.4 Aug 18, 2021
0.2.0 Jun 24, 2021
0.0.0 Jun 12, 2021

#52 in Rust patterns

Download history 249/week @ 2021-07-06 3418/week @ 2021-07-13 2863/week @ 2021-07-20 2832/week @ 2021-07-27 2751/week @ 2021-08-03 3005/week @ 2021-08-10 4117/week @ 2021-08-17 4790/week @ 2021-08-24 5299/week @ 2021-08-31 5248/week @ 2021-09-07 8333/week @ 2021-09-14 5213/week @ 2021-09-21 6618/week @ 2021-09-28 6735/week @ 2021-10-05 5708/week @ 2021-10-12 7818/week @ 2021-10-19

13,459 downloads per month
Used in 45 crates (20 directly)

Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT

110KB
2.5K SLoC

io-lifetimes

A low-level I/O ownership and borrowing library

Github Actions CI Status crates.io page docs.rs docs

This library introduces OwnedFd, BorrowedFd, and supporting types and traits, and corresponding features for Windows, which implement safe owning and borrowing I/O lifetime patterns.

This is associated with RFC 3128, the I/O Safety RFC, which is now merged. Work is now underway to move the OwnedFd and BorrowedFd types and AsFd trait developed here into std.

Some features currently require nightly Rust, as they depend on rustc_attrs to perform niche optimizations needed for FFI use cases.

For a quick taste, check out the code examples:

  • hello, a basic demo of this API, doing low-level I/O manually, using the provided example FFI bindings
  • easy-conversions, demonstrating the from_into convenience feature for converting from an impl Into* into an impl From*.
  • portable-views, demonstrating the convenience feature which allows one to temporarily "view" a file descriptor as any owning type such as File
  • flexible-apis, demonstrating how to write library APIs that accept untyped I/O resources.
  • owning-wrapper, demonstrating how to implement a type which wraps an Owned* type.

The core of the API is very simple, and consists of two main types and three main traits:

pub struct BorrowedFd<'fd> { ... }
pub struct OwnedFd { ... }

pub trait AsFd { ... }
pub trait IntoFd { ... }
pub trait FromFd { ... }

impl AsRawFd for BorrowedFd<'_> { ... }
impl AsRawFd for OwnedFd { ... }
impl IntoRawFd for OwnedFd { ... }
impl FromRawFd for OwnedFd { ... }

impl Drop for OwnedFd { ... }

impl AsFd for BorrowedFd<'_> { ... }
impl AsFd for OwnedFd { ... }
impl IntoFd for OwnedFd { ... }
impl FromFd for OwnedFd { ... }

On Windows, there are Handle and Socket versions of every Fd thing, and a special HandleOrInvalid type to cope with inconsistent error reporting in the Windows API.

Full API documentation:

The magic of transparency

Here's the fun part. BorrowedFd and OwnedFd are repr(transparent) and hold RawFd values, and Option<BorrowedFd> and Option<OwnedFd> are FFI-safe (on nightly Rust), so they can all be used in FFI directly:

extern "C" {
    pub fn open(pathname: *const c_char, flags: c_int, ...) -> Option<OwnedFd>;
    pub fn read(fd: BorrowedFd<'_>, ptr: *mut c_void, size: size_t) -> ssize_t;
    pub fn write(fd: BorrowedFd<'_>, ptr: *const c_void, size: size_t) -> ssize_t;
    pub fn close(fd: OwnedFd) -> c_int;
}

With bindings like this, users never have to touch RawFd values. Of course, not all code will do this, but it is a fun feature for code that can. This is what motivates having BorrowedFd instead of just using &OwnedFd.

Note the use of Option<OwnedFd> as the return value of open, representing the fact that it can either succeed or fail.

I/O Safety in Rust Nightly

The I/O Safety implementation PR has now landed and is available on Rust Nightly. It can be used directly, or through io-lifetimes: when io_lifetimes_use_std mode is enabled, io-lifetimes uses the std's OwnedFd, BorrowedFd, and AsFd instead of defining its own.

To enable io_lifetimes_use_std mode:

  • Set the environment variable RUSTFLAGS=--cfg=io_lifetimes_use_std, and
  • add #![cfg_attr(io_lifetimes_use_std, feature(io_safety))] to your lib.rs or main.rs.

Note that, unfortunately, io_lifetimes_use_std mode doesn't support the optional impls for third-party crates.

The code in std uses From<OwnedFd> and Into<OwnedFd> instead of FromFd and IntoFd. io-lifetimes is unable to provide impls for these for third-party types, so it continues to provide FromFd and IntoFd for now, with default impls that forward to From<OwnedFd> and Into<OwnedFd> in io_lifetimes_use_std mode.

io-lifetimes also includes several features which are not (yet?) in std, including the portability traits AsFilelike/AsSocketlike/etc., the from_into_* functions in the From* traits, and views.

If you test a crate with the std I/O safety types and traits, or io-lifetimes in io_lifetimes_use_std mode, please post a note about it in the I/O safety tracking issue as an example of usage.

Prior Art

There are several similar crates: fd, filedesc, filedescriptor, owned-fd, and unsafe-io.

Some of these provide additional features such as the ability to create pipes or sockets, to get and set flags, and to do read and write operations. io-lifetimes omits these features, leaving them to to be provided as separate layers on top.

Most of these crates provide ways to duplicate a file descriptor. io-lifetimes currently treats this as another feature that can be provided by a layer on top, though if there are use cases where this is a common operation, it could be added.

io-lifetimes's distinguishing features are its use of repr(transparent) to support direct FFI usage, niche optimizations so Option can support direct FFI usafe as well (on nightly Rust), lifetime-aware As*/Into*/From* traits which leverage Rust's lifetime system and allow safe and checked from_* and as_*/into_* functions, and powerful convenience features enabled by its underlying safety.

io-lifetimes also has full Windows support, as well as Unix/Windows portability abstractions, covering both file-like and socket-like types.

io-lifetimes's OwnedFd type is similar to fd's FileDesc. io-lifetimes doesn't have a close_on_drop parameter, and instead uses OwnedFd and BorrowedFd to represent dropping and non-dropping handles, respectively, in a way that is checked at compile time rather than runtime.

io-lifetimes's OwnedFd type is also similar to filedesc's FileDesc io-lifetimes's OwnedFd reserves the value -1, so it doesn't need to test for -1 in its Drop, and Option<OwnedFd> (on nightly Rust) is the same size as FileDesc.

io-lifetimes's OwnedFd type is also similar to owned-fd's OwnedFd. io-lifetimes doesn't implement Clone, because duplicating a file descriptor can fail due to OS process limits, while Clone is an infallible interface.

io-lifetimes's BorrowedFd is similar to owned-fd's FdRef, except it uses a lifetime parameter and PhantomData rather than transmuting a raw file descriptor value into a reference value.

io-lifetimes's convenience features are similar to those of unsafe-io, but io-lifetimes is built on its own As*/Into*/From* traits, rather than extending AsRaw*/IntoRaw*/FromRaw* with OwnsRaw, so they're simpler and safer to use. io-lifetimes doesn't include unsafe-io's *ReadWrite* or *HandleOrSocket* abstractions, and leaves these as features to be provided by separate layers on top.

Dependencies

~0–2.4MB
~44K SLoC