UFOTOFU

Ufotofu provides APIs for lazily producing or consuming sequences of arbitrary length. Highlights of ufotofu include

• consistent error handling semantics across all supported modes of sequence processing,
• meaningful subtyping relations between, for example, streams and readers,
• absence of needless specialization of error types or item types,
• fully analogous APIs for synchronous and asynchronous code,
• the ability to chain sequences of heterogenous types, and
• nostd support.

You can read an in-depth discussion of the API designs here.

Core Abstractions

Ufotofu is built around a small hierarchy of traits that describe how to produce or consume a sequence item by item.

A Producer provides the items of a sequence to some client code, similar to the futures::Stream or the core::iter::Iterator traits. Client code can repeatedly request the next item, and receives either another item, an error, or a dedicated final item which may be of a different type than the repeated items. An iterator of Ts corresponds to a producer of Ts with final item type () and error type !.

A Consumer accepts the items of a sequence from some client code, similar to the futures::Sink traits. Client code can repeatedly add new items to the sequence, until it adds a single final item which may be of a different type than the repeated items. A final item type of () makes adding the final item equivalent to calling a conventional close method.

Producers and consumers are fully dual; the pipe function writes as much data as possible from a producer into a consumer.

Consumers often buffer items in an internal queue before performing side-effects on data in larger chunks, such as writing data to the network only once a full packet can be filled. The BufferedConsumer trait extends the Consumer trait to allow client code to trigger effectful flushing of internal buffers. Dually, the BufferedProducer trait extends the Producer trait to allow client code to trigger effectful prefetching of data into internal buffers.

Finally, the BulkProducer and BulkConsumer traits extend BufferedProducer and BufferedConsumer respectively with the ability to operate on whole slices of items at a time, similar to std::io::Read and std::io::Write. The bulk_pipe function leverages this ability to efficiently pipe data — unlike the standard library's Read and Write traits, this is possible without allocating an auxilliary buffer.

Crate Organisation

The ufotofu crate is split into three high-level modules:

• sync provides APIs for synchronous, blocking abstractions (think core::iter::Iterator),
• local_nb provides Future-based, non-blocking APIs (think futures::stream::Stream) for single-threaded executors, and
• nb provides Future-based, non-blocking APIs for multi-threaded executors.

All three modules implement the same concepts; the only differences are whether functions are asynchronous, and, if so, whether futures implement Send. In particular, each module has its own version of the core traits for interacting with sequences.

The nb module lacks most features of the sync and local_nb modules, but the core trait definitions are there, and we happily accept pull-requests.

Feature Flags

Ufotofu gates several features that are only interesting under certain circumstances behind feature flags. These API docs document all functionality, though, as if all feature flags were activated.

All functionality that relies on the Rust standard library is gated behind the std feature flag (enabled by default).

All functionality that performs dynamic memory allocations is gated behind the alloc feature flag (disabled by default).

All functionality that aids in testing and development is gated behind the dev feature flag (disabled by default).

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101092990.