7 unstable releases (3 breaking)
new 0.4.3 | Aug 31, 2024 |
---|---|
0.4.2 | Aug 9, 2024 |
0.3.0 | Jul 21, 2024 |
0.2.0 | Jul 20, 2024 |
0.1.0 | Jun 26, 2024 |
#207 in Rust patterns
1,476 downloads per month
Used in 3 crates
265KB
4K
SLoC
UFOTOFU
Ufotofu provides APIs for lazily producing or consuming sequences of arbitrary length. Highlights of ufotofu include
- consistent error handling semantics across all supported modes of sequence processing,
- meaningful subtyping relations between, for example, streams and readers,
- absence of needless specialization of error types or item types,
- fully analogous APIs for synchronous and asynchronous code,
- the ability to chain sequences of heterogenous types, and
nostd
support.
You can read an in-depth discussion of the API designs here.
Core Abstractions
Ufotofu is built around a small hierarchy of traits that describe how to produce or consume a sequence item by item.
A Producer
provides the
items of a sequence to some client code, similar to the
futures::Stream
or the
core::iter::Iterator
traits. Client
code can repeatedly request the next item, and receives either another item, an error, or a
dedicated final item which may be of a different type than the repeated items. An iterator of
T
s corresponds to a producer of T
s with final item type ()
and error type !
.
A Consumer
accepts the
items of a sequence from some client code, similar to the
futures::Sink
traits. Client code
can repeatedly add new items to the sequence, until it adds a single final item which may be of a
different type than the repeated items. A final item type of ()
makes adding the final item
equivalent to calling a conventional
close
method.
Producers and consumers are fully dual; the pipe function writes as much data as possible from a producer into a consumer.
Consumers often buffer items in an internal queue before performing side-effects on data in larger
chunks, such as writing data to the network only once a full packet can be filled. The
BufferedConsumer
trait
extends the Consumer
trait to
allow client code to trigger effectful flushing of internal buffers. Dually, the
BufferedProducer
trait
extends the Producer
trait to
allow client code to trigger effectful prefetching of data into internal buffers.
Finally, the BulkProducer
and BulkConsumer
traits
extend BufferedProducer
and BufferedConsumer
respectively with the ability to operate on whole slices of items at a time, similar to
std::io::Read
and
std::io::Write
. The
bulk_pipe function leverages this
ability to efficiently pipe data — unlike the standard library's
Read and
Write traits, this is possible without
allocating an auxilliary buffer.
Crate Organisation
The ufotofu crate is split into three high-level modules:
sync
provides APIs for synchronous, blocking abstractions (thinkcore::iter::Iterator
),local_nb
providesFuture
-based, non-blocking APIs (thinkfutures::stream::Stream
) for single-threaded executors, andnb
providesFuture
-based, non-blocking APIs for multi-threaded executors.
All three modules implement the same concepts; the only differences are whether functions are
asynchronous, and, if so, whether futures implement
Send
. In particular, each module has
its own version of the core traits for interacting with sequences.
The nb
module lacks most features of the
sync
and
local_nb
modules, but the core trait
definitions are there, and we happily accept pull-requests.
Feature Flags
Ufotofu gates several features that are only interesting under certain circumstances behind feature flags. These API docs document all functionality, though, as if all feature flags were activated.
All functionality that relies on the Rust standard library is gated behind the std
feature flag
(enabled by default).
All functionality that performs dynamic memory allocations is gated behind the alloc
feature flag
(disabled by default).
All functionality that aids in testing and development is gated behind the dev
feature flag
(disabled by default).
License
Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
This project was funded through the NGI0 Core Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 101092990.
Dependencies
~225KB