Downstream --- Rust Implementation

downstream provides efficient, constant-space implementations of stream curation algorithms.

• Free software: MIT license
• Documentation: https://mmore500.github.io/downstream

Installation

Add downstream as a dependency in your Cargo.toml:

[dependencies]
downstream = ">=0.0.0"

API Reference

See the Python quickstart for outline and intuition.

Each algorithm variant is accessible through the downstream::dstream module:

• Circular: downstream::dstream::CircularAlgo
• Compressing: downstream::dstream::CompressingAlgo
• Steady: downstream::dstream::SteadyAlgo
• Sticky: downstream::dstream::StickyAlgo
• Stretched: downstream::dstream::StretchedAlgo
• Tilted: downstream::dstream::TiltedAlgo

See selecting a dstream algorithm for more information.

has_ingest_capacity

pub fn has_ingest_capacity<Uint: downstream::_auxlib::UnsignedTrait>(S: Uint, T: Uint) -> bool

Determines if there is capacity to ingest a data item at logical time T.

• Uint: Integer type (e.g., u32)
• S: Buffer size (must be a power of two)
• T: Stream position of data item (zero-indexed)

assign_storage_site

pub fn assign_storage_site<Uint: downstream::_auxlib::UnsignedTrait>(S: Uint, T: Uint) -> Option<Uint>

Site selection algorithm for steady curation. Returns selected site or None if data should be discarded.

• Uint: Integer type (e.g., u32)
• S: Buffer size (must be a power of two)
• T: Stream position of data item (zero-indexed)

_assign_storage_site (low-level interface)

pub fn _assign_storage_site<Uint: downstream::_auxlib::UnsignedTrait>(S: Uint, T: Uint) -> Uint

Returns S if data should be discarded.

Citing

If downstream contributes to a scientific publication, please cite it as

Yang C., Wagner J., Dolson E., Zaman L., & Moreno M. A. (2025). Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling. arXiv preprint arXiv:2506.12975. https://doi.org/10.48550/arXiv.2506.12975

@misc{yang2025downstream,
      doi={10.48550/arXiv.2506.12975},
      url={https://arxiv.org/abs/2506.12975},
      title={Downstream: efficient cross-platform algorithms for fixed-capacity stream downsampling},
      author={Connor Yang and Joey Wagner and Emily Dolson and Luis Zaman and Matthew Andres Moreno},
      year={2025},
      eprint={2506.12975},
      archivePrefix={arXiv},
      primaryClass={cs.DS},
}

And don't forget to leave a star on GitHub!