#parallel #performance #join #thread

dynqueue

Dynamically extendable Rayon parallel iterator

7 releases

0.3.0 Oct 12, 2020
0.2.1 Jul 20, 2020
0.1.3 Jun 22, 2020
0.1.2 May 25, 2020

#798 in Concurrency

50 downloads per month
Used in 4 crates (via shredder)

MIT license

16KB
358 lines

Rust Coverage Status

DynQueue - dynamically extendable Rayon parallel iterator

A DynQueue<T> can be iterated with into_par_iter producing (DynQueueHandle, T) elements. With the DynQueueHandle<T> a new T can be inserted in the DynQueue<T>, which is currently iterated over.

A Vec<T>, VecDeque<T> and crossbeam_queue::SegQueue<T> (with feature = "crossbeam-queue") can be turned into a DynQueue<T> with .into_dyn_queue().

use rayon::iter::IntoParallelIterator as _;
use rayon::iter::ParallelIterator as _;

use dynqueue::IntoDynQueue as _;

fn main() {
    let mut result = vec![1, 2, 3]
        .into_dyn_queue()
        .into_par_iter()
        .map(|(handle, value)| { if value == 2 { handle.enqueue(4) }; value })
        .collect::<Vec<_>>();
    result.sort();

    assert_eq!(result, vec![1, 2, 3, 4]);
}

Features

  • crossbeam-queue : to use crossbeam::queue::SegQueue as the inner collection.

Changelog

0.2.0

  • introduce IntoDynQueue
  • handle lockless collections

0.1.0

  • initial version

Dependencies

~1.5MB
~26K SLoC