41 releases

0.3.6 Aug 28, 2024
0.3.5 Jul 3, 2024
0.3.2 Jan 29, 2024
0.3.0 Nov 3, 2023
0.1.0 Jun 24, 2022

#243 in Operating systems

Download history 166/week @ 2024-07-06 9/week @ 2024-07-13 2/week @ 2024-07-20 114/week @ 2024-07-27 2/week @ 2024-08-03 6/week @ 2024-08-10 111/week @ 2024-08-17 148/week @ 2024-08-24 25/week @ 2024-08-31 2/week @ 2024-09-07 50/week @ 2024-09-14 57/week @ 2024-09-21 32/week @ 2024-09-28 28/week @ 2024-10-05 20/week @ 2024-10-12 1/week @ 2024-10-19

86 downloads per month
Used in dora-cli

Apache-2.0

140KB
3.5K SLoC

Coordinator

Prototype for a process/library-based dora-rs implementation, instead of framework-based. The idea is that each operator is compiled as a separate executable. The dora-coordinator runtime is responsible for reading the dataflow descriptor file and launching the operators accordingly. The operators use a common library called dora-api, which implements the communication layer based on zenoh.

This approach has the following advantages:

  • Less overhead
    • No data transfer between a runtime and the operator
    • The compiler can inline and optimize the full process
  • More flexibility
    • Operators can be sync or async
    • They can decide how many threads and which execution model they use
    • The OS ensures fair share of resources (e.g. CPU time) -> no need to cooperate with other operators
    • Operators get all inputs immediately -> no need for input rules
    • Keeping local state is easily possible
  • Separate address spaces
    • The operators are isolated from each other.

There are drawbacks too, for example:

  • Less control
    • Processes run independently -> need to cooperate with the runtime, e.g. on stop signals
    • Operator migration is more difficult
  • Operators are always isolated
    • No way of using in-memory channels
    • Local sockets and shared memory should be still possible

Dependencies

~13–44MB
~717K SLoC