43 releases

new 0.3.7 Nov 18, 2024
0.3.6 Aug 28, 2024
0.3.5 Jul 3, 2024
0.3.2 Jan 29, 2024
0.1.0 Jun 24, 2022

#254 in Operating systems

Download history 114/week @ 2024-07-29 2/week @ 2024-08-05 105/week @ 2024-08-12 15/week @ 2024-08-19 167/week @ 2024-08-26 3/week @ 2024-09-02 3/week @ 2024-09-09 51/week @ 2024-09-16 70/week @ 2024-09-23 22/week @ 2024-09-30 35/week @ 2024-10-07 9/week @ 2024-10-14 21/week @ 2024-11-04 103/week @ 2024-11-11

125 downloads per month
Used in dora-cli

Apache-2.0

140KB
3.5K SLoC

Coordinator

Prototype for a process/library-based dora-rs implementation, instead of framework-based. The idea is that each operator is compiled as a separate executable. The dora-coordinator runtime is responsible for reading the dataflow descriptor file and launching the operators accordingly. The operators use a common library called dora-api, which implements the communication layer based on zenoh.

This approach has the following advantages:

  • Less overhead
    • No data transfer between a runtime and the operator
    • The compiler can inline and optimize the full process
  • More flexibility
    • Operators can be sync or async
    • They can decide how many threads and which execution model they use
    • The OS ensures fair share of resources (e.g. CPU time) -> no need to cooperate with other operators
    • Operators get all inputs immediately -> no need for input rules
    • Keeping local state is easily possible
  • Separate address spaces
    • The operators are isolated from each other.

There are drawbacks too, for example:

  • Less control
    • Processes run independently -> need to cooperate with the runtime, e.g. on stop signals
    • Operator migration is more difficult
  • Operators are always isolated
    • No way of using in-memory channels
    • Local sockets and shared memory should be still possible

Dependencies

~13–44MB
~717K SLoC