node.rs

A nodejs-like framework for Rust.

The biggest thing that makes nodejs pleasent to program in is the API. Even the most ardent Javascript fan will agree that Js has a certain amount of wat inducing design decisions, but what's important about javascript, and especially nodejs, is that the code written in it usually Just Works. There isn't a complex type system (or any type system) to impede quick development and the APIs strike a good balance between making it easy to do what people typically want to do, and avoiding "magic" commands, which are difficult to understand and reason about, or worse, which the API designed expects the consumer to invoke blindly.

Why API matters

Here's an example of why API matters so much. This is a little program which reads out a file and replaces all instances of the word "cloud" (case insensitive) with the word "butt". Our first example showcases all the terrible API of the Java standard library:

import java.util.regex.Pattern;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

public class ApiMatters
{
    static final Pattern REGEX = Pattern.compile("cloud", Pattern.CASE_INSENSITIVE);
    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new FileReader("file.txt"));
        try {
            String line = br.readLine();
            while (line != null) {
                System.out.println(REGEX.matcher(line).replaceAll("butt"));
                line = br.readLine();
            }
        } finally {
            br.close();
        }
    }
}

Java really wants you to know how things are happening, every little detail is necessary, you need to know that you're reading a File, with a FileReader, and in case you might not want to cause a syscall every time you read, well you need a BufferedReader. And doing regex replacement requires that you create a Pattern object, which you should put as a static field in the class of your Object, you are making an Object, right ?

At the other extreme you have magic, the best example I can think of is in the old Bitcoin codebase, when the data structures are to be read or written to/from the disk or network, they need to be converted into a serialized form. Satoshi's lovely solution for this is one magic macro: IMPLEMENT_SERIALIZE which expands to about 1000 lines of nested C++ macro code. Satoshi to his credit only used this monstrosity for his own code. The real offense is exporting these magical constructs as API.

Magic APIs writers don't try to explain what's happening, instead they write documentation which amounts to "just call it, don't ask too many questions, it will work". But if ever it doesn't work, you're gonna be in a world of pain trying to understand what that thousand lines of meta-meta-meta-polymorphic programming actually does. The fundimental problem with magical APIs is they lack a solid metaphore, you really don't know if switching on the lights might cause the toilet to flush, and if it does, whether that is a bug, or some Rube Goldberg "feature" to save the mad scientist API author.

Nodejs is pleasent to work with because the API creators took the middle road. Javascript doesn't (as of 2018) have powerful macros so creating magical APIs is not easy, and the creators of nodejs made a solid effort to avoid global flags and side effects in their API design, while still hiding most of the things which the typical programmer is not likely to care about.

Take this snippet as example:

const Fs = require('fs');
Fs.readFile('./file.txt', 'utf8', (err, data) => {
    if (err) { throw err; }
    data.split('\n').forEach((l) => {
        console.log(l.replace(/cloud/i, 'butt'));
    })
});

Like the big Java example, it also reads the file, replaces cloud with butt and writes out the result. Unlike the big Java example, it doesn't require the programmer to know about buffer, regular expression compiling or any of the many types of errors which can occur throughout the process. A fair criticism of nodejs is that it is not easy to verify that all exceptional cases have been handled, but when you're trying to get a project off the ground, handling all exceptional cases is the least of your concerns.

What is Rust

Rust is a compiled language, so like C/C++ it can make small fast standalone binaries. Rust is also a memory safe language, so like Javascript, it cannot segfault*. Rust's type system is a state of the art system, comparable to that of a function language like haskell but Rust itself is procedural, often resembling C++. In some ways, you could imagine Rust as two languages, the procedural language which you use to write the code and the functional language which you use to convince the type system that your code is safe.

Introducing node.rs

Node.rs is an attempt at bringing the good stuff from Nodejs to Rust. It is built on top of MIO and contains an embedded event loop and callback functionality.

Simple example

The most simple example is a setTimeout(), unlike nodejs, you need to launch the event loop explicitly, you do that with the module builder. After building the module, you are called with a Scope. The Scope provides you access to the underlying event loop and is the first argument which is passed to every callback that is called. We'll get to the module builder later on, but for now you can just wrap your program with module().run((), |s| { ..... });.

extern crate noders;
fn main() {
    noders::module().run((), |s| {
        noders::time::set_timeout(s, |s, _| {
            println!("Hello1");
        }, 100);
    });
}

The Scope

In Javascript and other Scheme-like languages, nested scopes can access and modify variables of parent scopes like this:

let sum = 0;
[1,2,3,4,5].forEach((i) => {
    sum += i
});
console.log(sum);

This example is rather simple because everything happens synchronously. The number sum is gone by the time the code snippet completes.

However, in this example:

let x = 0;
setTimeout(() => { x++; }, 100);
setTimeout(() => { console.log(x); }, 200);

The number, x needs to continue to exist after the function where it was declared returns. Javascript achieves this by means of single-threaded execution and garbage collection, because two closures have been registered to the setTimeout function and those two closures hold a reference to x, Javascript will keep the memory location for x in memory until they are complete.

Because Rust has no garbage collector, every object in memory must have a unique owner, furthermore, in order to avoid pointer aliasing issues, the Rust language rules specify that if there is a mutable pointer to an object, there cannot be any other pointer to the same object at the same time.

So in Rust, the first example works:

fn main() {
    let mut sum = 0;
    vec![1,2,3,4,5].iter().for_each(|i|{
        sum += i
    });
    println!("{}", sum);
}

But the second example fails, because i is owned by the main function, and so it is de-allocated when the main function completes.

// error[E0597]: `i` does not live long enough
fn main() {
    let fake_event_loop = || {
        let mut i = 0;
        return (
            || { i += 1; },
            || { println!("{}", i); }
        );
    };
    let mut callbacks = fake_event_loop();
    (callbacks.0)();
    (callbacks.1)();
}

Try it out in the Rust Playground

How the Scope works

When you created a module with module().run((), |s| { ..... });, you might have noticed that the first argument to run() is (), the unit (similar to null in other programming languages). When you invoke run(), you can pass in any object and this object will be wrapped to create a scope, so with the following pattern, you can get a working scope:

extern crate noders;

struct Context {
    integer: i32,
    hi: &'static str,
    number: f32
}

fn main() {
    let ctx = Context {
        integer: 1,
        hi: &"Hello world",
        number: 3.5
    };

    noders::module().run(ctx, |s| {
        noders::time::set_timeout(s, |s,_| {
            println!("{} {}", s.hi, s.number);
            s.integer += 1;
        }, 100);
        noders::time::set_timeout(s, |s,_| {
            println!("{}", s.integer);
        }, 200);
    });
}

However, note that the scope wrapper adds the following functions cb(), as_rc(), and core(). So if you pass in an object with a core() method (for example), calling the core() method will not do what you expect.

The rec!{} macro

Since using noders can lead to creating lots of temporary scope objects, the rec!{} macro exists to help you create quick anonymous objects. Because Rust has strong type inferrence, you can often just pass the value and Rust will detect the type. So this:

extern crate noders;
struct Context {
    integer: i32,
    hi: &'static str,
    number: f32
}
fn main() {
    noders::module().run(Context {
        integer: 1,
        hi: &"Hello world",
        number: 3.5
    }, |s| {
        noders::time::set_timeout(s, |s,_| {
            println!("{} {}", s.hi, s.number);
            s.integer += 1;
        }, 100);
        noders::time::set_timeout(s, |s,_| {
            println!("{}", s.integer);
        }, 200);
    });
}

Can be simplified to this:

#[macro_use(rec)] extern crate noders;
fn main() {
    noders::module().run(rec!{
        integer: 1,
        hi: &"Hello world",
        number: 3.5
    }, |s| {
        noders::time::set_timeout(s, |s,_| {
            println!("{} {}", s.hi, s.number);
            s.integer += 1;
        }, 100);
        noders::time::set_timeout(s, |s,_| {
            println!("{}", s.integer);
        }, 200);
    });
}

Internally, what rec!{} does is examine entries which you create and craft some code like the following:

{
    struct Rec<A,B,C> { integer: A hi: B, number: C }
    Rec { integer: 1, hi: &"Hello world", number: 3.5 }
}

And Rust's strong type inferrence system is able to determine what the types of the objects are. CAUTION: If you create a rec!{} with ambiguous values (for example None), Rust may not be able to detect the type and you may have to use an explicit structure.

Throughout this document, we will use the rec!{} macro to simplify examples.

The time module

The heart of any event based system is the means to schedule a callback to trigger at some point in the future. The set_timeout(), set_interval(), clear_timeout() and clear_interval() functions are part of the time module. Like their Javascript cousins, the set_timeout() and set_interval() functions return a Token which can be used with clear_timeout() or clear_interval() to cancel the timeout or interval.

extern crate noders;
use noders::time;
fn main() {
    let x = 3;
    noders::module().run(rec!{
        to: noders::Token(0)
    }, |s| {
        s.to = time::set_timeout(s, |_,_| {
            println!("This should never happen");
        }, 100);
        time::set_timeout(s, |_,_| {
            time::clear_timeout(s, s.to);
        }, 50);
    });
}

The rec!{} macro

Making SubScopes