#interpreter #language #lua #embed #scripting

bin+lib safe_wren

A safe, pure-rust implementation of wren.io, drop-in replacement for wren.io's C implementation

1 unstable release

0.1.0 Oct 22, 2021

#57 in Programming languages

MIT license

5MB
38K SLoC

Rust 21K SLoC // 0.0% comments C 10K SLoC // 0.1% comments JavaScript 2.5K SLoC // 0.2% comments Python 2K SLoC // 0.3% comments Visual Studio Project 2K SLoC Dart 531 SLoC // 0.3% comments Lua 382 SLoC // 0.1% comments Ruby 266 SLoC // 0.1% comments GLSL 152 SLoC Visual Studio Solution 130 SLoC Shell 24 SLoC // 0.2% comments

safe_wren

A nearly-complete implementation of the Wren language (wren.io) in Rust.

The original https://github.com/wren-lang/wren from wren.io is refered to as wren_c to disambiguate from this safe_wren.

Similaries to wren_c

  • Passes ~90% of wren_c tests
  • Exposes ~90% of wren_c C API

Differences from wren_c

  • Never crashes to bad input (but can currently still be timed-out via infinite loops)
  • Reference-counted, no garbage collected (in progress).
  • Stops at first compile error (instead of continuing)
  • Requires utf8 input and strings are always utf8 (does not allow invalid utf8 bytes)
  • Does not allow overriding allocator (yet)
  • Missing opt-meta, and class attributes
  • About 2x slower than wren_c on some microbenchmarks (should be compariable after GC work completes)

Usage

From an existing C project:

cargo build --release produces target/release/libsafe_wren.a and target/release/libsafe_wren.so which are drop-in replacements for wren.a and wren.so and compatible with wren.h found at (wren_c/src/include/wren.h).

From Rust:

Add a dependency to your cargo.toml, e.g.

[dependencies]
safe_wren = "0.1.0"

Development

There are two binaries:

  • wren_test -- used for testing uses only public API
  • wren_debug -- used for debugging vm, uses private calls.

cargo run FILENAME_OR_STRING will run wren_test against a file or string.

cargo run --bin=wren_debug FILENAME_OR_STRING will run wren_debug

wren_debug sub-commands: tokenize Dumps token stream. compile Dumps compiler bytecode. interpret Similar to no arguments, excepts prints VM state after run.

python3 util/test.py will run the tests, including updating test_results/* with error text from any failed tests. test.py will also update test_results/passes.txt with the list of passing tests.

test_results/test_expectations.txt lists all currently skipped tests and why.

Work yet to do

Launch list?

  • Example using rust API
  • Example using C API
  • Publish to Cargo
  • Announce to wren-lang
  • Generate and publish Rust docs.

Ordered goals?

  • Fix delta blue (closure error!)
  • fix local_outside_method.wren
  • Time the tests / make faster (next is vec::alloc from method calls)
  • Fancier test_expectations system ** Config / Expectation pairs (c | FAIL, RUST | TIMEOUT)
  • Teach utils/test.py how to easily switch between rust and c_rust and c
  • remove all uses of 'as' (use into() instead).
  • validate_superclass could now use ClassSource to validate internal, etc.
  • String codepoint APIs (including String.iterate)
  • wrong line numbers for foreign method runtime errors.
  • attributes
  • rust implementation of meta package.
  • continue after failure during compiling?
  • \x should not round-trip through char.
  • Garbage Collection?
  • Sort methods to match wren_c order?
  • Variable should use a different type for each scope type.
  • fuzz both wren_c and safe_wren and compare output?
  • Look at some of the slow-unit fuzz results ** fuzz/artifacts/fuzz_target_1/slow-unit-63ea01d2d5ba869bdb889c3b51b21350d5a4ffea (lookup_symbol should be a hash) ** fuzz/artifacts/fuzz_target_1/slow-unit-355b25c3fc10bfd14a363cf737abf3a07bab4a1e (needless stack resizing)
  • wren_debug interpret wren_c/test/language/static_field/nested_class.wren does out of bound lookup in line_for_pc.

Leads to pursue

Benchmarking notes

  • Current numbers show safe_wren to be about 2.5x-9x slower than wren_c across the various microbenchmarks. Unclear what real-world effect this would have.
  • Value::clone is apparent in many benchmarks.
  • Using move/drain semantics when calling args from the stack could help avoid needing to clone values when converting them with try_into_X. Or at least make try_into_X use move semenatics and the clone explicit in the caller.
  • class_for_value under call_method shows up at ~5% on several benchmarks.
  • map_numeric heavily tests Value::PartialEq
  • binary_trees leans heavily (at least 20% of time) on ptr::drop_in_place (deallocation) of RefCell. GC would reduce this.
  • map_string spends 57% of time in core::string_plus, and 20% of time truncating the stack.

wren_c bugs

  • closures/functions defined in wren_core.wren end up with a null class pointer?
  • If you yield from the root, it gets set to state=OTHER, presumably later you might be able to call things on it?
  • WrenConfiguration likely leaked for each WrenVM constructed/destructed?

Dependencies