6 releases
0.2.3 | Oct 14, 2020 |
---|---|
0.2.2 | Sep 14, 2020 |
0.2.1 | Jul 21, 2020 |
0.1.1 | May 24, 2020 |
#20 in #wasm-file
25 downloads per month
82KB
859 lines
cargo watt
Watt is a runtime for executing Rust procedural macros compiled as WebAssembly.
I assume you are familiar with watt, dtolnay's crate for executing procedural macros in a web assembly interpreter.
There, tooling improvements are listed as remaining work, and this cargo subcommand aims to achieve that. Its purposes are
- compile existing proc-macro crates without manual intervention for the watt runtime
- verify that a wasm file is compiled from a particular source
A list of some popular procedural macros compiled with cargo watt
is available here.
Prerequisites
To use cargo watt
you will need to install rustc's wasm32 toolchain:
$ rustup target add wasm32-unknown-unknown
Also, for optimizing the size of the wasm
-file, wasm-strip (wabt) and wasm-opt (binaryen) will be used. To opt out, use --no-wasm-opt
or --no-wasm-strip
.
Building proc-macro crates (cargo watt build
)
Building works by first copying a crate (either from a local directory, a git repository or crates.io) into /tmp
.
The crate type is then changed to cdylib
, proc-macro2
is being patched to dtolnay's proc_macro2
.
Next, all procedural macros in it are being replaced with pub #[no_mangle] extern "C"
fns and proc_macro
is replaced with proc_macro2
, see this.
At this point, simple crates already compile, but there is more to be done to support a wider range of crates. Since we just change some signatures and hope for the best, sometimes stuff stops working. To 'fix' that (altough it's more of a hack), we do the following:
- replace
syn
with this syn patch, which basically has all instances ofproc_macro
replaced withproc_macro2
(see here) and the conditional compilation forwasm32-unknown-unknown
is removed - do a literal search and replace of
proc_macro
toproc_macro2
. This may sound stupid, but in my testing this works alright.
Of course, some crates still don't compile, in that case you need tweak things yourself.
Notably, anything depending on synstructure
or proc_macro_error
won't work, maybe patches for those will be provided in the future aswell.
Lastly, a shim crate is generated which calls into the generated web assembly file and executes the token tree transformation.
As a user, all you need to do is
$ cargo watt build --crate serde-derive
INFO cargo_watt > download crate 'serde-derive' into temporary directory...
INFO cargo_watt > begin compiling crate...
Updating git repository `https://github.com/dtolnay/watt`
Updating git repository `https://github.com/jakobhellermann/syn-watt`
Updating crates.io index
Compiling syn v1.0.22 (https://github.com/jakobhellermann/syn-watt#0f0ace5e)
Compiling serde_derive v1.0.110 (/tmp/cargo-watt-crate)
Finished release [optimized] target(s) in 19.65s
INFO cargo_watt > finished in 19.65s
INFO cargo_watt > compiled wasm file is 2.65mb large
INFO cargo_watt > generated crate in "serde_derive-watt"
Alternatively you can fetch a git repository (cargo watt build --git https://github.com/idanarye/rust-typed-builder
) or use a local path (cargo watt build ./path/to/crate
).
By default, cargo watt
will include all files of original crate (i.e. tests, documentation etc.) in the newly generated one.
If you'd like to only have Cargo.toml
, src/lib.rs
and src/the-macro.wasm
there is the --only-copy-essential
option.
Caveats
Some proc-macro crates need to export other things then the actual macros, so they are split into a regular rust crate exporting some Traits/Functions, which then reexports the macros from another crate.
This is why for example cargo watt --crate thiserror
will tell you that thiserror is not a proc macro crate. Instead, what you need to do is run cargo watt --crate thiserror-impl
and [patch]
thiserror-impl
to your generated crate.
Also, if the syn version specified in the Cargo.lock
file is newer than my patched version, that will result in a compilation error.
In most cases updating is as easy as rebasing the fork on upstream, but there should be some kind of automation for that which there isn't so far.
Verifying compilation (cargo watt verify
)
The isolation properties of running the macro inside web assembly ensure that it doesn't have unwanted access to files or the network, but the code it generates can still be mailcious. Therefor it is important to be able to verify that a compiled binary wasm file was indeed compiled by some source, which can then be audited manually.
Just as the build subcommand, cargo watt verify
works with local projects, remote git repos and crates.io crates, you can use it like this:
$ cargo watt build --crate serde-derive
...
$ cargo watt verify serde-derive_watt/src/serde-derive.wasm --crate serde-derive
INFO cargo_watt::wasm > finished in 17.3s
Success!
Currently though, a crate compiled an linux will be different than on macos. If you know why this is and how to fix it, let me know.
Installation
$ cargo install cargo-watt
Performance
How much of a difference does this make regarding compile times?
I profiled a crate with the following dependencies:
tokio = { version = "0.2", features = ["macros"] }
thiserror = "1.0"
[patch.crates-io]
tokio-macros = { git = "https://github.com/jakobhellermann/watt-contrib" }
thiserror-impl = { git = "https://github.com/jakobhellermann/watt-contrib" }
Without the patches:
With patches:
That's a difference of 6 seconds vs 17 seconds, so not bad. Of course, in a real project you're gonna have a more non-macro crates so the speed-up is less noticable, but it's still faster.
LICENSE
MIT © Jakob Hellermann
Dependencies
~9–21MB
~323K SLoC