9 releases
new 0.2.5-rc1 | Dec 12, 2024 |
---|---|
0.2.4 | Nov 26, 2024 |
0.2.1-alpha.0 | Oct 23, 2024 |
0.2.0-alpha.1 | Sep 27, 2024 |
#38 in Magic Beans
5,369 downloads per month
1.5MB
37K
SLoC
⚡ Cairo Native ⚡
A compiler to convert Cairo's intermediate representation "Sierra" code
to machine code via MLIR and LLVM.
For in-depth documentation, see the developer documentation.
Disclaimer
🚧 Cairo Native is still being built therefore API breaking changes might happen often so use it at your own risk. 🚧
For versions under 1.0
cargo
doesn't comply with
semver, so we advise to pin the version the version you
use. This can be done by adding cairo-native = "0.1.0"
to your Cargo.toml
Getting Started
Dependencies
- Linux or macOS (aarch64 included) only for now
- LLVM 19 with MLIR: On debian you can use apt.llvm.org, on macOS you can use brew
- Rust 1.78.0 or later, since we make use of the u128 abi change.
- Git
Setup
This step applies to all operating systems.
Run the following make target to install the dependencies (both Linux and macOS):
make deps
Linux
Since Linux distributions change widely, you need to install LLVM 19 via your package manager, compile it or check if the current release has a Linux binary.
If you are on Debian/Ubuntu, check out the repository https://apt.llvm.org/ Then you can install with:
sudo apt-get install llvm-19 llvm-19-dev llvm-19-runtime clang-19 clang-tools-19 lld-19 libpolly-19-dev libmlir-19-dev mlir-19-tools
If you decide to build from source, here are some indications:
Install LLVM from source instructions
# Go to https://github.com/llvm/llvm-project/releases
# Download the latest LLVM 19 release:
# The blob to download is called llvm-project-19.x.x.src.tar.xz
# For example
wget https://github.com/llvm/llvm-project/releases/download/llvmorg-19.1.0/llvm-project-19.1.0.src.tar.xz
tar xf llvm-project-19.1.0.src.tar.xz
cd llvm-project-19.1.0.src.tar
mkdir build
cd build
# The following cmake command configures the build to be installed to /opt/llvm-19
cmake -G Ninja ../llvm \
-DLLVM_ENABLE_PROJECTS="mlir;clang;clang-tools-extra;lld;polly" \
-DLLVM_BUILD_EXAMPLES=OFF \
-DLLVM_TARGETS_TO_BUILD="Native" \
-DCMAKE_INSTALL_PREFIX=/opt/llvm-19 \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DLLVM_PARALLEL_LINK_JOBS=4 \
-DLLVM_ENABLE_BINDINGS=OFF \
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_ENABLE_LLD=ON \
-DLLVM_ENABLE_ASSERTIONS=OFF
ninja install
Setup a environment variable called MLIR_SYS_190_PREFIX
, LLVM_SYS_191_PREFIX
and TABLEGEN_190_PREFIX
pointing to the llvm directory:
# For Debian/Ubuntu using the repository, the path will be /usr/lib/llvm-19
export MLIR_SYS_190_PREFIX=/usr/lib/llvm-19
export LLVM_SYS_191_PREFIX=/usr/lib/llvm-19
export TABLEGEN_190_PREFIX=/usr/lib/llvm-19
Alternatively, if installed from Debian/Ubuntu repository, then you can use
env.sh
to automatically setup the environment variables.
source env.sh
MacOS
The makefile deps
target (which you should have ran before) installs LLVM 19
with brew for you, afterwards you need to execute the env.sh
script to setup
the needed environment variables.
source env.sh
Make targets:
Running make
by itself will check whether the required LLVM installation and
corelib is found, and then list available targets.
% make
LLVM is correctly set at /opt/homebrew/opt/llvm.
./scripts/check-corelib-version.sh 2.6.4
Usage:
deps: Installs the necesary dependencies.
build: Builds the cairo-native library and binaries in release mode.
build-native: Builds cairo-native with the target-cpu=native rust flag.
build-dev: Builds cairo-native under a development-optimized profile.
runtime: Builds the runtime library required for AOT compilation.
check: Checks format and lints.
test: Runs all tests.
proptest: Runs property tests.
coverage: Runs all tests and computes test coverage.
doc: Builds documentation.
doc-open: Builds and opens documentation in browser.
bench: Runs the hyperfine benchmark script.
bench-ci: Runs the criterion benchmarks for CI.
install: Invokes cargo to install the cairo-native tools.
clean: Cleans the built artifacts.
stress-test Runs a command which runs stress tests.
stress-plot Plots the results of the stress test command.
stress-clean Clean the cache of AOT compiled code of the stress test command.
Included Tools
Aside from the compilation and execution engine library, Cairo Native includes a few command-line tools to aid development, and some useful scripts.
These are:
- The contents of the
/scripts/
folder cairo-native-compile
cairo-native-dump
cairo-native-run
cairo-native-test
cairo-native-stress
scarb-native-dump
scarb-native-test
cairo-native-compile
Compiles a Cairo project outputting the generated MLIR and the shared library.
Exits with 1 if the compilation or run fails, otherwise 0.
Usage: cairo-native-compile [OPTIONS] <PATH> [OUTPUT_MLIR] [OUTPUT_LIBRARY]
Arguments:
<PATH> The Cairo project path to compile and run its tests
[OUTPUT_MLIR] The output path for the mlir, if none is passed, out.mlir will be the default
[OUTPUT_LIBRARY] If a path is passed, a dynamic library will be compiled and saved at that path
Options:
-s, --single-file Whether path is a single file
--allow-warnings Allows the compilation to succeed with warnings
-r, --replace-ids Replaces sierra ids with human-readable ones
-O, --opt-level <OPT_LEVEL> Optimization level, Valid: 0, 1, 2, 3. Values higher than 3 are considered as 3 [default: 0]
-h, --help Print help
-V, --version Print version
cairo-native-dump
Usage: cairo-native-dump [OPTIONS] <INPUT>
Arguments:
<INPUT>
Options:
-o, --output <OUTPUT> [default: -]
--starknet Compile a starknet contract
-h, --help Print help
cairo-native-run
This tool allows to run programs using the JIT engine, like the cairo-run
tool, the parameters can only be felt values.
Example: echo '1' | cairo-native-run 'program.cairo' 'program::program::main' --inputs - --outputs -
Exits with 1 if the compilation or run fails, otherwise 0.
Usage: cairo-native-run [OPTIONS] <PATH>
Arguments:
<PATH> The Cairo project path to compile and run its tests
Options:
-s, --single-file Whether path is a single file
--allow-warnings Allows the compilation to succeed with warnings
--available-gas <AVAILABLE_GAS> In cases where gas is available, the amount of provided gas
--run-mode <RUN_MODE> Run with JIT or AOT (compiled) [default: jit] [possible values: aot, jit]
-O, --opt-level <OPT_LEVEL> Optimization level, Valid: 0, 1, 2, 3. Values higher than 3 are considered as 3 [default: 0]
-h, --help Print help
-V, --version Print version
cairo-native-test
This tool mimics the cairo-test
tool
and is identical to it in interface, the only feature it doesn't have is the profiler.
Compiles a Cairo project and runs all the functions marked as `#[test]`.
Exits with 1 if the compilation or run fails, otherwise 0.
Usage: cairo-native-test [OPTIONS] <PATH>
Arguments:
<PATH> The Cairo project path to compile and run its tests
Options:
-s, --single-file Whether path is a single file
--allow-warnings Allows the compilation to succeed with warnings
-f, --filter <FILTER> The filter for the tests, running only tests containing the filter string [default: ]
--include-ignored Should we run ignored tests as well
--ignored Should we run only the ignored tests
--starknet Should we add the starknet plugin to run the tests
--run-mode <RUN_MODE> Run with JIT or AOT (compiled) [default: jit] [possible values: aot, jit]
-O, --opt-level <OPT_LEVEL> Optimization level, Valid: 0, 1, 2, 3. Values higher than 3 are considered as 3 [default: 0]
-h, --help Print help
-V, --version Print version
For single files, you can use the -s, --single-file
option.
For a project, it needs to have a cairo_project.toml
specifying the
crate_roots
. You can find an example under the cairo-tests/
folder, which
is a cairo project that works with this tool.
cairo-native-test -s myfile.cairo
cairo-native-test ./cairo-tests/
This will run all the tests (functions marked with the #[test]
attribute).
cairo-native-stress
This tool runs a stress test on Cairo Native.
A stress tester for Cairo Native
It compiles Sierra programs with Cairo Native, caches, and executes them with AOT runner. The compiled dynamic libraries are stored in `AOT_CACHE_DIR` relative to the current working directory.
Usage: cairo-native-stress [OPTIONS] <ROUNDS>
Arguments:
<ROUNDS>
Amount of rounds to execute
Options:
-o, --output <OUTPUT>
Output file for JSON formatted logs
-h, --help
Print help (see a summary with '-h')
To quickly run a stress test and save logs as json, run:
make stress-test
This takes a lot of time to finish (it will probably crash first), you can kill the program at any time.
To plot the results, run:
make stress-plot
To clear the cache directory, run:
make stress-clean
scarb-native-dump
This tool mimics the scarb build
command.
You can download it on our releases page.
This tool should be run at the directory where a Scarb.toml
file is and it will
behave like scarb build
, leaving the MLIR files under the target/
folder
besides the generated JSON sierra files.
scarb-native-test
This tool mimics the scarb test
command.
You can download it on our releases page.
Compiles all packages from a Scarb project matching `packages_filter` and
runs all functions marked with `#[test]`. Exits with 1 if the compilation
or run fails, otherwise 0.
Usage: scarb-native-test [OPTIONS]
Options:
-p, --package <SPEC> Packages to run this command on, can be a concrete package name (`foobar`) or a prefix glob (`foo*`) [env: SCARB_PACKAGES_FILTER=] [default: *]
-w, --workspace Run for all packages in the workspace
-f, --filter <FILTER> Run only tests whose name contain FILTER [default: ]
--include-ignored Run ignored and not ignored tests
--ignored Run only ignored tests
--run-mode <RUN_MODE> Run with JIT or AOT (compiled) [default: jit] [possible values: aot, jit]
-O, --opt-level <OPT_LEVEL> Optimization level, Valid: 0, 1, 2, 3. Values higher than 3 are considered as 3 [default: 0]
-h, --help Print help
-V, --version Print version
Benchmarking
Requirements
- hyperfine:
cargo install hyperfine
- cairo 2.9.0-dev.0
- Cairo Corelibs
- LLVM 19 with MLIR
You need to setup some environment variables:
$MLIR_SYS_190_PREFIX=/path/to/llvm19 # Required for non-standard LLVM install locations.
$LLVM_SYS_191_PREFIX=/path/to/llvm19 # Required for non-standard LLVM install locations.
$TABLEGEN_190_PREFIX=/path/to/llvm19 # Required for non-standard LLVM install locations.
You can then run the bench
makefile target:
make bench
The bench
target will run the ./scripts/bench-hyperfine.sh
script.
This script runs hyperfine commands to compare the execution time of programs in the ./programs/benches/
folder.
Each program is compiled and executed via the execution engine with the cairo-native-run
command and via the cairo-vm with the cairo-run
command provided by the cairo
codebase.
The cairo-run
command should be available in the $PATH
and ideally compiled with cargo build --release
.
If you want the benchmarks to run using a specific build, or the cairo-run
commands conflicts with something (e.g. the cairo-svg package binaries in macos) then the command to run cairo-run
with a full path can be specified with the $CAIRO_RUN
environment variable.
Dependencies
~33–51MB
~877K SLoC