5 releases (3 breaking)

0.4.0 Jan 11, 2025
0.3.0 Jan 6, 2025
0.2.1 Jan 5, 2025
0.2.0 Dec 17, 2024
0.1.0 Dec 15, 2024

#1596 in Machine learning

Download history 247/week @ 2024-12-15 4/week @ 2024-12-22 346/week @ 2025-01-05 71/week @ 2025-01-12 5/week @ 2025-01-19 8/week @ 2025-02-02 2/week @ 2025-02-09 12/week @ 2025-02-16 1/week @ 2025-02-23

579 downloads per month

MIT/Apache

51KB
1K SLoC

Rust 1K SLoC // 0.0% comments C++ 123 SLoC

xctch

Experimental bindings for executorch, currently compatible with version 0.5.

Building executorch

From the executorch directory:

rm -Rf cmake-out
cmake . \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_ENABLE_LOGGING=ON \
  -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
  -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DEXECUTORCH_BUILD_XNNPACK=ON \
  -DEXECUTORCH_BUILD_DEVTOOLS=ON \
  -DCMAKE_INSTALL_PREFIX=cmake-out \
  -DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
  -Bcmake-out
cmake --build cmake-out -j16 --target install --config Release

Running the nanogpt example

This is based on the Into to LLMs in ExecuTorch tutorial.

curl https://huggingface.co/openai-community/gpt2/resolve/main/vocab.json -o scripts/nanogpt/vocab.json
curl https://raw.githubusercontent.com/karpathy/nanoGPT/master/model.py -o scripts/nanogpt/model.py
python scripts/nanogpt/export_nanogpt.py

Cross-Compilation for Android

From the executorch directory:

rm -Rf cmake-android-out
cmake . \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_ENABLE_LOGGING=ON \
  -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
  -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
  -DEXECUTORCH_BUILD_DEVTOOLS=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DEXECUTORCH_BUILD_XNNPACK=ON \
  -DCMAKE_INSTALL_PREFIX=cmake-android-out \
  -DCMAKE_TOOLCHAIN_FILE=$HOME/Android/Sdk/ndk/26.2.11394342/build/cmake/android.toolchain.cmake \
  -DANDROID_ABI=arm64-v8a \
  -DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
  -Bcmake-android-out
cmake --build cmake-android-out -j16 --target install --config Release

Common Issues

E 00:00:00.000061 executorch:operator_registry.cpp:85] Re-registering aten::add.out, from NOT_SUPPORTED
E 00:00:00.000075 executorch:operator_registry.cpp:86] key: (null), is_fallback: true
F 00:00:00.000076 executorch:operator_registry.cpp:106] In function register_kernels(), assert failed (false): Kernel registration failed with error 18, see error log for details.

This is caused by the portable and optimized ops being linked together in the final binary. These are supposed to be merged together here and the binary shoud link with optimized_native_cpu_ops_lib rather than the other .*_ops_lib files.

Dependencies

~1.3–3MB
~51K SLoC