18 releases (6 breaking)
new 0.7.2 | Sep 29, 2024 |
---|---|
0.6.0 | Jun 29, 2024 |
0.4.1 | Feb 28, 2024 |
0.3.2 | Dec 20, 2023 |
0.3.1 | Nov 12, 2023 |
#126 in Machine learning
33,381 downloads per month
Used in 102 crates
(69 directly)
1MB
29K
SLoC
Contains (Zip file, 2KB) tests/fortran_tensor_3d.pth, (Zip file, 2KB) tests/test.pt, (Zip file, 2KB) tests/test_with_key.pt
candle
Minimalist ML framework for Rust
lib.rs
:
ML framework for Rust
use candle_core::{Tensor, DType, Device};
let a = Tensor::arange(0f32, 6f32, &Device::Cpu)?.reshape((2, 3))?;
let b = Tensor::arange(0f32, 12f32, &Device::Cpu)?.reshape((3, 4))?;
let c = a.matmul(&b)?;
Features
- Simple syntax (looks and feels like PyTorch)
- CPU and Cuda backends (and M1 support)
- Enable serverless (CPU) small and fast deployments
- Model training
- Distributed computing (NCCL).
- Models out of the box (Llama, Whisper, Falcon, ...)
FAQ
- Why Candle?
Candle stems from the need to reduce binary size in order to enable serverless possible by making the whole engine smaller than PyTorch very large library volume
And simply removing Python from production workloads. Python can really add overhead in more complex workflows and the GIL is a notorious source of headaches.
Rust is cool, and a lot of the HF ecosystem already has Rust crates safetensors and tokenizers
Dependencies
~9–21MB
~352K SLoC