6 releases
0.0.7 | May 13, 2025 |
---|---|
0.0.6 | May 5, 2025 |
0.0.0-reserved |
|
#3 in #classification
535 downloads per month
155KB
3K
SLoC
transformers v0.0.7
[!warning] This crate is under active development. APIs may change as features are still being added, and things tweaked.
Transformers provides a simple, intuitive interface for Rust developers who want to work with Large Language Models locally, powered by the Candle crate. It offers an API inspired by Python's Transformers, tailored for Rust developers.
Supported Models & Pipelines
Text Generation:
Fill-Mask:
- ModernBERT
- Base
- Large
Sentiment Analysis:
- ModernBERT-multilingual-sentiment
- Base
- Large
Zero-Shot Classification:
- ModernBERT-zeroshot
- Base
- Large
Installation
cargo add transformers
Usage
At this point in development the only real way to interact with the models is through the given pipelines, I plan to eventually provide you with a simple interface to work with the models directly if you would like.
Inference will be quite slow at the moment, this is mostly due to not using the CUDA feature when compiling candle. I will be working on integrating this smoothly in future updates for much faster inference.
Some examples of how to use the existing pipelines:
Text Generation
There are two ways to generate text: by providing a simple prompt string, or by providing a list of messages for chat-like interactions.
Using prompt_completion
use transformers::pipelines::text_generation_pipeline::{
TextGenerationPipelineBuilder, ModelOptions, Qwen3Size,
};
fn main() {
// 1. Choose a model family and size
let model_choice = ModelOptions::Qwen3(Qwen3Size::Size0_6B);
// 2. Build the pipeline with optional parameters
let pipeline = TextGenerationPipelineBuilder::new(model_choice)
.temperature(0.7)
.repeat_penalty(1.1)
.build().unwrap();
// 3. Generate text from a prompt
let prompt = "Explain the concept of Large Language Models in simple terms.";
let max_tokens = 100;
let generated = pipeline.prompt_completion(prompt, max_tokens).unwrap();
println!("{}", generated);
}
Using message_completion
with the Message
Type
For more conversational interactions, you can use the message_completion
method, which takes a vector of Message
structs.
The Message
struct represents a single message in a chat and has a role
(system, user, or assistant) and content
. You can create messages using:
Message::system(content: &str)
: For system prompts.Message::user(content: &str)
: For user prompts.Message::assistant(content: &str)
: For model responses.
use transformers::pipelines::text_generation_pipeline::{
TextGenerationPipelineBuilder, ModelOptions, Phi4Size,
};
use transformers::Message; // Import the Message type
fn main() -> anyhow::Result<()> {
// 1. Choose a model family and size
let model_choice = ModelOptions::Phi4(Phi4Size::Size14B); // Example with a different model
// 2. Build the pipeline
let pipeline = TextGenerationPipelineBuilder::new(model_choice)
.temperature(0.8)
.build()?;
// 3. Create a sequence of messages
let messages = vec![
Message::system("You are a friendly and helpful AI assistant."),
Message::user("What is the capital of France?"),
];
let max_tokens = 50;
// 4. Generate a completion based on the messages
let generated_response = pipeline.message_completion(messages, max_tokens)?;
println!("Assistant: {}", generated_response);
// You can continue the conversation by adding the assistant's response
// and a new user message to the messages vector.
Ok(())
}
Fill Mask (ModernBERT)
use transformers::pipelines::fill_mask_pipeline::{
FillMaskPipelineBuilder, ModernBertSize,
};
fn main() -> anyhow::Result<()> {
// 1. Choose a model size (Base or Large)
let size = ModernBertSize::Base;
// 2. Build the pipeline
let pipeline = FillMaskPipelineBuilder::new(size).build()?;
// 3. Fill the mask
let prompt = "The capital of France is [MASK].";
let filled_text = pipeline.fill_mask(prompt)?;
println!("{}", filled_text); // Should print: The capital of France is Paris.
Ok(())
}
Sentiment Analysis (ModernBERT Finetune)
use transformers::pipelines::sentiment_analysis_pipeline::{SentimentAnalysisPipelineBuilder, SentimentModernBertSize};
use anyhow::Result;
fn main() -> Result<()> {
// 1. Choose a model size (Base or Large)
let size = SentimentModernBertSize::Base;
// 2. Build the pipeline
let pipeline = SentimentAnalysisPipelineBuilder::new(size).build()?;
// 3. Analyze sentiment
let sentence = "I love using Rust for my projects!";
let sentiment = pipeline.predict(sentence)?;
println!("Text: {}", sentence);
println!("Predicted Sentiment: {}", sentiment); // Should predict positive sentiment
Ok(())
}
Zero-Shot Classification (ModernBERT NLI Finetune)
use transformers::pipelines::zero_shot_classification_pipeline::{
ZeroShotClassificationPipelineBuilder, ZeroShotModernBertSize,
};
use anyhow::Result;
fn main() -> Result<()> {
// 1. Choose a model size (Base or Large)
let size = ZeroShotModernBertSize::Base;
// 2. Build the pipeline
let pipeline = ZeroShotClassificationPipelineBuilder::new(size).build()?;
// 3. Classify text using candidate labels
let text = "The Federal Reserve raised interest rates.";
let candidate_labels = &["economics", "politics", "sports", "technology"];
let results = pipeline.predict(text, candidate_labels)?;
println!("Text: {}", text);
for (label, score) in results {
println!("- {}: {:.4}", label, score);
}
// Example output:
// - economics: 0.8721
// - politics: 0.1134
// - technology: 0.0098
// - sports: 0.0047
Ok(())
}
Future Plans
- Add more model families and sizes
- Support additional pipelines (summarization, classification)
- Improve performance and error handling
Credits
A special thanks to Diaconu Radu-Mihai for transferring the transformers
crate name on crates.io
Dependencies
~26–42MB
~641K SLoC