Unified syntax highlighting for arborium.

This crate provides the core highlighting engine that works with both:

• Statically linked Rust grammars: For CLI tools and servers
• Dynamically loaded WASM plugins: For browser contexts

Why Async in a Highlighting Library?

You might wonder why a syntax highlighting library has async code. The answer is browser support.

• Parsing is synchronous: Tree-sitter parsing cannot be async—it's a fundamentally synchronous operation that walks the syntax tree.

• Getting a grammar can be async: In browser contexts, grammar plugins are loaded from a CDN via JavaScript's dynamic import(). This is inherently async since it involves network requests and WASM instantiation.

In native Rust, grammars are statically linked, so the provider returns immediately. But the trait is async to support both use cases with the same code.

Architecture

The highlighting system is built around two key traits:

• Grammar: What a grammar can do — parse text and return spans
• GrammarProvider: How grammars are obtained — this is where sync vs async differs

The Sync-in-Async-Clothing Pattern

The core highlighting logic (including injection handling) is written once as async code in HighlighterCore. Two wrappers provide the sync and async APIs:

• SyncHighlighter: Polls the async future once and panics if it yields. This is safe for native Rust where providers return immediately.

• AsyncHighlighter: Actually awaits provider calls. Use this for browser/WASM contexts where grammar loading involves network requests.

This design ensures both environments share the exact same injection-handling logic, avoiding subtle bugs from duplicated code.

When to Use Which

Quick Start

use arborium_highlight::{SyncHighlighter, Grammar, GrammarProvider, ParseResult, Span};
use arborium_highlight::{HighlightConfig, HtmlFormat};

// Define your grammar (implements Grammar trait)
struct MyGrammar { /* ... */ }
impl Grammar for MyGrammar {
    fn parse(&mut self, text: &str) -> ParseResult {
        // Parse and return spans + injections
        ParseResult::default()
    }
}

// Define your provider (implements GrammarProvider trait)
struct MyProvider { /* ... */ }
impl GrammarProvider for MyProvider {
    type Grammar = MyGrammar;
    async fn get(&mut self, language: &str) -> Option<&mut Self::Grammar> {
        // Return grammar for language
        None
    }
}

// Use with default configuration (custom elements: <a-k>, <a-f>, etc.)
let mut highlighter = SyncHighlighter::new(MyProvider { /* ... */ });
let html = highlighter.highlight("rust", "fn main() {}");
// Output: <a-k>fn</a-k> <a-f>main</a-f>() {}

// Or use class-based output for compatibility with existing CSS
let config = HighlightConfig {
    html_format: HtmlFormat::ClassNames,
    ..Default::default()
};
let mut highlighter = SyncHighlighter::with_config(MyProvider { /* ... */ }, config);
let html = highlighter.highlight("rust", "fn main() {}");
// Output: <span class="keyword">fn</span> <span class="function">main</span>() {}

HTML Output Formats

Arborium supports multiple HTML output formats via HtmlFormat:

• CustomElements (default): Compact custom elements like <a-k>, <a-f>, etc.
• CustomElementsWithPrefix(prefix): Custom elements with your prefix, e.g., <code-k>
• ClassNames: Traditional <span class="keyword"> for compatibility
• ClassNamesWithPrefix(prefix): Namespaced classes like <span class="arb-keyword">

See HtmlFormat for examples and use cases.

arborium-highlight

Core syntax highlighting engine for arborium.

Features

This crate provides the unified highlighting engine that works with both:

• Statically linked Rust grammars: For CLI tools and servers
• Dynamically loaded WASM plugins: For browser contexts

Why Async?

The parsing is synchronous (tree-sitter fundamentally is), but getting a grammar can be async in browser contexts where plugins are loaded from a CDN via dynamic import(). The async trait supports both use cases with the same API.

Usage

use arborium_highlight::{Span, spans_to_html, HtmlFormat};

// After getting spans from a grammar...
let html = spans_to_html(source, spans, &HtmlFormat::CustomElements);

Part of the arborium project. See arborium.bearcove.eu for more information.