Cargo Features

TextSplitter has no features set by default.

[dependencies]
text-splitter = { version = "0.20.0", features = ["code", "markdown", "rust-tokenizers", "tiktoken-rs", "tokenizers"] }
code

Enables tree-sitter

markdown

Enables pulldown-cmark

rust-tokenizers

Enables rust_tokenizers

tiktoken-rs

Enables tiktoken-rs

tokenizers

Enables onig of tokenizers

TextSplitter has 5 features without comments.