Cargo Features

TextSplitter has no features set by default.

[dependencies]
text-splitter = { version = "0.25.1", features = ["code", "markdown", "tiktoken-rs", "tokenizers"] }
code

Enables tree-sitter

markdown

Enables pulldown-cmark

tiktoken-rs

Enables tiktoken-rs

tokenizers

Enables onig of tokenizers

TextSplitter has 4 features without comments.