Cargo Features
TextSplitter has no features set by default.
[dependencies]
text-splitter = { version = "0.12.1", features = ["markdown", "rust-tokenizers", "tiktoken-rs", "tokenizers"] }- markdown
-
Enables pulldown-cmark
- rust-tokenizers
-
Enables rust_tokenizers
- tiktoken-rs
-
Enables tiktoken-rs
- tokenizers
-
Enables tokenizers
TextSplitter has 4 features without comments.