text-scanner

Warning: This library is experimental and may change drastically in 0.0.* versions.

A UTF-8 char-oriented, zero-copy, text and code scanning library.

This crate implements a UTF-8 char-based text Scanner, which includes various methods for scanning a string slice, as well as backtracking capabilities, which can be used to implement lexers for tokenizing text or code.

Scanning extensions for existing languages and formats have already been implemented, such as for Rust, C, Python, CSS, SCSS, JSON, JSON with Comments, and many more.

For examples of lexers implemented using Scanner, see the any-lexer crate, which implements lexers for e.g. Rust, C, Python, CSS, SCSS, JSON, JSON with Comments, and many more.