regex-literal - regex literal enclosed by delimiters

This crate provides a quick approach of creating regular expression Regex and sequence ReSequence from delimited literals at runtime. Its aim is to formalize regex literal in Rust computing.

Background

In Rust Reference[^1], primitive types (boolean, numeric and textual) have own literal expressions that are evaluated as single tokens in source code at compile time. But it is not the case for regular expression (abbr. regex).

In many scripting languages that implement PCRE library[^2], a regex pattern is enclosed by a pair of delimiters, for example,/pattern/im in JavaScript. Regex engines in Rust crate regex-automata, can only receive a general literal (&str) in building a one-pattern regex. In the interface of Regex::new_many, an array of many pattern strings is required, as there is no syntax for one string literal representing a compound regex.

Features

The crate delivers literal formats for regex and regex sets with the following punctuations:

• // (a pair of forward slashes) as the default delimiters that enclose a pattern.

• [] (a pair of square brackets) that hold a union of multiple patterns.

• <> (a pair of angle brackets) that hold a sequence of regex patterns and/or pattern unions that iterates over consecutive matchings.

• , (comma) serves as seperator in between regex pattern literals, while any whitespace unicode character[^3] is skipped in parsing.

Samples of regex literals

1. a simple pattern : r#"/ab+c/"#
2. a regex union literal: r#"[/(?i)ab+c/,/(?u)\s{2}D+/]"#
3. a regex sequence literal: r#"</(?i)ab+c/,/(?u)\s{2}D+/>"#
4. another regex sequence literal: r#"<[/(?i)ab+c/,/(?u)\s{2}D+/],/\s*\w+/>"#

Note that [crate::delimited::set_delimiter()] allows choosing a customized delimiter from crate::delimited::DELIMITER_CHARS. In addition, crate::util module provides public functions of text convertion between undelimited and delimited patterns.

Building Regex structs from regex-literal

The regular expression structs can be constructed via either XRegex::from_str or XRegex::new. The former uses the default regex literal delimiter ("/" transcoded in crate::delimited::DELIMITER); the latter allows a customised delimiter.

Examples

use regex_literal::{XRegex,FromStr,Regex,Match,PatternID,Input,Anchored};

//example 1: create a XRegex struct from a one-pattern literal
let text0 = "abc123";
//create one-pattern literal
let re0 = r#"/^[a-z]+\d{3}$/"#;
//construct XRegex
let mut x = XRegex::from_str(re0).unwrap();
//get Regex from XRegex struct
let x_one_pattern = x.get_regex().unwrap();
//check if the one pattern regex matches with the target text
assert!(x_one_pattern.is_match(text0));
//find the first match if it exists
let m = x_one_pattern.find(text0);
assert_eq!(m,Some(Match::must(0,0..6)));

//example 2: create a XRegex struct from a one-pattern literal
let text1 = "ABBBC abc123";
let re1 = "!!!!(?i)ab+c!!!!";
//construct XRegex
let mut y = XRegex::new(re1,&[b'!',b'!',b'!',b'!']).unwrap();
//get Regex from XRegex struct
let y_one_pattern = y.get_regex().unwrap();
// check if this one pattern regex matches with the input
assert!(y_one_pattern.is_match(text1));
//find all non-overlapping leftmost matches
let matches:Vec<Match> = y_one_pattern.find_iter(text1).collect();
assert_eq!(matches,vec![Match::must(0,0..5),Match::must(0,6..9),]);

//example 3: create a XRegex struct from a multiple-pattern literal
let reu = r"[/(?i)ab+c/,/\w+/]";
let mut m1 = XRegex::from_str(reu).unwrap();
//get Regex from XRegex struct
let m_patterns = m1.get_regex().unwrap();
assert!(m_patterns.is_match(text1));
let m_matches:Vec<Match> = m_patterns.find_iter(text1).collect();
assert_eq!(m_matches,vec![Match::must(0,0..5),Match::must(0,6..9),Match::must(1,9..12)]); //non-overlapping leftmost matches

let expected = Some(Match::must(1,0..7));
let input = Input::new("23ABBBC abc&").anchored(Anchored::Pattern(PatternID::must(1)));//choose the specific pattern for input
let n_patterns = XRegex::from_str(reu).unwrap().get_regex().unwrap();
let mut caps = n_patterns.create_captures();
n_patterns.search_captures(&input,&mut caps);
assert_eq!(expected, caps.get_match());

//example 4: create a XRegex struct from a regex sequence literal
let xre2 = XRegex::from_str(r"</(?i)ab+c/,/^\w+?\d+$/>").unwrap();
let seq_slice = xre2.as_slice().unwrap();
let child_regex = &seq_slice[0];
assert!(child_regex.is_match("abc333"));

Conversion of regex literals

1. crate::util::delimit and crate::util::undelimit provide regex literal conversion between undelimited and delimited forms.

Examples

# use regex_literal::util::{delimit,undelimit};
 
let delimiter = "/";
// a regex literal that includes delimiter(forward slash `/`)
let re1 = r"\d{2}/\d{2}/\d{4}";
let delimited1 = delimit(re1,delimiter);
let string1 = r"/\d{2}\/\d{2}\/\d{4}/";
assert_eq!(&delimited1[..],string1);

let undelimited = undelimit(&delimited1[..],delimiter).unwrap();
assert_eq!(&undelimited[..], re1);

2. crate::assembly::into_reu and crate::assembly::into_res annotate patterns with default delimiters into delimited literals of regular expression union and sequence accordingly. Note the transformations require feature "w".

Examples

# use regex_literal::assembly::into_reu;
let re1 = "(?i)ab+c";
let re2 = r"\w+";
let re_set = [re1,re2];
let reu = into_reu(&re_set);
assert_eq!(reu,r"[/(?i)ab+c/,/\w+/]".to_owned());

Acknowledgements

regex-literal has adopted PCRE-style delimiters on top of regex engines in Rust crate regex-automata.

[^1]: literal expressions

[^2]: PCRE flavor

[^3]: Unicode characters with property White_Space=yes

See the change list of the crate versions from CHANGELOG