Lib.rs

Lua string patterns in Rust

Lua string patterns are a powerful yet lightweight alternative to full regular expressions. They are not regexps, since there is no alternation (the | operator), but this is not usually a problem. In fact, full regexps become too powerful and power can be dangerous or just plain confusing. This is why OpenBSD's httpd has Lua patterns. The decision to use % as the escape rather than the traditional \ is refreshing. In the Rust context, lua-patterns is a very lightweight dependency, if you don't need the full power of the regex crate.

This library reuses the original source from Lua 5.2 - only 400 lines of battle-tested C. I originally did this for a similar project to bring these patterns to C++.

More information can be found on the Lua wiki. The cool thing is that Lua is a 300KB download, if you want to test patterns out without going through Rust.

I've organized the Rust interface much as the original Lua library, 'match', 'gmatch' and 'gsub', but made these methods of a LuaPattern struct. This is for two main reasons:

• although string patterns are not compiled, they can be validated upfront
• after a match, the struct contains the results

let mut m = lua_patterns::LuaPattern::new("one");
let text = "hello one two";
assert!(m.matches(text));
let r = m.range();
assert_eq!(r.start, 6);
assert_eq!(r.end, 9);

This not in itself impressive, since it can be done with the string find method. (new will panic if you feed it a bad pattern, so use new_try if you want more control.)

Once we start using patterns it gets more exciting, especially with captures:

let mut m = lua_patterns::LuaPattern::new("(%a+) one");
let text = " hello one two";
assert!(m.matches(text));
assert_eq!(m.capture(0),1..10); // "hello one"
assert_eq!(m.capture(1),1..6); // "hello"

Lua patterns (like regexps) are not anchored by default, so this finds the first match and works from there. The 0 capture always exists (the full match) and here the 1 capture just picks up the first word.

There is an obvious limitation: "%a" refers specifically to a single byte representing a letter according to the C locale. Lua people will often look for 'sequence of non-spaces' ("%S+"), etc - that is, identify maybe-UTF-8 sequences using surronding punctuation or spaces.

If you want your captures as strings, then there are several options. If there's just one, then match_maybe is useful:

let mut m = lua_patterns::LuaPattern::new("OK%s+(%d+)");
let res = m.match_maybe("and that's OK 400 to you");
assert_eq!(res, Some("400"));

You can grab them as a vector (it will be empty if the match fails.)

let mut m = lua_patterns::LuaPattern::new("(%a+) one");
let text = " hello one two";
let v = m.captures(text);
assert_eq!(v, &["hello one","hello"]);

This will create a vector. You can avoid excessive allocations with capture_into:

let mut m = lua_patterns::LuaPattern::new("(%a+) one");
let text = " hello one two";
let mut v = Vec::new();
if m.capture_into(text,&mut v) {
    assert_eq!(v, &["hello one","hello"]);
}

Imagine that this is happening in a loop - the vector is only allocated the first time it is filled, and thereafter there are no allocations. It's a convenient method if you are checking text against several patterns, and is actually more ergonomic than using Lua's string.match. (Personally I prefer to use those marvelous things called "if statements" rather than elaborate regular expressions.)

The gmatch method creates an interator over all matched strings.

let mut m = lua_patterns::LuaPattern::new("%S+");
let split: Vec<_> = m.gmatch("dog  cat leopard wolf  ").collect();
assert_eq!(split,&["dog","cat","leopard","wolf"]);

A single match is returned; if the pattern has no captures, you get the full match, otherwise you get the first match. So "(%S+)" would give you the same result.

A more general version is gmatch_captures which creates a streaming iterator over captures. You have to be a little careful with this one; in particular, you will get nonsense if you try to collect on the return captures: don't try to keep these values. It is fine to collect from an expression involving the get method however!

let mut m = lua_patterns::LuaPattern::new("(%S)%S+");
let split: Vec<_> = m.gmatch_captures("dog  cat leopard wolf")
       .map(|cc| cc.get(1)).collect();
assert_eq!(split,&["d","c","l","w"]);

Text substitution is an old favourite of mine, so here's gsub_with:

let mut m = lua_patterns::LuaPattern::new("%$(%S+)");
let res = m.gsub_with("hello $dolly you're so $fine",
    |cc| cc.get(1).to_uppercase()
);
assert_eq!(res,"hello DOLLY you're so FINE");

The closure is passed a Closures object and the captures are accessed using the get method; it returns a String.

The second form of gsub is convenient when you have a replacement string, which may contain closure references. (To add a literal "%" escape it like so "%%")

let mut m = lua_patterns::LuaPattern::new("%s+");
let res = m.gsub("hello dolly you're so fine","");
assert_eq!(res, "hellodollyyou'resofine");

let mut m = lua_patterns::LuaPattern::new("(%S+)%s*=%s*(%S+);%s*");
let res = m.gsub("a=2; b=3; c = 4;", "'%2':%1 ");
assert_eq!(res, "'2':a '3':b '4':c ");

The third form of string.gsub in Lua does lookup with a table - that is, a map. But for maps you really want to handle the 'not found' case in some special way:

use std::collections::HashMap;
let mut map = HashMap::new();
// updating old lines for the 21st Century
map.insert("dolly", "baby");
map.insert("fine", "cool");
map.insert("good-looking", "pretty");

let mut m = lua_patterns::LuaPattern::new("%$%((.-)%)");
let res = m.gsub_with("hello $(dolly) you're so $(fine) and $(good-looking)",
    |cc| map.get(cc.get(1)).unwrap_or(&"?").to_string()
);
assert_eq!(res,"hello baby you're so cool and pretty");

(The ".-" pattern means 'match as little as possible' - often called 'lazy' matching.)

This is equivalent to a replace string "%1:'%2'":

let mut m = lua_patterns::LuaPattern::new("(%S+)%s*=%s*([^;]+);");
let res = m.gsub_with("alpha=bonzo; beta=felix;",
    |cc| format!("{}:'{}',", cc.get(1), cc.get(2))
);
assert_eq!(res, "alpha:'bonzo', beta:'felix',");

Having a byte-oriented pattern matcher can be useful. For instance, this is basically the old strings utility - we read all of a 'binary' file into a vector of bytes, and then use gmatch_bytes to iterate over all &[u8] matches corresponding to two or more adjacent ASCII letters:

let mut words = lua_patterns::LuaPattern::new("%a%a+");
for w in words.gmatch_bytes(b"hello dolly") {
    println!("{}",String::from_utf8_lossy(w));
}

The pattern itself may be arbitrary bytes - Lua 'string' matching does not care about embedded nul bytes:

let patt = &[0xDE,0x00,b'+',0xBE];
let bytes = &[0xFF,0xEE,0x0,0xDE,0x0,0x0,0xBE,0x0,0x0];

let mut m = lua_patterns::LuaPattern::from_bytes(patt);
assert!(m.matches_bytes(bytes));
assert_eq!(&bytes[m.capture(0)], &[0xDE,0x00,0x00,0xBE]);

The problem here is that it's not obvious when our 'arbitrary' bytes include one of the special matching characters like $ (which is 0x24) and so on. Hence there is LuaPatternBuilder:

let bytes = &[0xFF,0xEE,0x0,0xDE,0x24,0x24,0xBE,0x0,0x0];

let patt = lua_patterns::LuaPatternBuilder::new()
    .bytes_as_hex("DE24") // less tedious than a byte slice
    .text("+")  // unescaped
    .bytes(&[0xBE]) // byte slice
    .build();

let mut m = lua_patterns::LuaPattern::from_bytes(&patt);
// picks up "DE2424BE"

Static verification: this version attempts to verify string patterns. If you want errors, use new_try and from_bytes_try, otherwise the constructors panic. If a match panics after successful verification, it is a BUG - please report the offending pattern.