#parse-tree #sql-query #postgresql #query-parser #sql-parser #internal #normalize

pg_query

PostgreSQL parser that uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parse tree

12 releases (2 stable)

5.1.0 Jan 10, 2024
0.8.2 Sep 11, 2023
0.8.0 Jul 25, 2023
0.7.0 Jul 20, 2022
0.1.1 May 3, 2021

#117 in Database interfaces

Download history 126/week @ 2023-12-07 112/week @ 2023-12-14 24/week @ 2023-12-21 35/week @ 2023-12-28 155/week @ 2024-01-04 196/week @ 2024-01-11 470/week @ 2024-01-18 170/week @ 2024-01-25 176/week @ 2024-02-01 240/week @ 2024-02-08 1497/week @ 2024-02-15 1632/week @ 2024-02-22 805/week @ 2024-02-29 796/week @ 2024-03-07 1346/week @ 2024-03-14 1220/week @ 2024-03-21

4,467 downloads per month
Used in 4 crates

MIT license

18MB
341K SLoC

C 170K SLoC // 0.1% comments C++ 93K SLoC // 0.1% comments SQL 71K SLoC // 0.2% comments Rust 5K SLoC // 0.0% comments Ruby 1.5K SLoC // 0.1% comments

pg_query.rs   Build Status Latest Version Docs Badge

This Rust library uses the actual PostgreSQL server source to parse SQL queries and return the internal PostgreSQL parse tree.

It also allows you to normalize queries (replacing constant values with $1, etc.) and parse these normalized queries into a parse tree again.

When you build this library, it builds parts of the PostgreSQL server source (see libpg_query), and then statically links it into this library.

You can find further examples and a longer rationale for the original Ruby implementation here. The Rust version tries to have a very similar API.

Getting started

Add the following to your Cargo.toml

[dependencies]
pg_query = "5.0"

Examples

Parsing a query

use pg_query::NodeRef;

let result = pg_query::parse("SELECT * FROM contacts");
assert!(result.is_ok());
let result = result.unwrap();
assert_eq!(result.tables(), vec!["contacts"]);
assert!(matches!(result.protobuf.nodes()[0].0, NodeRef::SelectStmt(_)));

Normalizing a query

let result = pg_query::normalize("SELECT 1 FROM x WHERE y = (SELECT 123 FROM a WHERE z = 'bla')").unwrap();
assert_eq!(result, "SELECT $1 FROM x WHERE y = (SELECT $2 FROM a WHERE z = $3)");

Fingerprinting a query

let result = pg_query::fingerprint("SELECT * FROM contacts.person WHERE id IN (1, 2, 3, 4);").unwrap();
assert_eq!(result.hex, "643d2a3c294ab8a7");

Truncating a query

let query = "INSERT INTO \"x\" (a, b, c, d, e, f) VALUES (?)";
let result = pg_query::parse(query).unwrap();
assert_eq!(result.truncate(32).unwrap(), "INSERT INTO x (...) VALUES (...)");

Credits

Thanks to Paul Mason for his work on pg_parse that this crate is based on.

After version 0.6.0, Paul donated the pg_query crate to the pganalyze team. pg_parse is a lighter alternative that focuses on query parsing, while pg_query aims for feature parity with the Ruby gem.

License

PostgreSQL server source code, used under the PostgreSQL license.
Portions Copyright (c) 1996-2023, The PostgreSQL Global Development Group
Portions Copyright (c) 1994, The Regents of the University of California

All other parts are licensed under the MIT license, see LICENSE file for details.
Copyright (c) 2021 Paul Mason paul@form1.co.nz Copyright (c) 2021-2023, Duboce Labs, Inc. (pganalyze) team@pganalyze.com

Dependencies

~1.4–5.5MB
~102K SLoC