27 releases
0.9.4 | Jan 2, 2025 |
---|---|
0.9.3 | Dec 24, 2024 |
0.9.1 | Apr 3, 2024 |
0.9.0 | Mar 28, 2024 |
0.1.2 | Nov 19, 2022 |
#406 in Text processing
530 downloads per month
Used in rsonpath
770KB
16K
SLoC
rsonpath-lib
– SIMD-powered JSONPath, as a library 🚀
Library for rsonpath
, the JSONPath engine for querying massive streamed datasets.
The main target of this crate is the rsonpath
CLI tool. Note that this API is unstable until we reach
v1.0.0. This is going to happen (we have a roadmap), but our dev resources are quite limited.
Contributions are welcome and appreciated.
Unsafe
The library uses unsafe
for SIMD operations, because it has to, at least until portable-simd
gets stabilized.
Because of this, a compiled library is not portable – if you build on a platform supporting
AVX2 and then use the same compiled code on an ARM platform, it will crash.
We put special care to not use unsafe
code anywhere else – in fact, the crate uses #[forbid(unsafe_code)]
when compiled without the default simd
feature.
Build & test
The dev workflow utilizes just
.
Use the included Justfile
. It will automatically install Rust for you using the rustup
tool if it detects there is no Cargo in your environment.
just build
just test
Architecture diagram
Below is a simplified overview of the module interactions and interfaces, and how data flows from the user's input (query, document) through the pipeline to produce results.
Optional features
The simd
feature is enabled by default and is recommended to make use of the performance benefits of the project.
The serde
feature is optional and enables the serde
dependency,
which allows serializing and deserializing the engine after compilation. Note: the binary format of the engine
is expected to evolve and so changing it is considered a minor update for semver purposes.
Dependencies
Showing direct dependencies.
cargo tree --package rsonpath-lib --edges normal --depth 1 --target=all --all-features
rsonpath-lib v0.9.4 (/home/mat/src/rsonpath/crates/rsonpath-lib)
├── cfg-if v1.0.0
├── log v0.4.22
├── memmap2 v0.9.5
├── rsonpath-syntax v0.4.0 (/home/mat/src/rsonpath/crates/rsonpath-syntax)
├── serde v1.0.217
├── smallvec v1.13.2
├── static_assertions v1.1.0
├── thiserror v2.0.9
└── vector-map v1.0.1
Justification
cfg-if
– used to support SIMD and no-SIMD versions.log
– Rust standard logging idiom.memmap2
– for fast reading of source files via a memory map instead of buffered copies.serde
– optional dependency for serialization and deserialization of compiled engines.smallvec
– crucial for small-stack performance.static_assertions
– additional reliability by some constant assumptions validated at compile time.thiserror
– idiomaticError
implementations.vector_map
– used in the query compiler for measurably better performance.
Dependencies
~5MB
~86K SLoC