2 unstable releases
0.2.0 | May 16, 2024 |
---|---|
0.1.0 | May 15, 2024 |
#207 in Compression
39 downloads per month
7KB
92 lines
NdjsonZst
NdjsonZst is a Rust library for writing and reading newline-delimited JSON (NDJSON) files that are compressed using Zstandard (zstd). This library provides two primary structures: NdjsonZstWriter
and NdjsonZstReader
, enabling you to efficiently handle NDJSON files with compression.
Features
- Write NDJSON with Zstandard Compression: Use
NdjsonZstWriter
to write NDJSON data to a compressed file. - Read NDJSON from Compressed Files: Use
NdjsonZstReader
to read NDJSON data from a compressed file. - Optional Line Removal:
NdjsonZstWriter
allows writing data while removing newline (\n
) and carriage return (\r
) characters from the input string.
Usage
Adding Dependencies
To use NdjsonZst, add the following dependencies to your Cargo.toml
:
[dependencies]
ndjson_zst = "0.1.0"
Example
Writing NDJSON Data
use std::fs::File;
use ndjson_zst::NdjsonZstWriter;
fn main() {
let file_path = "data.zst";
let mut writer = NdjsonZstWriter::from(file_path);
writer.write(r#"{"name": "Alice", "age": 30}"#);
writer.write_with_remove_line("Line with\nnewlines\n");
}
Reading NDJSON Data
use ndjson_zst::NdjsonZstReader;
fn main() {
let file_path = "data.zst";
let reader = NdjsonZstReader::from(file_path);
for line in reader.lines() {
println!("{}", line);
}
}
API Documentation
NdjsonZstWriter
from(value: &str) -> Self
Creates a new NdjsonZstWriter
instance from a file path.
new(f: File) -> Self
Creates a new NdjsonZstWriter
instance from a File
.
write(&mut self, data: &str)
Writes a string to the NDJSON file, followed by a newline character.
write_with_remove_line(&mut self, data: &str)
Writes a string to the NDJSON file after removing all newline (\n
) and carriage return (\r
) characters, followed by a newline character.
NdjsonZstReader
from(value: &str) -> Self
Creates a new NdjsonZstReader
instance from a file path.
new(f: File) -> Self
Creates a new NdjsonZstReader
instance from a File
.
lines(&self) -> std::str::Lines
Returns an iterator over the lines of the NDJSON file.
License
This project is licensed under the MIT License.
Contributing
Contributions are welcome! Please feel free to submit a pull request or open an issue if you have any suggestions or improvements.
Acknowledgements
- zstd-rs for the Zstandard bindings for Rust.
Notes
- The current implementation of
NdjsonZstReader
reads the entire file into memory. Future improvements may include supporting streaming reads for handling large files efficiently.
This README provides a basic overview and example usage of the NdjsonZst library. For more detailed information, please refer to the source code and inline documentation.
Dependencies
~3MB
~51K SLoC