#csv #batch #json #reader-writer #csv-reader #data-processing #json-file

spring-batch-rs

A toolkit for building enterprise-grade batch applications

3 unstable releases

0.2.1 Apr 2, 2024
0.2.0 Jan 31, 2024
0.1.5 Jan 10, 2024
0.1.1 Dec 22, 2023
0.1.0 Apr 15, 2023

#239 in Database interfaces

Download history 15/week @ 2023-12-30 7/week @ 2024-01-06 3/week @ 2024-01-27 7/week @ 2024-02-17 5/week @ 2024-02-24 1/week @ 2024-03-09 13/week @ 2024-03-23 117/week @ 2024-03-30 6/week @ 2024-04-06

136 downloads per month

MIT/Apache

91KB
1.5K SLoC

Spring-Batch for Rust

🐞 A toolkit for building enterprise-grade batch applications

crate docs build status Discord chat CodeCov license

Spring-Batch for Rust

Spring Batch for Rust, offers a robust and flexible framework for the development of batch processing applications, addressing the challenges of handling large-scale data processing tasks efficiently and reliably. It provides developers a comprehensive toolkit for building enterprise-grade batch applications.

Features

Feature Description
mongodb Enable reader and writer for Mongodb database
rdbc-postgres Enable rdbc reader and writer for Postgres database
rdbc-mysql Enable rdbc reader and writer for Mysql and MariaDb databases
rdbc-sqlite Enable rdbc reader and writer for Sqlite database
json Enable json reader and writer
csv Enable csv reader and writer
fake Enable fake reader. Useful for generate fake dataset
logger Enable logger writer. Useful for debugging

Roadmap

  • XML reader and writer
  • Filter items
  • Kafka reader and writer
  • Pulsar reader and writer
  • Retry/Skip policies
  • Save execution data in database

Getting Started

Make sure you activated the suitable features crate on Cargo.toml:

[dependencies]
spring-batch-rs = { version = "<version>", features = ["<full|json|csv|fake|logger>"] }

Then, on your main.rs:

fn main() -> Result<(), BatchError> {
    let csv = "year,make,model,description
   1948,Porsche,356,Luxury sports car
   1995,Peugeot,205,City car
   2021,Mazda,CX-30,SUV Compact
   1967,Ford,Mustang fastback 1967,American car";

    let reader = CsvItemReaderBuilder::new()
        .delimiter(b',')
        .has_headers(true)
        .from_reader(csv.as_bytes());

    let processor = UpperCaseProcessor::default();

    let writer = JsonItemWriterBuilder::new().from_path(temp_dir().join("cars.json"));

    let step: Step<Car, Car> = StepBuilder::new()
        .reader(&reader) // set csv reader
        .writer(&writer) // set json writer
        .processor(&processor) // set upper case processor
        .chunk(2) // set commit interval
        .skip_limit(2) // set fault tolerance
        .build();

    let job = JobBuilder::new().start(Box::new(&step)).build();
    let result = job.run();

    assert!(result.is_ok());
    assert!(step.get_status() == StepStatus::Success);

    Ok(())
}

Examples

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions

Dependencies

~4–25MB
~363K SLoC