#csv

bin+lib qsv-sniffer

A CSV file format sniffer for Rust, optimized for qsv

3 unstable releases

Uses new Rust 2021

0.5.0 May 29, 2022
0.4.1 May 28, 2022
0.4.0 May 24, 2022
0.2.1 Mar 19, 2022
0.2.0 Mar 19, 2022

#752 in Parser implementations

Download history 214/week @ 2022-08-08 111/week @ 2022-08-15 123/week @ 2022-08-22 108/week @ 2022-08-29 129/week @ 2022-09-05 175/week @ 2022-09-12 72/week @ 2022-09-19 97/week @ 2022-09-26 64/week @ 2022-10-03 93/week @ 2022-10-10 71/week @ 2022-10-17 81/week @ 2022-10-24 40/week @ 2022-10-31 72/week @ 2022-11-07 114/week @ 2022-11-14 127/week @ 2022-11-21

355 downloads per month
Used in qsv

MIT license

52KB
1K SLoC

qsv CSV sniffer

Documentation

qsv-sniffer provides methods to infer CSV file metadata (delimiter choice, quote character, number of fields, field names, field data types, etc.). See the documentation for more details.

Its a detached fork of csv-sniffer with these additional capabilities, detecting:

  • utf-8 encoding
  • field names
  • number of rows
  • additional data types - Date and DateTime

ℹ️ NOTE: This fork is optimized to support qsv, and its development will be primarily dictated by qsv's requirements. Please continue to use csv-sniffer if you want a general-purpose CSV sniffer.

Setup

As a Command-line application

cargo install qsv-sniffer

This will install a binary named sniff.

As a Library

Add this to your Cargo.toml:

[dependencies]
qsv-sniffer = "0.5"

and this to your crate root:

use qsv_sniffer;

Feature flags

  • cli - to build the sniff binary
  • runtime-dispatch-simd - enables detection of SIMD capabilities at runtime, which allows using the SSE2 and AVX2 code paths (only works on Intel and AMD architectures. Ignored on other architectures).
  • generic-simd - enables architecture-agnostic SIMD capabilities, but only works with Rust nightly.

The SIMD features are mutually exclusive and increase sampling performance.

Example

This example shows how to write a simple command-line tool for discovering the metadata of a CSV file:

use qsv_sniffer;

use std::env;

fn main() {
    let args: Vec<String> = env::args().collect();
    if args.len() != 2 {
        eprintln!("Usage: {} <file>", args[0]);
        ::std::process::exit(1);
    }

    // sniff the path provided by the first argument
    match qsv_sniffer::Sniffer::new().sniff_path(&args[1]) {
        Ok(metadata) => {
            println!("{}", metadata);
        },
        Err(err) => {
            eprintln!("ERROR: {}", err);
        }
    }
}

This example is provided as the primary binary for this crate. In the source directory, this can be run as:

$ cargo run -- tests/data/library-visitors.csv

Dependencies

~4.5MB
~82K SLoC