#odbc #database #sql #arrow


Read/Write Apache Arrow arrays from/to ODBC data sources

79 releases (27 breaking)

0.28.4 May 26, 2023
0.28.2 Apr 26, 2023
0.26.10 Mar 29, 2023
0.26.0 Dec 20, 2022
0.6.1 Nov 30, 2021

#304 in Database interfaces

Download history 445/week @ 2023-02-14 205/week @ 2023-02-21 94/week @ 2023-02-28 196/week @ 2023-03-07 417/week @ 2023-03-14 157/week @ 2023-03-21 299/week @ 2023-03-28 396/week @ 2023-04-04 210/week @ 2023-04-11 238/week @ 2023-04-18 316/week @ 2023-04-25 134/week @ 2023-05-02 257/week @ 2023-05-09 84/week @ 2023-05-16 133/week @ 2023-05-23 262/week @ 2023-05-30

787 downloads per month
Used in arrow-flightsql-odbc

MIT license



Docs Licence Crates.io

Fill Apache Arrow arrays from ODBC data sources. arrow-odbc is build on top of the arrow and odbc-api crates and enables you to read the data of an ODBC data source as sequence of Apache Arrow record batches. arrow-odbc can also be used to insert the contens of Arrow record batches into a database table.

This repository contains the code of the arrow-odbc Rust crate. The repository containing the code for the arrow-odbc Python wheel resides in the arrow-odbc-py repository.

About Arrow

Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead.

About ODBC

ODBC (Open DataBase Connectivity) is a standard which enables you to access data from a wide variaty of data sources using SQL.


use arrow_odbc::{odbc_api::{Environment, ConnectionOptions}, OdbcReader};

const CONNECTION_STRING: &str = "\
    Driver={ODBC Driver 17 for SQL Server};\

fn main() -> Result<(), anyhow::Error> {

    let odbc_environment = Environment::new()?;
    // Connect with database.
    let connection = odbc_environment.connect_with_connection_string(

    // This SQL statement does not require any arguments.
    let parameters = ();

    // Execute query and create result set
    let cursor = connection
        .execute("SELECT * FROM MyTable", parameters)?
        .expect("SELECT statement must produce a cursor");

    // Each batch shall only consist of maximum 10.000 rows.
    let max_batch_size = 10_000;

    // Read result set as arrow batches. Infer Arrow types automatically using the meta
    // information of `cursor`.
    let arrow_record_batches = OdbcReader::new(cursor, max_batch_size)?;

    for batch in arrow_record_batches {
        // ... process batch ...

Matching of ODBC to Arrow types then querying

ODBC Arrow
Numeric(p <= 38) Decimal128
Decimal(p <= 38, s >= 0) Decimal128
Integer Int32
SmallInt Int16
Real Float32
Float(p <=24) Float32
Double Float64
Float(p > 24) Float64
Date Date32
LongVarbinary Binary
Timestamp(p = 0) TimestampSecond
Timestamp(p: 1..3) TimestampMilliSecond
Timestamp(p: 4..6) TimestampMicroSecond
Timestamp(p >= 7 ) TimestampNanoSecond
BigInt Int64
TinyInt Int8
Bit Boolean
Varbinary Binary
Binary FixedSizedBinary
All others Utf8

Matching of Arrow to ODBC types then inserting

Arrow ODBC
Utf8 VarChar
LargeUtf8 VarChar
Decimal128(p, s = 0) VarChar(p + 1)
Decimal128(p, s != 0) VarChar(p + 2)
Decimal128(p, s < 0) VarChar(p - s + 1)
Decimal256(p, s = 0) VarChar(p + 1)
Decimal256(p, s != 0) VarChar(p + 2)
Decimal256(p, s < 0) VarChar(p - s + 1)
Int8 TinyInt
Int16 SmallInt
Int32 Integer
Int64 BigInt
Float16 Real
Float32 Real
Float64 Double
Timestamp s Timestamp(7)
Timestamp ms Timestamp(7)
Timestamp us Timestamp(7)
Timestamp ns Timestamp(7)
Date32 Date
Date64 Date
Time32 s Time
Time32 ms VarChar(12)
Time64 us VarChar(15)
Time64 ns VarChar(16)
Binary Varbinary
FixedBinary(l) Varbinary(l)
All others Unsupported

The mapping for insertion is not the optimal yet, but before spending a lot of work on improving it I was curious that usecase would pop up for users. So if something does not work, but maybe could provided a better mapping of Arrow to ODBC types, feel free to open an issue. If you do so please give a lot of context of what you are trying to do.

Supported Arrow types

Appart from the afformentioned Arrow types Uint8 is also supported if specifying the Arrow schema directly.


~749K SLoC