125 releases (45 breaking)

0.45.1 Mar 2, 2024
0.44.2 Jan 20, 2024
0.44.0 Dec 31, 2023
0.42.0 Nov 16, 2023
0.3.0 Mar 25, 2022

#20 in Filesystem

Download history 11540/week @ 2023-12-23 15180/week @ 2023-12-30 20094/week @ 2024-01-06 19242/week @ 2024-01-13 22743/week @ 2024-01-20 17597/week @ 2024-01-27 18045/week @ 2024-02-03 12395/week @ 2024-02-10 17968/week @ 2024-02-17 17573/week @ 2024-02-24 20438/week @ 2024-03-02 17655/week @ 2024-03-09 20365/week @ 2024-03-16 19516/week @ 2024-03-23 15352/week @ 2024-03-30 11779/week @ 2024-04-06

70,074 downloads per month
Used in 38 crates (27 directly)

Apache-2.0

3MB
59K SLoC

Apache OpenDAL™

Build Status Latest Version Crate Downloads chat

OpenDAL is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way.

Services

Standard Storage Protocols (like ftp, webdav)
  • ftp: FTP and FTPS
  • http: HTTP read-only services
  • sftp: SFTP services being worked on
  • webdav: WebDAV Service
Object Storage Services (like s3, gcs, azblob)
File Storage Services (like fs, azdls, hdfs)
Consumer Cloud Storage Service (like gdrive, onedrive, icloud drive)
Key-Value Storage Service (like rocksdb, sled)
Cache Storage Service (like memcached, moka)

Welcome to add any services that are not currently supported here.

Features

Access data freely

  • Access different storage services in the same way
  • Behavior tests for all services

Access data painlessly

Access data efficiently

  • Zero cost: Maps to API calls directly
  • Best effort: Automatically selects best read/seek/next based on services
  • Avoid extra calls: Reuses metadata when possible

Quickstart

use opendal::Result;
use opendal::layers::LoggingLayer;
use opendal::services;
use opendal::Operator;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let mut builder = services::S3::default();
    builder.bucket("test");

    // Init an operator
    let op = Operator::new(builder)?
        // Init with logging layer enabled.
        .layer(LoggingLayer::default())
        .finish();

    // Write data
    op.write("hello.txt", "Hello, World!").await?;

    // Read data
    let bs = op.read("hello.txt").await?;

    // Fetch metadata
    let meta = op.stat("hello.txt").await?;
    let mode = meta.mode();
    let length = meta.content_length();

    // Delete
    op.delete("hello.txt").await?;

    Ok(())
}

Examples

The examples are available at here.

Contributing

Check out the CONTRIBUTING guide for more details on getting started with contributing to this project.

License and Trademarks

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.

Dependencies

~9–61MB
~1M SLoC