#file-storage #internet-computer #icp #object-storage #oss #s3 #storage-service

ic-oss-can

A Rust library for implementing large file storage in ICP canisters

12 unstable releases (3 breaking)

new 0.9.3 Oct 11, 2024
0.9.0 Oct 9, 2024
0.8.1 Sep 27, 2024
0.8.0 Aug 29, 2024
0.6.6 Jul 17, 2024

#633 in Web programming

Download history 293/week @ 2024-07-12 22/week @ 2024-07-19 11/week @ 2024-07-26 76/week @ 2024-08-02 149/week @ 2024-08-09 175/week @ 2024-08-16 296/week @ 2024-08-23 31/week @ 2024-08-30 28/week @ 2024-09-13 27/week @ 2024-09-20 202/week @ 2024-09-27 124/week @ 2024-10-04

381 downloads per month

MIT/Apache

89KB
2K SLoC

ic-oss-can

License Crates.io CI Docs.rs Latest Version

ic-oss is a decentralized Object Storage Service on the Internet Computer.

ic-oss-can is a Rust library for implementing large file storage in ICP canisters. By including the ic_oss_fs! macro in your canister, a fs module and a set of Candid filesystem APIs will be automatically generated. You can use the ic-oss-cli tool to upload files to the ICP canister.

Usage

The following example is a minimal version using the ic_oss_fs! macro. Its only dependency is a thread-local constant named FS_CHUNKS_STORE of type RefCell<StableBTreeMap<FileId, Chunk, Memory>>.

use ic_stable_structures::{
    memory_manager::{MemoryId, MemoryManager, VirtualMemory},
    DefaultMemoryImpl, StableBTreeMap,
};
use std::cell::RefCell;

use ic_oss_can::ic_oss_fs;
use ic_oss_can::types::{Chunk, FileId, FileMetadata};

type Memory = VirtualMemory<DefaultMemoryImpl>;

const FS_DATA_MEMORY_ID: MemoryId = MemoryId::new(0);

thread_local! {

    static MEMORY_MANAGER: RefCell<MemoryManager<DefaultMemoryImpl>> =
        RefCell::new(MemoryManager::init(DefaultMemoryImpl::default()));


    // `FS_CHUNKS_STORE`` is needed by `ic_oss_can::ic_oss_fs!` macro
    static FS_CHUNKS_STORE: RefCell<StableBTreeMap<FileId, Chunk, Memory>> = RefCell::new(
        StableBTreeMap::init(
            MEMORY_MANAGER.with_borrow(|m| m.get(FS_DATA_MEMORY_ID)),
        )
    );
}

// need to define `FS_CHUNKS_STORE` before `ic_oss_can::ic_oss_fs!()`
ic_oss_fs!();

For a more complete example, refer to examples/ai_canister.

FS Module

fs::set_max_file_size(size: u64);
fs::set_visibility(visibility: u8);
fs::set_managers(managers: BTreeSet<Principal>);
fs::is_manager(caller: &Principal) -> bool;
fs::with<R>(f: impl FnOnce(&Files) -> R) -> R;
fs::load();
fs::save();
fs::get_file(id: u32) -> Option<FileMetadata>;
fs::list_files(prev: u32, take: u32) -> Vec<FileInfo>;
fs::add_file(file: FileMetadata) -> Result<u32, String>;
fs::update_file(change: UpdateFileInput, now_ms: u64) -> Result<(), String>;
fs::get_chunk(id: u32, chunk_index: u32) -> Option<FileChunk>;
fs::get_full_chunks(id: u32) -> Result<Vec<u8>, String>;
fs::update_chunk(id: u32, chunk_index: u32, now_ms: u64, chunk: Vec<u8>) -> Result<u64, String>;
fs::delete_file(id: u32) -> Result<bool, String>;

FS Candid API

create_file : (CreateFileInput, opt blob) -> (Result_2);
delete_file : (nat32, opt blob) -> (Result_3);
list_files : (nat32, opt nat32, opt nat32, opt blob) -> (Result_4) query;
update_file_chunk : (UpdateFileChunkInput, opt blob) -> (Result_6);
update_file_info : (UpdateFileInput, opt blob) -> (Result_7);

The complete module API Candid API definition can be found in the store.rs file.

License

Copyright © 2024 LDC Labs.

ldclabs/ic-oss is licensed under the MIT License. See LICENSE for the full license text.

Dependencies

~9–18MB
~283K SLoC