#archive #filesystem #compression #game #assets

vach

A simple archiving format, designed for storing assets in compact secure containers

8 releases

Uses new Rust 2021

new 0.2.3 Nov 23, 2021
0.2.2 Nov 19, 2021
0.1.9 Oct 24, 2021
0.1.3 Sep 11, 2021

#32 in Compression

Download history 12/week @ 2021-09-07 2/week @ 2021-09-14 16/week @ 2021-09-28 17/week @ 2021-10-05 7/week @ 2021-10-12 15/week @ 2021-10-19 3/week @ 2021-10-26 12/week @ 2021-11-02 24/week @ 2021-11-09 44/week @ 2021-11-16 23/week @ 2021-11-23

103 downloads per month
Used in vach-cli

GPL-2.0 license

70KB
1.5K SLoC

.vach logo

vach

A simple archiving format, designed for storing assets in compact secure containers

STABILITY WARNING! Some crates in this repo are in very early development, APIs and standards are expected to change!

docs.rs Crate Version on Crates.io
LISCENCE: GPL 2.0 GitHub Build and Test actions GitHub issues

Docs | Repo

👔 The official vach and vf crates' repo

vach, pronounced like "puck" but with a "v", is a archiving and resource transmission format. It was built to be secure, contained and protected. It was, in fact, designed by the SCP to keep your anomalous assets compact and secure during transmission. vach also has in-built support for compression, data signing, leaf bitflags, encryption and archive customization. Check out the vach spec at spec.txt. Any and all help will be much appreciated, especially proof reading the docs and code review.


🤷 Who is what, when where?

  • vach: An archiving format, like tar, zip and rar. Also the base crate for handling .vach files in your application.
  • vf: An asset management system utilizing vach for transmission and adds features like hot reloading, caching and archive management. WIP 🚧
  • vach-cli: A CLI tool for dealing with .vach files. WIP 🚧

👄 Terminologies

  • Archive Source: Any source of data. That implements io::Seek and io::Read, for example a file (fs::File) or io::Cursor, that matches the vach spec and is thus a valid archive.
  • Leaf: Any actual data endpoint within an archive, for example footstep1.wav in sounds.vach.
  • Entry: Some data in the registry section of a vach source on an corresponding leaf. For example, { id: footstep.wav, location: 45, offset: 2345, flags: 0b0000_0000_0000_0000u16 }.

🀄 Show me some code dang it!

> Building a basic unsigned .vach file
use std::{io::Cursor, fs::File};
use vach::prelude::{Builder, BuilderConfig};

let config = BuilderConfig::default();
let mut builder = Builder::default();

// Use `Builder::add( reader, ID )` to add data to the write queue
builder.add(File::open("test_data/background.wav")?, "ambient").unwrap();
builder.add(File::open("test_data/footstep.wav")?, "ftstep").unwrap();
builder.add(Cursor::new(b"Hello, Cassandra!"), "hello").unwrap();

// let mut target = File::create("sounds.vach")?;
let mut target = Cursor::new(Vec::new());

// The number of bytes written to the file
let size = builder.dump(&mut target, &config).unwrap();
> Loading resources from an unsigned .vach file
use std::fs::File;
use vach::prelude::{Archive, Resource, Flags};

let target = File::open("sounds.vach")?;
let mut archive = Archive::from_handle(target)?;
let resource: Resource = archive.fetch("ambient")?;

// By default all resources are flagged as NOT secured
println!("{}", Sound::new(&resource.data)?);
assert!(!resource.secured);

let mut buffer = Vec::new();
let (flags, content_version, is_secure) = archive.fetch_write("ftstep", &mut buffer)?;
> Build a signed .vach file
use std::{io::Cursor, fs::File};
use vach::prelude::{Builder, BuilderConfig, Keypair};
use vach::utils::gen_keypair;

let keypair: Keypair = gen_keypair();
let config: BuilderConfig = BuilderConfig::default().keypair(keypair);
let mut builder = Builder::default();

// Use `Builder::add( reader, ID )` to add data to the write queue
builder.add(File::open("test_data/background.wav")?, "ambient").unwrap();
builder.add(File::open("test_data/footstep.wav")?, "ftstep").unwrap();
builder.add(Cursor::new(b"Hello, Cassandra!"), "hello").unwrap();

// let mut target = File::create("sounds.vach")?;
let mut target = Cursor::new(Vec::new());

builder.dump(&mut target, &config).unwrap();
> Serialize and de-serialize a Keypair, SecretKey and PublicKey

As Keypair, SecretKey and PublicKey are reflected from ed25519_dalek, you could refer to their docs to read further about them.

use vach;
use vach::prelude::{Keypair, SecretKey, PublicKey};
use vach::utils::gen_keypair;

// Generate keys
let keypair : Keypair  = gen_keypair();
let secret : SecretKey = keypair.secret;
let public : PublicKey = keypair.public;

// Serialize
let public_key_bytes : [u8; vach::PUBLIC_KEY_LENGTH] = public.to_bytes();
let secret_key_bytes : [u8; vach::SECRET_KEY_LENGTH] = secret.to_bytes();
let keypair_bytes : [u8; vach::KEYPAIR_LENGTH]    = keypair.to_bytes();

// Deserialize
let public_key : PublicKey = PublicKey::from_bytes(&public_key_bytes).unwrap();
let secret_key : SecretKey = SecretKey::from_bytes(&secret_key_bytes).unwrap();
let keypair : Keypair   = Keypair::from_bytes(&keypair_bytes).unwrap();
> Load resources from a signed .vach source
// Load public_key
let mut public_key = File::open(PUBLIC_KEY)?;
let mut public_key_bytes: [u8; crate::PUBLIC_KEY_LENGTH];
public_key.read_exact(&mut public_key_bytes)?;

// Build the Loader config
let mut config = HeaderConfig::default().key(PublicKey::from_bytes(&public_key_bytes)?);

let target = File::open("sounds.vach")?;
let mut archive = Archive::with_config(target, &config)?;

// Resources are marked as secure (=true) if the signatures match the data
let resource = archive.fetch("ambient")?;
println!("{}", Sound::new(&resource.data)?);
assert!(resource.secured);
> A quick consolidated example
const MAGIC: &[u8; 5] = b"CSDTD";
let mut target = Cursor::new(Vec::<u8>::new());

// Data to be written
let data_1 = b"Around The World, Fatter better stronker" as &[u8];
let data_2 = b"Imagine if this made sense" as &[u8];
let data_3 = b"Fast-Acting Long-Lasting, *Bathroom Reader*" as &[u8];

// Builder definition
let mut builder = Builder::new();
let config = BuilderConfig::default().magic(*MAGIC);

// Add data
builder.add_leaf(Leaf::from_handle(data_1).id("d1").compress(CompressMode::Always))?;
builder.add_leaf(Leaf::from_handle(data_2).id("d2").compress(CompressMode::Never))?;
builder.add_leaf(Leaf::from_handle(data_3).id("d3").compress(CompressMode::Detect))?;

// Dump data
builder.dump(&mut target, &config)?;

// Load data
let config = HeaderConfig::default().magic(*MAGIC);
let mut archive = Archive::with_config(target, &config)?;

// Quick assertions
assert_eq!(archive.fetch("d1")?.data.as_slice(), data_1);
assert_eq!(archive.fetch("d2")?.data.as_slice(), data_2);
assert_eq!(archive.fetch("d3")?.data.as_slice(), data_3);

For more information on how to use the library, read the documentation. Always read the documentation! And pass by the examples folder( not yet implemented ).


🛠 Yet to be implemented

  • An official CLI.
  • Data encryption.
  • Benchmarks.
  • Skynet, (coming very soon).
  • Some(examples) directory instead of None

Dependencies

~3.5MB
~76K SLoC