#serialization #encode #url

docchi_compaction

A serializer which compresses integers

3 unstable releases

0.8.0 Dec 8, 2021
0.7.3 Dec 3, 2021
0.7.0 Nov 21, 2021

#2281 in Encoding

Download history 10/week @ 2024-04-11 9/week @ 2024-04-18 13/week @ 2024-04-25 10/week @ 2024-05-02 14/week @ 2024-05-09 9/week @ 2024-05-16 7/week @ 2024-05-23 8/week @ 2024-05-30 10/week @ 2024-06-06 12/week @ 2024-06-13 15/week @ 2024-06-20 7/week @ 2024-06-27 11/week @ 2024-07-04 10/week @ 2024-07-11 12/week @ 2024-07-18 30/week @ 2024-07-25

63 downloads per month
Used in 6 crates (3 directly)

MIT/Apache

50KB
1K SLoC

A serializer which serializes values with dynamic types and compresses integers, bits, and strings which represents decimals.

Even if a variable's size is 8 byte, when the actual value is 0 or 1, this library can compress it to two bits.

This is a base component of the Docchi File System.

pub struct SomeCollection{
    vec : Vec<SomeItem>
}

impl SomeCollection{
    pub fn encode<W : std::io::Write>(&self, write : &mut W) -> Result<()>{
    
        //KVal is the value type of this crate
        let mut vec : Vec<docchi_compaction::KVal> = vec![];

        //comp_int compresses integers considering actual size. 0,1 -> Bit. -128 to 127 -> Byte...
        vec.push(comp_int(self.len() as i64));  
        
        for item in self.vec.iter(){
            item.write(&mut vec);
        }

        //encode values to bytes
        docchi_compaction::encode(&vec, write)?;
        Ok(())
    }
    
    pub fn decode<R : std::io::Read>(read : &mut R) -> Result<Self>{
        //decode to Vec<KVal>
        let (kvals,_) = docchi_compaction::decode(read)?;
        
        let mut iter = kvals.iter();
        let len = iter.next()?.as_i64()? as usize;
        
        let mut vec : Vec<SomeItem> = Vec::with_capacity(len);

        for _ in 0..len{
            vec.push(SomeItem::read(&mut iter)?)
        }

        Ok(SomeCollection{ vec })
    }
}

pub struct SomeItem{
    index : usize,
    value : u8,
}

impl SomeItem{
    pub fn write(&self, vec : &mut Vec<KVal>){
        vec.push(comp_int(self.index as i64));  
        vec.push(comp_int(self.value as i64));
    }
    pub fn read<'a>(iter : &mut impl Iterator<Item=&'a KVal>) -> Result<SomeItem>{ //This Result implements From<NoneError>
        let index = iter.next()?.as_i64()? as usize; 
        let value = iter.next()?.as_i64()? as u8;

        Ok(SomeItem{ index, value })
    }
}

changelog

0.4.0

changed encode of the values "undefined".

added Binary, Binary8, Binary4, and Binary2 types.

Dependencies

~2.5–3.5MB
~59K SLoC