5 releases
0.1.4 | May 31, 2021 |
---|---|
0.1.3 | Dec 25, 2020 |
0.1.2 | Dec 25, 2020 |
0.1.1 | Dec 18, 2020 |
0.1.0 | Dec 5, 2020 |
#180 in #binary-format
Used in binserde
14KB
357 lines
binserde
A crate similar to serde, but specialized for serializing into a compact binary format, including features like string deduplication.
This crate is very WIP. Features currently not implemented but planned include incremental versioning support so that old formats can still be loaded when the data format changes, deduplication of arbitrary data structures, and explicit tagging (writing a struct or enum as a set of key/value pairs instead of serializing the items in order of declaration, for higher resistance to format changes at the expense of output size)
Usage
use std::fs::File;
use std::io::BufReader;
#[derive(BinSerialize, BinDeserialize, Eq, PartialEq)]
struct MyData {
v1: String,
v2: Option<usize>,
}
let my_data = MyData {
v1: "Some Text".to_string(),
v2: Some(12415165),
};
let vec = binserde::serialize(&my_data).unwrap();
let copy_of_my_data: MyData = binserde::deserialize(&vec).unwrap();
assert_eq!(my_data, copy_of_my_data);
Macro Attributes
#[derive(BinSerialize)]
and #[derive(BinDeserialize)]
allows using
attributes on the type itself and its fields to control (de)serialization.
#[binserde(skip)]
Valid for: fields
Skips the field when serializing. When deserializing, uses
[Default::default()
] instead of reading from the stream to fill the field.
#[binserde(no_dedup)]
Valid for: fields
Turns off deduplication for this field. See [Deduplication] for more information about how it works.
#[binserde(index = n)]
Valid for: fields
not implemented
Moves the field and all following fields to the specified position n
when
serializing, shifting everything originally after that position to the
right.
Example:
#[derive(BinSerialize)]
struct S {
w: u8,
x: u8,
#[binserde(index = 0)]
y: u8,
z: u8,
}
let vec = binserde::serialize(&S { w: 0, x: 1, y: 2, z: 3 });
assert_eq!(&[2, 3, 0, 1], &vec);
The attribute moved y
and z
into position 0, pushing w
and x
back to
positions 2 and 3 respectively.
The attribute can be applied on more than one field, in which case moving operations will be evaluated from top to bottom. That means, the following struct serializes in the order z, x, y, w and not x, y, z, w or any other order:
#[derive(BinSerialize)]
struct S {
w: u8,
#[binserde(index = 0)]
x: u8,
y: u8,
#[binserde(index = 0)]
z: u8,
}
Deduplication
Deduplication is currently only implemented for strings. It works by taking
any String
or [str
] that is serialized using its BinSerializer
implementation and adds it to a seperate list which is written to the
beginning of the buffer given to serialize
(or an equivalent function,
after which the actual data follows. In that data, the string is replaced by
a usize
pointing to the index in the string list. Effectively, a
deduplicated data structure gets transformed from this:
struct S {
s1: String,
s2: String,
strs: Vec<String>,
something_else: u32,
}
to this:
struct S1 {
strings: Vec<String>,
s1: usize,
s2: usize,
strs: Vec<usize>,
something_else: u32,
}
when serializing. This can have a major impact on the resulting size of the serialized data structure when multiple occurrences of the same string appear.
Dependencies
~2MB
~43K SLoC