3 releases
0.1.2 | Nov 1, 2023 |
---|---|
0.1.1 | Oct 31, 2023 |
0.1.0 | Oct 31, 2023 |
#852 in Rust patterns
16KB
118 lines
Proto Mapper
Macro implementation library for mapping between custom models and protobuf generated code
Notice
This library is an (almost) complete rewrite of the protobuf-convert library.
The purpose of the rewrite is to adapt it to specific needs of our projects.
The main concept and the idea remains the same, so the credit goes to the original authors of the protobuf-convert
library.
What changed
This library:
- changes the main name of the macro to
ProtoMap
- changes the main way the macro is used and is interfaced with external traits
- avoids the use of re implementing ProtoMap trait to client modules
- is restructured to different crates
- contains excessive testing for edge cases
- introduces ProtoScalar types
- introduces ProtoScalarMap trait for protobuf scalar types
- handles enumeration protobuf generation code automatically
- handles option values via scanning the types of the applied struct and chooses different implementation paths
- supports prost
Install
First, add the dependency in Cargo.toml
:
proto-mapper = {version = "0.1.2", features = ["protobuf"] }
or
proto-mapper = {version = "0.1.2", features = ["prost"] }
NOTE: Features prost
or protobuf
are mutually exclusive and required.
Use one of them according to targeted generated code proto framework that you use
Usage
A proof of concept that demonstrates the use of this library can be found here. Keep in mind that the PoC is still work in progress.
Mapping scalar values and enumerations
Given the protobuf enumeration and message
syntax = "proto3";
enum EntityStatus {
STATUS_A = 0;
STATUS_B = 1;
STATUS_C = 2;
}
message ScalarEntity {
uint32 uint32_f = 1;
int32 int32_f= 2;
bool bool_f = 4;
string string_f = 5;
int64 int64_f = 6;
uint64 uint64_f = 7;
bytes bytes_f = 8;
float float_f = 9;
double double_f = 10;
EntityStatus status = 11;
}
After using prost
or rust-protobuf
library to generate code, you can map your custom model to the generated structs as follows:
#[derive(Debug, Clone, Copy, Default, PartialEq, ProtoMap)]
#[proto_map(
source = "proto::EntityStatus",
enumeration,
)]
enum EntityStatus {
#[default]
StatusA,
StatusB,
StatusC,
}
#[derive(Debug, ProtoMap, Default)]
#[proto_map(source = "proto::ScalarEntity")]
struct ScalarEntity {
pub uint32_f: u32,
pub int32_f: i32,
pub bool_f: bool,
pub string_f: String,
pub bytes_f: Vec<u8>,
pub int64_f: i64,
pub uint64_f: u64,
pub float_f: f32,
pub double_f: f64,
#[proto_map(enumeration)]
pub status: EntityStatus,
}
Then you can convert between your defined struct and the generated code as follows:
let e = ScalarEntity::default();
let p = e.to_proto();
You can also convert a proto instance to your custom struct.
let p = proto::ScalarEntity::default();
let e = ScalarEntity::from_proto(p)?;
Note that the mapping code for the enumeration requires #[proto_map(..., enumeration)]
attribute on the rust enumeration
and also needs to mark the field inside the ScalarEntity
as well.
Mapping optional scalar values and enumerations
Given the same proto file. Out of the box you can map to optional values
That is:
#[derive(Debug, ProtoMap, PartialEq, Default)]
#[proto_map(source = "proto::prost::ScalarEntity")]
struct ScalarEntityOptions {
pub uint32_f: Option<u32>,
pub int32_f: Option<i32>,
pub bool_f: Option<bool>,
pub string_f: Option<String>,
pub bytes_f: Option<Vec<u8>>,
pub int64_f: Option<i64>,
pub uint64_f: Option<u64>,
pub float_f: Option<f32>,
pub double_f: Option<f64>,
#[proto_map(enumeration)]
pub status: Option<EntityStatus>,
}
The macro scans the types of the custom struct that annotates and chooses different implementation paths for the conversion code.
Mapping non scalar values
Given the proto file
syntax = "proto3";
// ... definitions of ScalarEntity
message NestedEntity {
ScalarEntity first = 1;
ScalarEntity second = 2;
}
You can map non scalar values as follows
#[derive(Debug, ProtoMap, PartialEq)]
#[proto_map(source = "proto::NestedEntity")]
struct NestedEntity {
pub first: ScalarEntity,
pub second: Option<ScalarEntity>,
}
Mapping non scalar oneof
field to rust enumeration
You can map top level oneof
protobuf fields as follows
Given the proto file
syntax = "proto3";
// ... definitions of ScalarEntity
message HierarchyEntity {
oneof data {
ScalarEntity first_entity = 1;
NestedEntity second_entity = 2;
}
}
Then one implementation of the custom struct may be
#[derive(Debug, ProtoMap, PartialEq)]
#[proto_map(
source = "proto::HierarchyEntity",
one_of(field = "data"),
rename_variants = "snake_case"
)]
enum HierarchyEntity {
FirstEntity(ScalarEntity),
SecondEntity(NestedEntity),
}
Note that the rename_variants
attribute may take two values snake_case
and STREAMING_SNAKE_CASE
according to the target generated struct.
Custom mapping scalar values
See examples at tests for prost and for rust-protobuf
Differences between prost
and rust-protobuf
usage
TODO
How it works
Internally the macro discriminates between scalar and not scalar types.
Scalar types are shown below as well as the protobuf types that rust-protobuf
and prost
autogenerated code maps to.
Protobuf Type | Rust Type |
---|---|
double |
f64 |
float |
f32 |
int32 |
i32 |
int64 |
i64 |
uint32 |
u32 |
uint64 |
u64 |
sint32 |
i32 |
sint64 |
i64 |
fixed32 |
u32 |
fixed64 |
u64 |
sfixed32 |
i32 |
sfixed64 |
i64 |
bool |
bool |
string |
String |
bytes |
Vec<u8> |
(table taken from prost project README.md) |
All other rust types are considered as non scalar.
An exception to that rule is protobuf enum
types that need to be marked with meta attribute #[proto_map(enumeration)]
,
The library implements automatically two traits according to the struct types.
For scalar types:
pub trait ProtoMapScalar<P: ProtoScalar>: Sized {
/// Converts a reference of [`Self`] to a [`ProtoScalar`]
fn to_scalar(&self) -> P;
/// Consumes a [`ProtoScalar`] and returns a [`Self`] or error in the conversion failed
fn from_scalar(proto: P) -> Result<Self, anyhow::Error>;
}
For non scalar types:
pub trait ProtoMap
where
Self: Sized,
{
type ProtoStruct;
/// Converts a reference of [`Self`] struct to proto [`Self::ProtoStruct`]
fn to_proto(&self) -> Self::ProtoStruct;
/// Consumes a proto [`Self::ProtoStruct`] and returns a [`Self`] struct or error in the conversion failed
fn from_proto(proto: Self::ProtoStruct) -> Result<Self, anyhow::Error>;
}
Note that protobuf enum
types are treated as non scalar with rust-protobuf
but as scalar (i32 value) with prost
.
Also a third trait named ProtoScalar
implementation is provided by the library for all proto scalar types.
To get a rough view of what the macro implement see the prost manual tests or the rust-protobuf manual tests used as guides for creating the implementations.
Limitations
TODO
Related Projects
Resources
Dependencies
~0.8–3MB
~56K SLoC