#python #object #serialization #deserialize #communication #py #options

pyany-serde

Serialization and deserialization for Python objects

2 releases

new 0.1.1 Feb 1, 2025
0.1.0 Feb 1, 2025

#1067 in Encoding

Apache-2.0

91KB
2.5K SLoC

A Rust crate which defines a trait for serializing and deserializing Python objects. Because this trait does not have the 'py lifetime specifier, trait objects can be stored in Python classes (see DynPyAnySerde). This allows for the different PyAnySerde implementations to be created and bound in the Python interpreter. There are a variety of implementations provided for standard Python objects, and there is a PythonSerde abstract Python class which can be implemented and turned into a Box for custom composition with other instances of Box (such as that for List) without needing to write any Rust code. The constructors for these are exposed via DynPyAnySerdeFactory.

The use case for this crate is demonstrated in rlgym-learn - the user writes Python code which instantiates a struct defining Rust serialization and deserialization strategies (the PyAnySerde trait object, held inside a DynPyAnySerde) for a given generic type. In Rust, these DynPyAnySerdes can be received as function parameters and used via the append_python and retrieve_python functions in the communication module. These functions are designed to work with each other and allow for the Box to be wrapped in an Option. In the case that the Option is None, the Box to be used is dynamically determined based on the object provided for serialization, and the information needed to reconstruct this Box when deserializing is passed in the buffer (except in some cases where the information needed for reconstruction is too complex, such as UnionSerde, PythonSerdeSerde, or some other struct implementing Box not defined in this crate). Some structs (like NumpyDynamicShapeSerde) implementing the PyAnySerde trait rely on alignment for deserialization, and so these append_python and retrieve_python functions must write directly to the buffer so that the alignment is known.

Dependencies

~7–32MB
~450K SLoC