pyo3-object_store

Integration between object_store and pyo3.

This provides Python builder classes so that Python users can easily create Arc<dyn ObjectStore> instances, which can then be used in pure-Rust code.

Usage

1. Register the builders.

   #[pymodule]
   fn python_module(py: Python, m: &Bound<PyModule>) -> PyResult<()> {
       pyo3_object_store::register_store_module(py, m, "python_module", "store")?;
       pyo3_object_store::register_exceptions_module(py, m, "python_module", "exceptions")?;
   }
   

   This exports the underlying Python classes from your own Rust-Python library.

   Refer to register_store_module and register_exceptions_module for more information.

2. Accept PyObjectStore as a parameter in your function exported to Python. Its into_dyn method (or Into impl) gives you an Arc<dyn ObjectStore>.

   #[pyfunction]
   pub fn use_object_store(store: PyObjectStore) {
       let store: Arc<dyn ObjectStore> = store.into_dyn();
   }
   

   You can also accept AnyObjectStore as a parameter, which wraps PyObjectStore and PyExternalObjectStore. This allows you to seamlessly recreate ObjectStore instances that users pass in from other Python libraries (like obstore) that themselves export pyo3-object_store builders.

   Note however that due to lack of ABI stability, ObjectStore instances will be recreated, and so there will be no connection pooling across the external store.

Example

The obstore Python library gives a full real-world example of using pyo3-object_store, exporting a Python API that mimics the Rust ObjectStore API.

ABI stability

It's not currently possible to share a #[pyclass] across multiple Python libraries, except in special cases where the underlying data has a stable ABI.

As object_store does not currently have a stable ABI, we can't share PyObjectStore instances across multiple separately-compiled Python libraries.

We have two ways to get around this:

• Export your own Python classes so that users can construct ObjectStore instances that were compiled with your library. See register_store_module.
• Accept AnyObjectStore or PyExternalObjectStore as a parameter, which allows for seamlessly reconstructing stores from an external Python library, like obstore. This has some overhead and removes any possibility of connection pooling across the two instances.

Note about not being able to use these across Python packages. It has to be used with the exported classes from your own library.

Python Type hints

We don't yet have a great solution here for reusing the store builder type hints in your own library. Type hints are shipped with the cargo dependency. Or, you can use a submodule on the obstore repo. See async-tiff for an example.

Version compatibility

Note that 0.3.x and 0.4.x are compatibility releases to use pyo3-object_store with older versions of pyo3 and object_store.