pyo3_bindgen

Automatic generation of Rust FFI bindings to Python modules via PyO3. Python modules are analyzed recursively to generate Rust bindings with an identical structure for all public classes, functions, properties, and constants. Any available docstrings and type annotations are also preserved in their Rust equivalents.

An example of a generated Rust function signature and its intended usage is shown below. Of course, manually wrapping parts of the generated bindings in a more idiomatic Rust API might be beneficial in most cases.

Source code (Python)	Generated code (Rust)
def answer_to(question: str) -> int: """Returns answer to a question.""" return 42     def main(): assert answer_to("life") == 42 if __name__ == "__main__": main()	/// Returns answer to a question. pub fn answer_to<'py>( py: ::pyo3::Python<'py>, question: &str, ) -> ::pyo3::PyResult<i64> { ... // Calls function via `pyo3` } pub fn main() -> pyo3::PyResult<()> { pyo3::Python::with_gil(|py| { assert_eq!( answer_to(py, "universe")?, 42 ); Ok(()) }) }

This project is intended to simplify the integration or transition of existing Python codebases into Rust. You, as a developer, gain immediate access to the Rust type system and countless other benefits of modern compiled languages with the generated bindings. Furthermore, the entire stock of high-quality crates from crates.io becomes at your disposal.

On its own, the generated Rust code does not provide any performance benefits over using the Python code. However, it can be used as a starting point for further optimization if you decide to rewrite performance-critical parts of your codebase in pure Rust.

Overview

The workspace contains these packages:

• pyo3_bindgen: Public API for generation of bindings (in build.rs or via procedural macros)
• pyo3_bindgen_cli: CLI tool for generation of bindings via pyo3_bindgen executable
• pyo3_bindgen_engine: The underlying engine for generation of bindings
• pyo3_bindgen_macros: Procedural macros for in-place generation

Instructions

Option 1: Build script

First, add pyo3_bindgen as a build dependency to your Cargo.toml manifest. To actually use the generated bindings, you will also need to add pyo3 as a regular dependency (or use the re-exported pyo3_bindgen::pyo3 module).

[build-dependencies]
pyo3_bindgen = { version = "0.4" }

[dependencies]
pyo3 = { version = "0.20", features = ["auto-initialize"] }

Then, create a build.rs script in the root of your crate that generates bindings to the selected Python modules. In this example, the bindings are simultaneously generated for the "os", "posixpath", and "sys" Python modules. At the end of the generation process, the Rust bindings are written to ${OUT_DIR}/bindings.rs.

With this approach, you can also customize the generation process via pyo3_bindgen::Config that can be passed to the constructor, e.g. Codegen::new(Config::builder().include_private(true).build()).

//! build.rs
use pyo3_bindgen::Codegen;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    Codegen::default()
        .module_names(["os", "posixpath", "sys"])?
        .build(format!("{}/bindings.rs", std::env::var("OUT_DIR")?))?;
    Ok(())
}

Afterwards, you can include the generated Rust code via the include! macro anywhere in your crate and use the generated bindings as regular Rust modules. However, the bindings must be used within the pyo3::Python::with_gil closure to ensure that Python GIL is held.

//! src/main.rs
include!(concat!(env!("OUT_DIR"), "/bindings.rs"));

fn main() -> pyo3::PyResult<()> {
    pyo3::Python::with_gil(|py| {
        // Get the path to the Python executable via "sys" Python module
        let python_exe_path = sys::executable(py)?;
        // Get the current working directory via "os" Python module
        let current_dir = os::getcwd(py)?;
        // Get the relative path to the Python executable via "posixpath" Python module
        let relpath_to_python_exe = posixpath::relpath(py, python_exe_path, current_dir)?;

        println!("Relative path to Python executable: '{relpath_to_python_exe}'");
        Ok(())
    })
}

Option 2: Procedural macros (experimental)

As an alternative to build scripts, you can use procedural macros to generate the bindings in-place. First, add pyo3_bindgen_macros as a regular dependency to your Cargo.toml manifest.

[dependencies]
pyo3_bindgen = { version = "0.4" }

Subsequently, the import_python! macro can be used to generate Rust bindings for the selected Python modules anywhere in your crate. As demonstrated in the example below, Rust bindings are generated for the "math" Python module and can directly be used in the same scope. Similar to the previous approach, the generated bindings must be used within the pyo3::Python::with_gil closure to ensure that Python GIL is held.

As opposed to using build scripts, this approach does not offer the same level of customization via pyo3_bindgen::Config. Furthermore, the procedural macro is quite experimental and might not work in all cases.

use pyo3_bindgen::import_python;
import_python!("math");

// Which Pi do you prefer?
// a) 🐍 Pi from Python "math" module
// b) 🦀 Pi from Rust standard library
// c) 🥧 Pi from your favourite bakery
pyo3::Python::with_gil(|py| {
    let python_pi = math::pi(py).unwrap();
    let rust_pi = std::f64::consts::PI;
    assert_eq!(python_pi, rust_pi);
})

Option 3: CLI tool

For a quick start and testing purposes, you can use the pyo3_bindgen executable to generate and inspect bindings for the selected Python modules. The executable is available as a standalone package and can be installed via cargo.

cargo install --locked pyo3_bindgen_cli

Afterwards, run the pyo3_bindgen executable to generate Rust bindings for the selected Python modules. The generated bindings are printed to STDOUT by default, but they can also be written to a file via the -o option (see pyo3_bindgen --help for more options).

pyo3_bindgen -m os sys numpy -o bindings.rs

Status

This project is in early development, and as such, the API of the generated bindings is not yet stable.

• The binding generation is primarily designed to be used inside build scripts or via procedural macros. Therefore, the performance of the codegen process is benchmarked to understand the potential impact on build times. Here are some preliminary results for version 0.3 with the default configuration (measured: parsing IO & codegen | not measured: compilation of the generated bindings, which takes much longer):
  • sys: 1.24 ms (0.66k total LoC)
  • os: 8.38 ms (3.88k total LoC)
  • numpy: 1.02 s (294k total LoC)
  • torch: 7.05 s (1.08M total LoC)
• The generation of bindings should never panic as long as the target Python module can be successfully imported. If it does, please report this as a bug.
• The generated bindings should always be compilable and usable in Rust. If you encounter any issues, consider manually fixing the problematic parts of the bindings and please report this as a bug.
• However, the generated bindings are based on the introspection of the target Python module. Therefore, the completeness and correctness of the generated bindings are directly dependent on the quality of the module structure, type annotations and docstrings in the target Python module. Ideally, the generated bindings should be considered unsafe and serve as a starting point for safe and idiomatic Rust APIs. If you find that something in the generated bindings is incorrect or missing, please report this as well.
• Not all Python types are mapped to their Rust equivalents yet. For this reason, some additional type-casting might be required when using the generated bindings (e.g. let typed_value: MyType = any_value.extract()?;).
• Although implemented, the procedural macro might not work in many cases. Therefore, it is recommended that the build scripts be used wherever possible.

License

This project is dual-licensed to be compatible with the Rust project, under either the MIT or Apache 2.0 licenses.

Contributing

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.