34 releases
0.4.5 | Mar 29, 2024 |
---|---|
0.4.4 | Mar 23, 2024 |
0.4.3 | Sep 21, 2023 |
0.3.21 | Sep 5, 2023 |
0.1.0 | Jun 8, 2023 |
#104 in FFI
30KB
644 lines
fcplug
Foreign-Clang-Plugin solution, such as solving rust and go two-way calls.
Features
⇊Caller \ Callee⇉ | Go | Rust |
---|---|---|
Go | - | ✅ |
Rust | ✅ | - |
- Protobuf IDL codec solution: Supported!
- Thrift IDL codec solution: In development...
- No codec solution: In development...
Schematic
Prepare
- Install rust nightly
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup default nightly
- Install go
Download Go
Version go≥1.18
Set environment variables:
CGO_ENABLED=1
- Install protoc
Use protoc v23.2
go install github.com/golang/protobuf/protoc-gen-go@v1.5.3
Example of use
Take Protobuf IDL serialization solution as an example.
See the echo_pb
Step 1: create/prepare a crate
Generally, Fcplug is executed in a Crate's build.sh, and the code is automatically generated to the current Crate.
- If you do not have a Crate, execute the following command to create it:
cargo new --lib {crate_name}
- Add
staticlib
crate-type and some dependent packages, open debug log ofbuild.rs
, edited in Cargo.toml as follows:
[lib]
crate-type = ["rlib", "staticlib"]
[profile.dev.build-override]
opt-level = 0
debug = true
[dependencies]
fcplug = "0.3"
pilota = "0.7.0"
serde = "1"
serde_json = "1"
[build-dependencies]
fcplug-build = "0.3"
Step 2: Write the IDL file that defines the FFI interface
Write the IDL file {ffi_name} .proto in ProtoBuf format, you can put it in the root directory of {crate_name}, the content example is as follows:
syntax = "proto3";
message Ping {
string msg = 1;
}
message Pong {
string msg = 1;
}
// go call rust
service RustFFI {
rpc echo_rs (Ping) returns (Pong) {}
}
// rust call go
service GoFFI {
rpc echo_go (Ping) returns (Pong) {}
}
Step 3: Scripting auto-generated code build.rs
#![allow(unused_imports)]
use fcplug_build::{Config, generate_code, UnitLikeStructPath};
fn main() {
generate_code(Config {
idl_file: "./echo.proto".into(),
// go command dir, default to find from $GOROOT > $PATH
go_root_path: None,
go_mod_parent: "github.com/andeya/fcplug/samples",
target_crate_dir: None,
});
}
Step 4: Preliminary Code Generation
- Execute under the current Crate:
cargo build
# `cargo test` and `cargo install` will also trigger the execution of build.rs to generate code
- Attach the generated src/{ffi_name}_ffi mod to Crate, that is, add mod {ffi_name}_ffi to the
lib.rs
file
Step 5: Implement the FFI interface
- On the rust side, you need to implement the specific trait RustFfi and trait GoFfi methods in the newly initialized
file src/{ffi_name}_ffi/mod.rs.
The complete sample code of the file is as follows:
#![allow(unused_variables)]
pub use echo_pb_gen::*;
use fcplug::{GoFfiResult, TryIntoTBytes};
use fcplug::protobuf::PbMessage;
mod echo_pb_gen;
impl RustFfi for FfiImpl {
fn echo_rs(mut req: ::fcplug::RustFfiArg<Ping>) -> ::fcplug::ABIResult<::fcplug::TBytes<Pong>> {
let _req = req.try_to_object::<PbMessage<_>>();
#[cfg(debug_assertions)]
println!("rust receive req: {:?}", _req);
Pong {
msg: "this is pong from rust".to_string(),
}
.try_into_tbytes::<PbMessage<_>>()
}
}
impl GoFfi for FfiImpl {
#[allow(unused_mut)]
unsafe fn echo_go_set_result(mut go_ret: ::fcplug::RustFfiArg<Pong>) -> ::fcplug::GoFfiResult {
#[cfg(debug_assertions)]
return GoFfiResult::from_ok(go_ret.try_to_object::<PbMessage<_>>()?);
#[cfg(not(debug_assertions))]
return GoFfiResult::from_ok(go_ret.bytes().to_owned());
}
}
- Implement the go GoFfi interface in the one-time generated file ./cgobin/clib_goffi_impl.go.
The complete sample code of this file is as follows:
package main
import (
"fmt"
"github.com/andeya/fcplug/samples/echo_pb"
"github.com/andeya/gust"
)
func init() {
// TODO: Replace with your own implementation, then re-execute `cargo build`
GlobalGoFfi = GoFfiImpl{}
}
type GoFfiImpl struct{}
func (g GoFfiImpl) EchoGo(req echo_pb.TBytes[echo_pb.Ping]) gust.EnumResult[echo_pb.TBytes[*echo_pb.Pong], ResultMsg] {
_ = req.PbUnmarshalUnchecked()
fmt.Printf("go receive req: %v\n", req.PbUnmarshalUnchecked())
return gust.EnumOk[echo_pb.TBytes[*echo_pb.Pong], ResultMsg](echo_pb.TBytesFromPbUnchecked(&echo_pb.Pong{
Msg: "this is pong from go",
}))
}
Step 6: Generate Final Code
Execute cargo build
cargo test
or cargo install
under the current Crate, trigger the execution of build.rs, and
generate code.
Note: When GoFfi is defined, after compiling or changing the code for the first time, a warning similar to the following will occur, and you should execute cargo build twice at this time
warning: ... to re-execute 'cargo build' to ensure the correctness of 'libgo_echo.a'
Therefore, it is recommended to repeat cargo build three times directly in the build.sh
script
#!/bin/bash
cargo build --release
cargo build --release
cargo build --release
Step 7: Testing
- Rust calls Go tests, you can add test functions in
lib.rs
,
the sample code is as follows:
#![feature(test)]
extern crate test;
mod echo_pb_ffi;
#[cfg(test)]
mod tests {
use test::Bencher;
use fcplug::protobuf::PbMessage;
use fcplug::TryIntoTBytes;
use crate::echo_pb_ffi::{FfiImpl, GoFfiCall, Ping, Pong};
#[test]
fn test_call_echo_go() {
let pong = unsafe {
FfiImpl::echo_go::<Pong>(Ping {
msg: "this is ping from rust".to_string(),
}.try_into_tbytes::<PbMessage<_>>().unwrap())
};
println!("{:?}", pong);
}
#[bench]
fn bench_call_echo_go(b: &mut Bencher) {
let req = Ping {
msg: "this is ping from rust".to_string(),
}
.try_into_tbytes::<PbMessage<_>>()
.unwrap();
b.iter(|| {
let pong = unsafe { FfiImpl::echo_go::<Vec<u8>>(req.clone()) };
let _ = test::black_box(pong);
});
}
}
- Go calls Rust test, add the file
go_call_rust_test.go
in the root directory,
the sample code is as follows:
package echo_pb_test
import (
"testing"
"github.com/andeya/fcplug/samples/echo_pb"
)
func TestEcho(t *testing.T) {
ret := echo_pb.GlobalRustFfi.EchoRs(echo_pb.TBytesFromPbUnchecked[*echo_pb.Ping](&echo_pb.Ping{
Msg: "this is ping from go",
}))
if ret.IsOk() {
t.Logf("%#v", ret.PbUnmarshalUnchecked())
} else {
t.Logf("fail: err=%v", ret.AsError())
}
ret.Free()
}
Asynchronous programming
- Rust Tokio asynchronous function calling Go synchronous function
use fcplug::protobuf::PbMessage;
use fcplug::TryIntoTBytes;
use fcplug-build::task;
use crate::echo_ffi::{FfiImpl, GoFfiCall, Ping, Pong};
let pong = task::spawn_blocking(move | | {
// The opened task runs in a dedicated thread pool.
// If this task is blocked, it will not affect the completion of other tasks
unsafe {
FfiImpl::echo_go::< Pong > (Ping {
msg: "this is ping from rust".to_string(),
}.try_into_tbytes::< PbMessage < _ > > ().unwrap())
}
}).await?;
- Go calls Rust, at least one side is an async function
in development
Benchmark
goos: darwin
goarch: amd64
pkg: github.com/andeya/fcplug/demo
cpu: Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
Dependencies
~9–17MB
~192K SLoC