7 unstable releases (3 breaking)
0.4.0 | Jun 25, 2019 |
---|---|
0.3.3 | Jun 14, 2019 |
0.2.0 | Jun 13, 2019 |
0.1.0 | Jun 6, 2019 |
#28 in #msgpack
53 downloads per month
145KB
4K
SLoC
lavish-compiler
lavish
lets you declare services, and implement/consume them
easily from a variety of languages.
It is opinionated:
- It has its own schema language & compiler (written in Rust)
- It targets (for now) Rust, Go, and TypeScript
- It comes with its own RPC runtime for Rust
- It's designed with "bidirectional framed MessagePack-RPC over TCP" in mind
Status
lavish is still under development, it is not usable yet:
- Prerelease: rules language, schema language
- Prerelease: parser, checker
- Rust
- Go
- Researched only: runtime
- Not started: codegen
- TypeScript
- Researched only: runtime
- Not started: codegen
Schemas
Schemas can define "functions", that take arguments and return results.
server fn log(message: string)
server fn get_version() -> (major: i64, minor: i64, patch: i64)
server fn shutdown()
All input parameters and output parameters (results) are named.
Functions can be namespaced:
namespace utils {
server fn log()
server fn get_version()
}
namespace system {
server fn shutdown()
}
Functions can be implemented by the server or the client:
namespace session {
// try to log in. if password authentication is
// enabled, @get_password is called.
server fn login(username: string)
client fn get_password(username: string) -> (password: string)
server fn logout()
}
Built-in types (lowercase) are:
i8
,u16
,u32
,u64
: Unsigned integeri8
,i16
,i32
,i64
: Signed integer- Note that JavaScript only has 53-bit precision.
f32
,f64
: Floating-point numberbool
: Booleanstring
: UTF-8 stringdata
: Raw byte arraytimestamp
: UTC date + time
Custom types can be declared, those should be CamelCase
:
enum LoginType {
Anonymous = "anonymous",
Password = "password",
}
struct Session {
login_type: LoginType,
connected_at: timestamp,
}
namespace session {
server fn login() -> (session: Session)
// etc.
}
By default, all fields must be specified - there are no default
values. However, fields can be made optional with option<T>
:
// password can be None in Rust, nil in Go, undefined in TypeScript
server fn login(password: option<string>)
Arrays are declared with array<T>
:
server fn login(ciphers: array<Cipher>)
Maps are declared with map<K, V>
:
server fn login(options: map<string, string>)
option
, map
, and array
can be nested:
server fn login(options: option<map<string, string>>)
Third-party schemas can be imported:
import itchio from "github.com/itchio/go-itchio"
namespace fetch {
server fn game(id: i64) -> (game: option<itchio.Game>)
}
(More on the import mechanism later.)
Workspaces
A workspace is a directory that contains a lavish-rules
file.
A lavish-rules
file is like a Makefile
for lavish - it tells
it what to compile, and for which language.
The lavish
command-line tool compiles schema files to Rust, Go,
TypeScript code.
Each workspace:
- targets a single language (Rust, Go, TypeScript)
- can build various services
- ...which share imports
Making a clock service
Let's say we're writing a simple Go service that returns the current time.
Before running the lavish compiler, our repo looks like:
- go.mod
- main.go
- services/
- clock.lavish
- lavish-rules
clock.lavish
contains:
server fn current_time() -> (time: timestamp)
And lavish-rules
contains:
target go
build clock from "./clock.lavish"
The lavish compiler accepts the path to the workspace:
lavish build ./services
After running the lavish compiler, our repo will look like:
- go.mod
- main.go
- services/
- clock.lavish
- lavish-rules
- clock/ <-- generated
- clock.go <-- generated
We can now implement the clock server, with something like:
package main
import (
"github.com/fasterthanlime/clock/services/clock"
"time"
)
func Handler() clock.ServerHandler {
var h clock.ServerHandler
h.OnCurrentTime(func () (clock.CurrentTimeResults, error) {
res := clock.CurrentTimeResults{
time: time.Now(),
}
return res, nil
})
return h
}
Finally, we can add a lavish-rules
file to the top-level, so that
we can later seemlessly import it from other projects:
export "./services/clock.lavish" as clock
Consuming the clock service from Rust
Let's say we want to call our clock service from rust.
Our initial Rust repo will look like:
- Cargo.toml
- src
- main.rs
- services/
- lavish-rules
Our lavish-rules
file will look like:
target rust
build clock from "github.com/fasterthanlime/clock"
Running the compiler with:
lavish build ./src/services
...will complain that clock
is missing.
Running:
lavish fetch ./src/services
Will populate the lavish-vendor
folder:
- Cargo.toml
- src
- main.rs
- lavish-rules
- lavish-vendor/ <-- new
- clock.lavish <-- new
Running compile again will generate rust code:
- Cargo.toml
- src
- main.rs
- lavish-rules
- lavish-vendor/
- clock.lavish
- clock/ <-- new
- mod.rs <-- new
Now, the clock
module can be imported from Rust and used
to consume the service, with something like:
mod clock;
type Error = Box<dyn std::error::Error + 'static>;
async fn example() -> Result<(), Error> {
// Create router - don't implement any functions from our side.
let r = clock::client::Router::new();
// Connect to server over TCP, with default timeout.
let client = lavish::connect(r, "localhost:5959")?.client();
{
let time = client.call(clock::current_time::Params {})?.time;
println!("Server time: {:#?}", time);
}
// when all handles go out of scope, the connection is closed
}
Consuming the clock service from TypeScript
Initial repo:
- src/
- main.ts
- services/
- lavish-rules
Contents of lavish-rules
:
target ts
build clock from "github.com/itchio"
lavish fetch src/services
- src/
- main.ts
- services/
- lavish-rules
- lavish-vendor/ <-- new
- clock.lavish <-- new
lavish compile src/services
- src/
- main.ts
- services/
- lavish-rules
- lavish-vendor/
- clock.lavish
- clock <-- new
- index.ts <-- new
We can then use it, from index.ts
:
import clock from "./services/clock"
async function main() {
let socket = new net.Socket();
await new Promise((resolve, reject) => {
socket.on("error", reject);
socket.connect({ host: "localhost", port: 5959 }, resolve);
});
let client = new clock.Client(socket);
console.log(`Server time: `, await client.getTime());
socket.close();
}
main().catch((e) => {
console.error(e);
process.exit(1);
});
That's all well and good, but... (FAQ)
Why workspaces?
Say you use two services, A
and B
, and they both use types from schema C
.
You want to be able to pass a result from a call to A
, as a parameter into
a call to B
.
If you build
both A
and B
in the same workspace, you'll end up with three directories: A
, B
, and C
. Both A
and B
will use the types
from C
.
Also:
- Passing a million command-line options is no fun
- Neither are a millions environment variables
- A minimal config language (
lavish-rules
) is, uh, not that bad - Not a big difference between writing one and two parsers anyway
What happens if A and B import a different C?
Then you can't use A
and B
in the same workspace. You can make two
workspaces though!
This seems like an arbitrary limitation. Does it simplify implementation somewhat?
It does, very much so.
Why one target per workspace?
Again, simpler implementation. If you want to generate bindings for multiple languages in a single repo, you can have:
- foobar/
- lavish-rules
- foobar-js/
- lavish-rules
- foobar-go/
- lavish-rules
- foobar-rs/
- lavish-rules
What's the format for import from
paths?
My idea for the import syntax is, for local files:
import foo from "./foo.lavish"
import bar from "../bar.lavish"
And for repos:
import foo from "github.com/user/foo"
import foo from "gitlab.com/user/bar"
How does it know what to git clone?
Given host/user/project
, it tries:
https://host/user/project.git
git@host:user/project.git
So does lavish build
need internet connectivity?
No, it does not. lavish fetch
does.
So is lavish fetch
a mini package manager, sorta?
Sorta, yes. You caught me. The alternative seems to involve copying lots of files around or manually cloning repos which sucks for a variety of reasons.
TL;DR: lavish fetch
vendors, lavish build
works offline.
How does it compare with other projects?
I like JSON-RPC a lot, because of its simplicity. That's what I used before. Msgpack-RPC is very similar, except with faster serialization, a proper timestamp type, and the ability to pass raw bytes around.
Cap'n Proto RPC is awe-inspiring. Not only is it fast, it also brings unique features - capabilities, and promise pipelining. I got really really excited about it.
However, after spending some time implementing capnp-rpc on top of an existing TypeScript serialization library, I finally conceded that:
- Implementation complexity is too high for me. It would take a lot of effort to write another implementation from scratch (for a new language), I do not understand the Rust implementation, if something broke I would have a very hard time tracking it down.
- Capabilities make it hard to use from the browser. It's no accident that, for the JavaScript world, the recommended implementation is node-only (a binding to the C++ library). Although I managed to get RPC working in pure TypeScript, I had to use electron and node-specific facilities to hook into the GC (to know when to drop capabilities). Browser usage could easily leak capabilities, and browser do not want to expose GC hooks for security reasons.
- It's purpose-built. There is no great desire to push for its adoption. It is being used internally, but there is no interest from the developer to make it everything to everyone - which is fine! That's also what I'm doing with lavish.
tarpc looks great, but Rust-only.
grpc is definitely trying to be everything to everyone. I would like to consume services from a variety of applications written with a variety of languages - a MsgPack serialization lib + TCP sockets is a reasonable ask for that. ProtoBufs + HTTP/2 is not.
Dependencies
~5–15MB
~151K SLoC