#stdio #json-rpc #http-client #ipc #llm #http-server #front-end

llmvm-protocol

A library that contains protocol/communication elements for llmvm

3 stable releases

2.0.1 Sep 19, 2024
2.0.0 Jan 23, 2024
1.0.0 Aug 9, 2023

#386 in Network programming

Download history 14/week @ 2024-07-08 19/week @ 2024-07-15 41/week @ 2024-07-22 10/week @ 2024-07-29 17/week @ 2024-08-05 6/week @ 2024-08-12 7/week @ 2024-08-19 32/week @ 2024-08-26 10/week @ 2024-09-09 151/week @ 2024-09-16 64/week @ 2024-09-23 19/week @ 2024-09-30 5/week @ 2024-10-07 24/week @ 2024-10-14

120 downloads per month
Used in 7 crates

MPL-2.0 license

46KB
849 lines

llmvm-protocol

Crates.io docs.rs GitHub

A library that contains protocol/communication elements for llmvm.

Uses multilink for stdio/local process and HTTP/remote process communication.

Contains:

  • Request/response types for core and backend communication
  • Traits for core and backend implementations
  • multilink / tower services
  • JSON-RPC and HTTP request/response conversion trait implementations for multilink.

All llmvm backends should implement the Backend trait, and use the llmvm-backend-util crate to start a stdio or HTTP server.

Rust frontends should use util::build_core_service_from_config in this crate to create a stdio or HTTP client, to communicate with the core.

Usage

Add the dependency to Cargo.toml:

[dependencies]
llmvm-protocol = { version = "<version>", features = ["<http-server|http-client|stdio-server|stdio-client>"] }

Configuration

stdio client configuration

Core and frontends will typically expose optional stdio client configuration. Here are the keys for the configuration structure:

Key Required? Description
bin_path No Optional binary path for spawning child processes. Defaults to PATH.
timeout_secs No Timeout for client requests in seconds.

stdio server configuration

Core and backends will typically expose optional stdio server configuration. Here are the keys for the configuration structure:

Key Required? Description
service_timeout_secs No Timeout for service requests in seconds.

HTTP client configuration

Core and frontends will typically expose optional HTTP client configuration. Here are the keys for the configuration structure:

Key Required? Description
base_url Yes Base URL/prefix for all outgoing requests.
api_key If llmvm server has API key set HTTP server API key to append to llmvm requests. The key will be inserted into the X-API-Key header.
timeout_secs No Timeout for client requests in seconds.

HTTP server configuration

Core and backends will typically expose optional HTTP server configuration. Here are the keys for the configuration structure:

Key Required? Description
port No Port to listen on. Defaults to 8080.
api_keys[] No An optional set of permitted API keys for restricting access to the server. If omitted, an API key is not needed to make a request.
service_timeout_secs No Timeout for service requests in seconds.

Reference documentation

Reference documentation can be found on docs.rs.

License

Mozilla Public License, version 2.0

Dependencies

~5–16MB
~233K SLoC