#http #ipc #stdio #json-rpc #http-client #llm #http-server

llmvm-protocol

A library that contains protocol/communication elements for llmvm

2 stable releases

2.0.0 Jan 23, 2024
1.0.0 Aug 9, 2023

#1280 in Network programming

Download history 21/week @ 2024-03-11 14/week @ 2024-03-18 25/week @ 2024-03-25 77/week @ 2024-04-01 8/week @ 2024-04-08 10/week @ 2024-04-15 15/week @ 2024-04-22 10/week @ 2024-04-29 17/week @ 2024-05-06 2/week @ 2024-05-13 26/week @ 2024-05-20 19/week @ 2024-05-27 19/week @ 2024-06-03 16/week @ 2024-06-10 27/week @ 2024-06-17 30/week @ 2024-06-24

93 downloads per month
Used in 7 crates

MPL-2.0 license

47KB
861 lines

llmvm-protocol

Crates.io docs.rs GitHub

A library that contains protocol/communication elements for llmvm.

Uses multilink for stdio/local process and HTTP/remote process communication.

Contains:

  • Request/response types for core and backend communication
  • Traits for core and backend implementations
  • multilink / tower services
  • JSON-RPC and HTTP request/response conversion trait implementations for multilink.

All llmvm backends should implement the Backend trait, and use the llmvm-backend-util crate to start a stdio or HTTP server.

Rust frontends should use util::build_core_service_from_config in this crate to create a stdio or HTTP client, to communicate with the core.

Usage

Add the dependency to Cargo.toml:

[dependencies]
llmvm-protocol = { version = "<version>", features = ["<http-server|http-client|stdio-server|stdio-client>"] }

Configuration

stdio client configuration

Core and frontends will typically expose optional stdio client configuration. Here are the keys for the configuration structure:

Key Required? Description
bin_path No Optional binary path for spawning child processes. Defaults to PATH.
timeout_secs No Timeout for client requests in seconds.

stdio server configuration

Core and backends will typically expose optional stdio server configuration. Here are the keys for the configuration structure:

Key Required? Description
service_timeout_secs No Timeout for service requests in seconds.

HTTP client configuration

Core and frontends will typically expose optional HTTP client configuration. Here are the keys for the configuration structure:

Key Required? Description
base_url Yes Base URL/prefix for all outgoing requests.
api_key If llmvm server has API key set HTTP server API key to append to llmvm requests. The key will be inserted into the X-API-Key header.
timeout_secs No Timeout for client requests in seconds.

HTTP server configuration

Core and backends will typically expose optional HTTP server configuration. Here are the keys for the configuration structure:

Key Required? Description
port No Port to listen on. Defaults to 8080.
api_keys[] No An optional set of permitted API keys for restricting access to the server. If omitted, an API key is not needed to make a request.
service_timeout_secs No Timeout for service requests in seconds.

Reference documentation

Reference documentation can be found on docs.rs.

License

Mozilla Public License, version 2.0

Dependencies

~5–19MB
~251K SLoC