15 releases (4 breaking)
0.5.1 | Nov 30, 2024 |
---|---|
0.5.0 | Nov 14, 2024 |
0.4.2 | Oct 23, 2024 |
0.3.3 | Oct 13, 2024 |
0.1.4 | Aug 28, 2024 |
#877 in Network programming
349 downloads per month
215KB
4.5K
SLoC
misanthropic
Is an unofficial simple, ergonomic, client for the Anthropic Messages API.
Usage
Streaming
// Create a client. The key is encrypted in memory and source string is zeroed.
// When requests are made, the key header is marked as sensitive.
let client = Client::new(key)?;
// Request a stream of events or errors. `json!` can be used, the `Prompt`
// builder pattern (shown in the `Single Message` example below), or anything
// serializable.
let stream = client
// Forces `stream=true` in the request.
.stream(json!({
"model": Model::Sonnet35,
"max_tokens": args.max_tokens,
"temperature": 0,
"system": args.system,
"messages": [
{
"role": Role::User,
"content": specs,
}
],
}))
.await?
// Filter out rate limit and overloaded errors. This is optional but
// recommended for most use cases. The stream will continue when the
// server is ready. Otherwise the stream will include these errors.
.filter_rate_limit()
// Filter out everything but text pieces (and errors).
.text();
// Collect the stream into a single string.
let content: String = stream
.try_collect()
.await?;
Single Message
let client = Client::new(key)?;
// Many common usage patterns are supported out of the box for building
// `Prompt`s, such as messages from an iterable of tuples of `Role` and
// `String`.
let message = client
.message(Prompt::default().messages([(Role::User, args.prompt)]))
.await?;
println!("{}", message);
Features
- Async but does not directly depend on tokio
- Tool use,
- Streaming responses
- Message responses
- Image support with or without the
image
crate - Markdown formatting of messages, including images
- HTML formatting of messages*.
- Prompt caching support
- Custom request and endpoint support
- Zero-copy where possible
- Sanitization of input and output to mitigate injection attacks
- Amazon Bedrock support
- Vertex AI support
* Base64 encoded images are currently not implemented for HTML but this is a planned feature.
FAQ
- Why is it called
misanthropic
? No reason, really. I just like the word. Anthropic is both a company and a word meaning "relating to mankind". This crate is neither official or related to mankind so,misanthropic
it is. - Doesn't
reqwest
depend ontokio
? On some platforms, yes. - Can i use
misanthropic
with Amazon or Vertex? Not yet, but it's on the roadmap. for now theClient
does support custom endpoints and the innerreqwest::Client
can be accessed directly to make necessary adjustments to headers, etc. - Has this crate been audited? No, but auditing is welcome. A best effort
has been made to ensure security and privacy. The API key is encrypted in
memory using the
memsecurity
crate and any headers containing copies marked as sensitive.rustls
is an optional feature and is recommended for security. It is on by default.
Dependencies
~6–19MB
~255K SLoC