misanthropic

Is an unofficial simple, ergonomic, client for the Anthropic Messages API.

Usage

Streaming

// Create a client. The key is encrypted in memory and source string is zeroed.
// When requests are made, the key header is marked as sensitive.
let client = Client::new(key)?;

// Request a stream of events or errors. `json!` can be used, the `Request`
// builder pattern (shown in the `Single Message` example below), or anything
// serializable.
let stream = client
    // Forces `stream=true` in the request.
    .stream(json!({
      "model": Model::Sonnet35,
      "max_tokens": args.max_tokens,
      "temperature": 0,
      "system": args.system,
      "messages": [
        {
          "role": Role::User,
          "content": specs,
        }
      ],
    }))
    .await?
    // Filter out rate limit and overloaded errors. This is optional but
    // recommended for most use cases. The stream will continue when the
    // server is ready. Otherwise the stream will include these errors.
    .filter_rate_limit()
    // Filter out everything but text pieces (and errors).
    .text();

// Collect the stream into a single string.
let content: String = stream
    .try_collect()
    .await?;

Single Message

let client = Client::new(key)?;

// Many common usage patterns are supported out of the box for building
// `Request`s, such as messages from an iterable of tuples of `Role` and
// `String`.
let message = client
    .message(Request::default().messages([(Role::User, args.prompt)]))
    .await?;

println!("{}", message);

Features

• Async but does not directly depend on tokio
• Tool use,
• Streaming responses
• Message responses
• Image support with or without the image crate
• Markdown formatting of messages, including images
• Prompt caching support
• Custom request and endpoint support
• Zero-copy serde - Coming soon!
• Amazon Bedrock support
• Vertex AI support

FAQ

• Why is it called misanthropic? No reason, really. I just like the word. Anthropic is both a company and a word meaning "relating to mankind". This crate is neither official or related to mankind so, misanthropic it is.
• Doesn't reqwest depend on tokio? On some platforms, yes.
• Can i use misanthropic with Amazon or Vertex? Not yet, but it's on the roadmap. for now the Client does support custom endpoints and the inner reqwest::Client can be accessed directly to make necessary adjustments to headers, etc.
• Has this crate been audited? No, but auditing is welcome. A best effort has been made to ensure security and privacy. The API key is encrypted in memory using the memsecurity crate and any headers containing copies marked as sensitive. rustls is an optional feature and is recommended for security. It is on by default.