1 unstable release
0.13.0 | Nov 15, 2023 |
---|
#9 in #llm-chain
220KB
4K
SLoC
llm-chain-sagemaker-endpoint
Amazon SageMaker Endppoint driver. Allows you to invoke a model hosted on Amazon SageMaker Endpoint, this includes Amazon SageMaker Jumpstart models.
Getting Started
- This crate uses the AWS SDK for Rust to communicate with Amazon SageMaker. You need to set up the credentials and a region following this guide
- Follow the SageMaker JumpStart documentation to find an LLM, then deploy it.
- Note down the SageMaker Endpoint name created by SageMaker JumpStart.
- Some models is included in this crate, see
model::Model::<model_name>
. Select one in your executor options'sModel
field. Seeexamples/simple.rs
for example. - For custom models or models not included in
model::Model
, usemodel::Model::Other(<model_name>)
, wheremodel_name
is the SageMaker endpoint name.
Dependencies
~24–37MB
~613K SLoC