Cargo Features

another-tiktoken-rs has no features set by default.

[dependencies]
another-tiktoken-rs = { version = "0.1.2", features = ["python", "async-openai"] }
python

build python bindings

Enables pyo3 ^0.19.2

Affects vendor_tiktoken::CoreBPE

async-openai

Enables async-openai ^0.14.0

Affects api::async_openai