#nlp #llm #ai #mistral #llama #helper #text-generation

kalosm-sample

A common interface for token sampling and helpers for structered llm sampling

5 unstable releases

0.3.2 Aug 14, 2024
0.3.0 Aug 13, 2024
0.2.1 Feb 28, 2024
0.2.0 Feb 18, 2024
0.1.0 Dec 16, 2023

#464 in Machine learning

Download history 72/week @ 2024-08-17 54/week @ 2024-08-24 65/week @ 2024-08-31 27/week @ 2024-09-07 209/week @ 2024-09-14 185/week @ 2024-09-21 146/week @ 2024-09-28 84/week @ 2024-10-05 105/week @ 2024-10-12 64/week @ 2024-10-19 80/week @ 2024-10-26 66/week @ 2024-11-02 46/week @ 2024-11-09 56/week @ 2024-11-16 86/week @ 2024-11-23 68/week @ 2024-11-30

266 downloads per month
Used in 12 crates (5 directly)

MIT/Apache

165KB
4.5K SLoC

kalosm-sample

This is a sampling library for Kalosm.

It handles choosing a token from a probability distribution. Samplers can be used to constrain the generation of text for example you can use a sampler to prevent the model from generating the same word twice in a row. Or you could only allow the model to generate a list of single digit numbers.

Dependencies

~1.5–2.5MB
~37K SLoC