#nlp #llm #ai #mistral #llama #helper #text-generation

kalosm-sample

A common interface for token sampling and helpers for structered llm sampling

5 unstable releases

0.3.2 Aug 14, 2024
0.3.0 Aug 13, 2024
0.2.1 Feb 28, 2024
0.2.0 Feb 18, 2024
0.1.0 Dec 16, 2023

#426 in Machine learning

Download history 24/week @ 2024-07-11 26/week @ 2024-07-18 19/week @ 2024-07-25 26/week @ 2024-08-01 233/week @ 2024-08-08 122/week @ 2024-08-15 54/week @ 2024-08-22 67/week @ 2024-08-29 32/week @ 2024-09-05 128/week @ 2024-09-12 206/week @ 2024-09-19 186/week @ 2024-09-26 95/week @ 2024-10-03 62/week @ 2024-10-10 110/week @ 2024-10-17 54/week @ 2024-10-24

341 downloads per month
Used in 12 crates (5 directly)

MIT/Apache

165KB
4.5K SLoC

kalosm-sample

This is a sampling library for Kalosm.

It handles choosing a token from a probability distribution. Samplers can be used to constrain the generation of text for example you can use a sampler to prevent the model from generating the same word twice in a row. Or you could only allow the model to generate a list of single digit numbers.

Dependencies

~1.6–2.6MB
~37K SLoC