#language-model #nlp #ai #llm #llama #mistral

kalosm-language-model

A common interface for language models/transformers

7 releases

0.3.3 Aug 28, 2024
0.3.2 Aug 14, 2024
0.2.1 Feb 28, 2024
0.1.0 Dec 16, 2023

#892 in Machine learning

Download history 7/week @ 2024-07-22 23/week @ 2024-07-29 22/week @ 2024-08-05 428/week @ 2024-08-12 50/week @ 2024-08-19 212/week @ 2024-08-26 23/week @ 2024-09-02 37/week @ 2024-09-09 260/week @ 2024-09-16 209/week @ 2024-09-23 125/week @ 2024-09-30 65/week @ 2024-10-07 118/week @ 2024-10-14 46/week @ 2024-10-21 76/week @ 2024-10-28 59/week @ 2024-11-04

302 downloads per month
Used in 12 crates (8 directly)

MIT/Apache

300KB
7K SLoC

Language Model

This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.

Usage (with the RPhi implementation crate)

use kalosm::language::*;

#[tokio::main]
async fn main() {
    let mut model = Llama::phi_3().await.unwrap();
    let prompt = "The capital of France is ";
    let mut result = model.stream_text(prompt).await.unwrap();

    print!("{prompt}");
    while let Some(token) = result.next().await {
        print!("{token}");
    }
}

Dependencies

~33–48MB
~801K SLoC