7 releases
0.3.3 | Aug 28, 2024 |
---|---|
0.3.2 | Aug 14, 2024 |
0.2.1 | Feb 28, 2024 |
0.1.0 | Dec 16, 2023 |
#884 in Machine learning
300 downloads per month
Used in 12 crates
(8 directly)
300KB
7K
SLoC
Language Model
This crate provides a unified interface for language models. It supports streaming text, sampling, and embedding.
Usage (with the RPhi implementation crate)
use kalosm::language::*;
#[tokio::main]
async fn main() {
let mut model = Llama::phi_3().await.unwrap();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await.unwrap();
print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}
Dependencies
~33–48MB
~803K SLoC