5 unstable releases
0.3.2 | Aug 14, 2024 |
---|---|
0.3.0 | Aug 13, 2024 |
0.2.1 | Feb 28, 2024 |
0.2.0 | Feb 18, 2024 |
0.1.0 | Dec 16, 2023 |
#1082 in Machine learning
180 downloads per month
Used in 4 crates
(via kalosm-language)
345KB
8K
SLoC
RPhi
RPhi is a Rust implementation of the quantized Phi 1.5 language model.
Phi-1.5 is a very small but performant language model that can be easily run on your local machine.
This library uses Quantized Mixformer from Candle to run Phi-1.5.
Usage
use rphi::prelude::*;
#[tokio::main]
async fn main() {
let mut model = Phi::v2().await.unwrap();
let prompt = "The capital of France is ";
let mut result = model.stream_text(prompt).await.unwrap();
print!("{prompt}");
while let Some(token) = result.next().await {
print!("{token}");
}
}
Dependencies
~35–56MB
~1M SLoC