#ollama #model #bindings #async #api-bindings #embedding #running

ollama-rest

Asynchronous Rust bindings of Ollama REST API

13 releases

new 0.5.6 Mar 10, 2025
0.5.5 Feb 16, 2025
0.5.4 Jan 31, 2025
0.3.3 Nov 26, 2024
0.1.1 Jul 23, 2024

#1486 in Web programming

Download history 114/week @ 2024-11-20 47/week @ 2024-11-27 5/week @ 2024-12-04 10/week @ 2024-12-11 152/week @ 2025-01-08 239/week @ 2025-01-15 119/week @ 2025-01-22 134/week @ 2025-01-29 41/week @ 2025-02-05 134/week @ 2025-02-12 23/week @ 2025-02-19 14/week @ 2025-02-26

218 downloads per month

MIT license

28KB
610 lines

ollama-rest.rs

Asynchronous Rust bindings of Ollama REST API, using reqwest, tokio, serde, and chrono.

Install

cargo add ollama-rest@0.5

Features

name status
Completion Supported ✅
Embedding Supported ✅
Model creation Supported ✅
Model deletion Supported ✅
Model pulling Supported ✅
Model copying Supported ✅
Local models Supported ✅
Running models Supported ✅
Model pushing Experimental 🧪
Tools Experimental 🧪

At a glance

See source of this example.

use std::io::Write;

use ollama_rest::{models::generate::{GenerationRequest, GenerationResponse}, Ollama};
use serde_json::json;

// By default checking Ollama at 127.0.0.1:11434
let ollama = Ollama::default();

let request = serde_json::from_value::<GenerationRequest>(json!({
    "model": "llama3.2:1b",
    "prompt": "Why is the sky blue?",
})).unwrap();

let mut stream = ollama.generate_streamed(&request).await.unwrap();

while let Some(Ok(res)) = stream.next().await {
    if !res.done {
        print!("{}", res.response);
        // Flush stdout for each word to allow realtime output
        std::io::stdout().flush().unwrap();
    }
}

println!();

Or, make your own chatbot interface! See this example (CLI) and this example (REST API).

Dependencies

~7–19MB
~246K SLoC