OpenRouter API Client Library

OpenRouter API Client Library is a Rust client for interfacing with the OpenRouter API. The library is designed to be modular, type‑safe, and intuitive. It uses a type‑state builder pattern for configuring and validating the client at compile time, ensuring that all required configuration (such as setting the base URL and API key) happens before attempting a request.

Note: This project is still in development. Many features are planned but not yet fully implemented.

Features

• Modular Organization: Organized into clear modules for models, API endpoints, common types, and utilities.
• Type‑State Builder: Guarantees compile‑time validation of client configuration (e.g. base URL, API key, custom headers) for a robust development experience.
• HTTP Integration: Uses reqwest with rustls‑tls for secure asynchronous HTTP requests.
• Robust Error Handling: Centralized error management using the thiserror crate ensures consistent error types across the library.
• Streaming Support: Supports streaming chat completions via Server‑Sent Events (SSE). The library gracefully skips over comment lines and non‑JSON payloads, letting you update UIs in real‑time.
• Structured Outputs: Optionally request structured responses with JSON Schema validation so that responses strictly follow your defined schema.
• Tool Calling Capability: Define function‑type tools that the model can invoke. Supports concurrent tool calls in a single response with proper validation against expected formats.
• Provider Preferences & Routing: Configure model fallbacks, routing preferences, and provider filtering via a strongly‑typed interface.
• Web Search Endpoint: Easily perform web search queries with type‑safe request and response models.
• Text Completion Endpoint: Send a prompt (with a required model and prompt field) and receive generated text completions along with additional generation details. Extra parameters (e.g. temperature, top_p, etc.) can be provided as needed.

Getting Started

Installation

Add the following to your project's Cargo.toml:

cargo add openrouter_api

Ensure that you have Rust installed (tested with Rust v1.83.0) and that you're using Cargo for building and testing.

Example Usage

Minimal Chat Example

use openrouter_api::{OpenRouterClient, Ready, Result};
use openrouter_api::types::chat::{ChatCompletionRequest, Message};

#[tokio::main]
async fn main() -> Result<()> {
    // Ensure your API key is set in the environment.
    let api_key = std::env::var("OPENROUTER_API_KEY")
        .expect("OPENROUTER_API_KEY must be set");

    // Build the client (Unconfigured -> NoAuth -> Ready).
    let client = OpenRouterClient::new()
        .with_base_url("https://openrouter.ai/api/v1/")?
        .with_api_key(api_key)?;

    // Create a minimal chat completion request.
    let request = ChatCompletionRequest {
        model: "openai/gpt-4o".to_string(),
        messages: vec![Message {
            role: "user".to_string(),
            content: "Hello, world!".to_string(),
            name: None,
            tool_calls: None,
        }],
        stream: None,
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    };

    // Invoke the chat completion endpoint.
    let response = client.chat_completion(request).await?;

    // Output the model's response.
    if let Some(choice) = response.choices.first() {
        println!("Chat Response: {}", choice.message.content);
    }
    Ok(())
}

Minimal Text Completion Example

use openrouter_api::{OpenRouterClient, Ready, Result};
use openrouter_api::types::completion::{CompletionRequest, CompletionResponse};
use serde_json::json;

#[tokio::main]
async fn main() -> Result<()> {
    // Ensure your API key is set in the environment.
    let api_key = std::env::var("OPENROUTER_API_KEY")
        .expect("OPENROUTER_API_KEY must be set");

    // Build the client (Unconfigured -> NoAuth -> Ready).
    let client = OpenRouterClient::new()
        .with_base_url("https://openrouter.ai/api/v1/")?
        .with_api_key(api_key)?;

    // Create a minimal text completion request.
    let request = CompletionRequest {
        model: "model".to_string(),
        prompt: "Once upon a time".to_string(),
        // Additional generation parameters can be set here.
        extra_params: json!({
            "temperature": 0.8,
            "max_tokens": 50
        }),
    };

    // Invoke the text completion endpoint.
    let response: CompletionResponse = client.completions().text_completion(request).await?;

    // Print out the generated text from the first choice.
    if let Some(choice) = response.choices.first() {
        println!("Text Completion: {}", choice.text);
    }
    Ok(())
}

Minimal Web Search Example

use openrouter_api::{OpenRouterClient, Ready, Result};
use openrouter_api::types::web_search::{WebSearchRequest, WebSearchResponse};

#[tokio::main]
async fn main() -> Result<()> {
    // Ensure your API key is set in the environment.
    let api_key = std::env::var("OPENROUTER_API_KEY")
        .expect("OPENROUTER_API_KEY must be set");

    // Build the client (Unconfigured -> NoAuth -> Ready).
    let client = OpenRouterClient::new()
        .with_base_url("https://openrouter.ai/api/v1/")?
        .with_api_key(api_key)?;

    // Create a minimal web search request.
    let request = WebSearchRequest {
        query: "rust programming".into(),
        num_results: Some(5),
    };

    // Invoke the web search endpoint.
    let response: WebSearchResponse = client.web_search()?.search(request).await?;

    // Print out the search results.
    println!("Search query: {}", response.query);
    for result in response.results {
        println!("Title: {}\nURL: {}\n", result.title, result.url);
    }

    Ok(())
}

Streaming Chat Example

use openrouter_api::{OpenRouterClient, Ready, Result};
use openrouter_api::types::chat::{ChatCompletionRequest, Message, ChatCompletionChunk};
use futures::StreamExt;

#[tokio::main]
async fn main() -> Result<()> {
    // Ensure your API key is set in the environment.
    let api_key = std::env::var("OPENROUTER_API_KEY")
        .expect("OPENROUTER_API_KEY must be set");

    // Build the client.
    let client = OpenRouterClient::new()
        .with_base_url("https://openrouter.ai/api/v1/")?
        .with_api_key(api_key)?;

    // Create a chat completion request with streaming enabled.
    let request = ChatCompletionRequest {
        model: "openai/gpt-4o".to_string(),
        messages: vec![Message {
            role: "user".to_string(),
            content: "Tell me a story.".to_string(),
            name: None,
            tool_calls: None,
        }],
        stream: Some(true),
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    };

    // Invoke the streaming chat completion endpoint.
    let mut stream = client.chat()?.chat_completion_stream(request);

    while let Some(chunk) = stream.next().await {
        match chunk {
            Ok(c) => {
                // Incrementally output the content.
                print!("{}", c.message.content);
            }
            Err(e) => eprintln!("Error during streaming: {}", e),
        }
    }
    println!();
    Ok(())
}

Implementation Plan

The project is under active development. The roadmap outlines upcoming features and milestones:

Phase 1: Core Functionality (Completed/In Progress)

• Client Framework:
  • Type‑state builder pattern for configuration with compile‑time validations.
  • Custom headers and robust error propagation.
• Chat Completion Endpoint:
  • Synchronous chat completions with JSON decoding and streaming support.
• Core Data Models:
  • Definitions for chat messages, requests, responses, and usage.

Phase 2: Additional Endpoints and Features

• Streaming Support:
  • Streaming API for chat completions via Server‑Sent Events (SSE).
• Web Search Endpoint:
  • New endpoint for web search queries with strongly‑typed request/response models.
• Text Completion Endpoint:
  • New endpoint for text completions, accepting a prompt and returning generated text along with extra details.
• Tool Calling & Structured Outputs:
  • Support for invoking callable functions and validating structured responses via JSON Schema.
• Provider Preferences & Routing:
  • Configuration options for model fallbacks, routing, and provider filtering.
• Models Listing and Credits:
  • Implement endpoints to list models and fetch credit details.

Phase 3: Robust Testing & Documentation

• Test Coverage:
  • Expand unit and integration tests, including streaming-specific tests.
• Documentation Improvements:
  • Enhance inline documentation, API docs, and usage examples in the /examples directory.
• Continuous Integration (CI):
  • Set up CI pipelines for automated builds and tests.

Contributing

Contributions are welcome! Please open an issue or submit a pull request with your ideas or fixes. Follow the code style guidelines and ensure that all tests pass.

License

Distributed under either the MIT license or the Apache License, Version 2.0. See LICENSE for details.

OpenRouter API Rust Crate Documentation

Version: 0.1.2 • License: MIT / Apache‑2.0

The openrouter_api crate is a comprehensive client for interacting with the OpenRouter API. It provides strongly‑typed endpoints for chat completions, text completions, web search, and more. The crate is built using asynchronous Rust (with reqwest and tokio) and leverages advanced patterns such as type‑state and builder patterns for safe and flexible API usage.

Table of Contents

• Core Concepts
• Installation
• Architecture & Module Overview
• Client Setup & Type‑State Pattern
• API Endpoints
  • Chat Completions
  • Text Completions
  • Web Search
  • Tool Calling & Structured Output
• Error Handling
• Best Practices
• Examples
• Additional Resources

Core Concepts

• Type‑State Client Configuration: The client is built using a type‑state pattern to ensure that required parameters (e.g. API key and base URL) are set before making any API calls. This pattern leverages Rust’s type system (with PhantomData) to prevent misconfiguration at compile time.

• Flexible Request Building: The RequestBuilder lets you build rich API requests including optional provider preferences, structured output (using JSON Schema), and tool calling configurations.

• Asynchronous Streaming: The chat API supports streaming responses via asynchronous streams. This is implemented using crates like async-stream and tokio-util.

• Error Handling & Schema Validation: A comprehensive custom error type wraps HTTP errors, API errors, configuration mistakes, and JSON Schema validation issues.

• Tool Calling: Easily integrate with external functions using a structured representation of function calls. This enables the API to suggest or even invoke tools based on the conversation context.

Installation

Add the following to your Cargo.toml:

[dependencies]
openrouter_api = "0.1.2"
reqwest = { version = "0.11", features = ["json", "stream"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
async-stream = "0.3"
tokio-util = "0.7"
thiserror = "1.0"

Architecture & Module Overview

The crate is organized into several modules:

• client: Contains the client configuration and type‑state builder implementation. It ensures proper initialization before any API calls are made.

• api: Provides submodules for each API endpoint (chat, completion, web search, etc.). Each submodule includes detailed implementations for constructing requests and handling responses.

• models: Defines domain models for structured outputs, provider preferences, and tool calling.

• types: Contains type definitions for chat messages, completions, and web search responses.

• error: Centralized error types (wrapping reqwest errors, API errors, configuration issues, and schema validation problems).

• tests: Integration tests showcasing how to simulate and validate API responses.

Client Setup & Type‑State Pattern

The client configuration follows a three‑step type‑state builder pattern:

1. Unconfigured: The client is created in an unconfigured state using OpenRouterClient::new().
2. NoAuth: The base URL is set (via .with_base_url()), transitioning the client into a state where authentication is not yet provided.
3. Ready: The API key is added (via .with_api_key()), and the client becomes fully configured for making API calls.

Example: Configuring the Client

use openrouter_api::client::{OpenRouterClient, Unconfigured};
use std::time::Duration;

// Create an unconfigured client.
let client = OpenRouterClient::<Unconfigured>::new();

// Transition to a configured client (NoAuth state) by setting the base URL.
let client = client
    .with_base_url("https://openrouter.ai/api/v1/")?
    .with_timeout(Duration::from_secs(30))
    .with_http_referer("https://your-app.com/")
    .with_site_title("Your App Name");

// Supply the API key to transition into the Ready state.
let client = client.with_api_key(std::env::var("OPENROUTER_API_KEY")?)?;

Note: The type‑state pattern prevents you from accidentally making API calls without proper configuration. Attempting to call an endpoint on a client that isn’t in the Ready state will result in a compile‑time error.

API Endpoints

The crate supports multiple endpoints, each with its own module and specialized request/response types.

Chat Completions

The chat API supports both single‑shot and streaming completions.

Single‑Shot Chat Completion

use openrouter_api::types::chat::Message;

let messages = vec![Message {
    role: "user".to_string(),
    content: "Explain quantum computing".to_string(),
    name: None,
    tool_calls: None,
}];

// Issue a single chat completion call.
let chat_api = client.chat()?;
let response = chat_api.chat_completion(
    openrouter_api::types::chat::ChatCompletionRequest {
        model: "mistralai/mistral-small-latest".to_string(),
        messages: messages.clone(),
        stream: None,
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    }
).await?;

println!("Assistant: {}", response.choices[0].message.content);

Streaming Chat Completion

Streaming is useful for real‑time applications. The stream returns chunks as they become available.

use futures::StreamExt;

let chat_api = client.chat()?;
let mut stream = chat_api.chat_completion_stream(
    openrouter_api::types::chat::ChatCompletionRequest {
        model: "mistralai/mistral-small-latest".to_string(),
        messages: messages.clone(),
        stream: Some(true),
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    }
);

while let Some(chunk) = stream.next().await {
    match chunk {
        Ok(c) => {
            // Each chunk may include partial content updates.
            if let Some(choice) = c.choices.first() {
                // Print the delta (partial content) if available.
                print!("{}", choice.message.content);
            }
        }
        Err(e) => {
            eprintln!("Stream error: {:?}", e);
            break;
        }
    }
}

Text Completions

For pure text completions, the CompletionApi is used.

use openrouter_api::types::completion::CompletionRequest;

let request = CompletionRequest {
    model: "openai/gpt-4".to_string(),
    prompt: "Once upon a time".to_string(),
    extra_params: serde_json::json!({ "temperature": 0.7 }),
};

let completion_api = openrouter_api::api::completion::CompletionApi::new(
    client.http_client.clone().unwrap(),  // Clone the reqwest client
    &client.config,
);

let response = completion_api.text_completion(request).await?;
println!("Completed Text: {}", response.choices[0].text);

Web Search

The web search API allows you to perform simple search queries and receive structured results.

use openrouter_api::types::web_search::WebSearchRequest;

let search_request = WebSearchRequest {
    query: "rust programming".to_string(),
    num_results: Some(5),
};

let web_search_api = client.web_search()?;
let search_response = web_search_api.search(search_request).await?;

println!("Query: {}", search_response.query);
for result in search_response.results {
    println!("Title: {} • URL: {}", result.title, result.url);
}

Tool Calling & Structured Output

Enabling Structured Output

Structured output allows you to validate the model’s response against a JSON Schema. This is configured via the RequestBuilder.

use openrouter_api::models::structured::JsonSchemaConfig;
use openrouter_api::api::request::RequestBuilder;

let schema_config = JsonSchemaConfig {
    name: "Person".to_string(),
    strict: true,
    schema: serde_json::json!({
        "type": "object",
        "properties": {
            "name": { "type": "string" },
            "age": { "type": "integer" }
        },
        "required": ["name", "age"]
    }),
};

let request_payload = RequestBuilder::new("mistralai/mistral-small-latest", messages.clone(), serde_json::json!({}))
    .with_structured_output(schema_config, true, true)  // validate & fallback options
    .build();

Configuring Tool Calling

Tool calling lets you define external functions that the model can suggest invoking. This is useful for integrating live functions (e.g., fetching weather).

use openrouter_api::models::tool::{Tool, FunctionDescription};

let get_weather = FunctionDescription {
    name: "get_weather".to_string(),
    description: Some("Get weather information for a location".to_string()),
    parameters: serde_json::json!({
        "type": "object",
        "properties": {
            "location": {
                "type": "string",
                "description": "City and state (e.g., 'San Francisco, CA')"
            },
            "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
            }
        },
        "required": ["location"]
    }),
};

let request_payload = RequestBuilder::new("meta-llama/llama-3-70b-instruct", messages.clone(), serde_json::json!({}))
    .build();

// Call the chat endpoint with tool information.
let response = client.chat()?.chat_completion(
    openrouter_api::types::chat::ChatCompletionRequest {
        model: "meta-llama/llama-3-70b-instruct".to_string(),
        messages,
        stream: None,
        response_format: None,
        tools: Some(vec![Tool::Function { function: get_weather }]),
        provider: None,
        models: None,
        transforms: None,
    }
).await?;

// Process the tool call response.
if let Some(choice) = response.choices.first() {
    if let Some(tool_calls) = &choice.message.tool_calls {
        for call in tool_calls {
            if call.kind == "function" {
                println!("Tool call detected for function: {}", call.function_call.name);
                println!("Arguments: {}", call.function_call.arguments);
                // Here, you would implement your logic to execute the tool and return the result.
            }
        }
    }
}

Error Handling

The crate defines a central Error enum that wraps different error kinds:

• HTTP Errors: Errors coming from the reqwest library.

• API Errors: Errors returned from the OpenRouter API (with status codes and error messages).

• Configuration Errors: Issues with client setup (e.g., invalid API key or headers).

• Schema Validation Errors: When a response fails JSON Schema validation for structured output.

Example Error Handling

match client.chat()?.chat_completion(
    openrouter_api::types::chat::ChatCompletionRequest {
        model: "openai/gpt-4".to_string(),
        messages: messages.clone(),
        stream: None,
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    }
).await {
    Ok(response) => {
        println!("Chat completion succeeded: {:?}", response);
    },
    Err(e) => match e {
        openrouter_api::error::Error::ApiError { code, message, .. } => {
            eprintln!("API Error ({}): {}", code, message);
        },
        openrouter_api::error::Error::HttpError(ref err) if err.is_timeout() => {
            eprintln!("Request timed out!");
        },
        _ => eprintln!("Unexpected error: {:?}", e),
    },
}

Best Practices

1. Use the Type‑State Pattern: Always configure your client fully (base URL, API key, headers) before issuing requests. The type‑state pattern will help catch misconfiguration at compile time.

2. Set Appropriate Timeouts & Headers: Leverage with_timeout, with_http_referer, and with_site_title to ensure your requests are traceable and robust.

3. Validate Structured Responses: When using structured output, configure JSON Schema validation to catch unexpected responses early.

4. Stream Responsibly: When using streaming endpoints, ensure you handle errors gracefully and close streams when finished.

5. Secure Your API Keys: Store sensitive API keys in environment variables or secure storage rather than hardcoding them.

Examples

Full Client Initialization & Chat Request

Below is a complete example combining client initialization, a chat request, and error handling:

use openrouter_api::client::{OpenRouterClient, Unconfigured};
use openrouter_api::types::chat::{Message, ChatCompletionRequest};
use std::time::Duration;
use tokio;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize client (Unconfigured -> NoAuth -> Ready)
    let client = OpenRouterClient::<Unconfigured>::new()
        .with_base_url("https://openrouter.ai/api/v1/")?
        .with_timeout(Duration::from_secs(30))
        .with_http_referer("https://your-app.com/")
        .with_site_title("Example App")
        .with_api_key(std::env::var("OPENROUTER_API_KEY")?)?;

    // Prepare chat messages.
    let messages = vec![
        Message {
            role: "user".to_string(),
            content: "What is a phantom type in Rust?".to_string(),
            name: None,
            tool_calls: None,
        }
    ];

    // Build the chat completion request.
    let request = ChatCompletionRequest {
        model: "openai/gpt-4".to_string(),
        messages,
        stream: None,
        response_format: None,
        tools: None,
        provider: None,
        models: None,
        transforms: None,
    };

    // Execute the chat completion.
    let chat_api = client.chat()?;
    match chat_api.chat_completion(request).await {
        Ok(response) => {
            println!("Assistant says: {}", response.choices[0].message.content);
        },
        Err(err) => {
            eprintln!("Error: {:?}", err);
        }
    }

    Ok(())
}

Provider Preferences with Request Builder

use openrouter_api::api::request::RequestBuilder;
use serde_json::json;

let provider_preferences = json!({
    "order": ["openrouter"],
    "allowFallbacks": true
});

let request_payload = RequestBuilder::new("openai/gpt-4", messages.clone(), json!({}))
    .with_provider_preferences(provider_preferences, true, true)
    .build();

println!("Payload: {}", serde_json::to_string_pretty(&request_payload)?);

Additional Resources

• OpenRouter API Documentation