OpenRouter

What OpenRouter provides

OpenRouter is a unified API gateway that aggregates 200+ LLMs from OpenAI, Anthropic, Google, Meta, Mistral, Cohere, and dozens of other providers and model hosts. You get a single OpenAI-compatible endpoint (https://openrouter.ai/api/v1) and one API key that works for all models. Key features: real-time pricing comparison across providers, automatic fallback when a provider is down, provider routing (pick fastest, cheapest, or most reliable), and per-request cost tracking.

Getting started

import openai

# OpenRouter uses the OpenAI SDK with a custom base URL
client = openai.OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="sk-or-v1-...",  # OpenRouter API key
)

# Use any model by its OpenRouter name
response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_headers={
        "HTTP-Referer": "https://yourapp.com",  # optional, for analytics
        "X-Title": "My App",                    # optional, appears in dashboard
    },
)
print(response.choices[0].message.content)

# Other models — same code, different model string:
# "anthropic/claude-3-haiku"
# "meta-llama/llama-3.1-8b-instruct"
# "google/gemini-flash-1.5"
# "mistralai/mistral-7b-instruct"
# "qwen/qwen-2.5-72b-instruct"

Model selection and comparison

import requests

# Get current model list with pricing
models = requests.get(
    "https://openrouter.ai/api/v1/models",
    headers={"Authorization": f"Bearer {api_key}"},
).json()["data"]

# Sort by input cost
by_cost = sorted(models, key=lambda m: float(m.get("pricing", {}).get("prompt", 999)))
for m in by_cost[:5]:
    price = m.get("pricing", {})
    print(f"{m['id']}: ${float(price.get('prompt', 0))*1e6:.3f}/1M input tokens")

# Output (approximate, prices change):
# meta-llama/llama-3.2-1b-instruct:free: $0.000/1M
# google/gemma-2-9b-it:free: $0.000/1M
# meta-llama/llama-3.1-8b-instruct: $0.055/1M
# mistralai/mistral-7b-instruct: $0.055/1M
# google/gemini-flash-1.5: $0.075/1M

Automatic fallbacks

# OpenRouter automatically falls back to the next provider if one is down
# You can also specify explicit fallbacks:

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing"}],
    extra_body={
        "route": "fallback",
        "models": [               # try in order
            "openai/gpt-4o",
            "anthropic/claude-3-5-sonnet",
            "google/gemini-pro-1.5",
        ],
    },
)

# The response includes which model was actually used:
# response.model == "openai/gpt-4o" or whichever succeeded
print(f"Used model: {response.model}")

Cost optimisation

# Use OpenRouter to route to cheapest provider for a given model
response = client.chat.completions.create(
    model="meta-llama/llama-3.1-70b-instruct",
    messages=[{"role": "user", "content": "Summarise this document..."}],
    extra_body={
        "provider": {
            "order": ["Fireworks", "Together", "Groq"],  # try cheapest first
            "allow_fallbacks": True,
        }
    },
)

# Track cost per request via usage metadata
usage = response.usage
print(f"Tokens: {usage.prompt_tokens} in, {usage.completion_tokens} out")
# OpenRouter returns cost in the x-openrouter-cost response header
# or via the /api/v1/generation?id= endpoint

Provider routing

# Route to specific providers based on your needs

# Fastest (lowest latency) — Groq is typically fastest
fast_response = client.chat.completions.create(
    model="meta-llama/llama-3.1-8b-instruct",
    extra_body={"provider": {"order": ["Groq"], "allow_fallbacks": True}},
    messages=[{"role": "user", "content": "What is 2+2?"}],
)

# Most reliable (highest uptime SLA)
reliable_response = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    extra_body={"provider": {"order": ["OpenAI", "Azure"], "allow_fallbacks": True}},
    messages=[{"role": "user", "content": "Classify this email..."}],
)

# Data privacy (only providers with specific data policies)
private_response = client.chat.completions.create(
    model="anthropic/claude-3-haiku",
    extra_body={"provider": {"data_collection": "deny"}},  # opt out of training
    messages=[{"role": "user", "content": "Here is our private data..."}],
)

Gotchas

• Model name format: OpenRouter uses provider/model-name format (openai/gpt-4o, not just gpt-4o). Using the wrong format returns a 404. Check the OpenRouter model list for exact IDs.
• Free tier limits: Several models are available free on OpenRouter (:free suffix) but with low rate limits. Don't use free tier models in production.
• Latency overhead: OpenRouter adds ~50–100ms latency vs calling providers directly. For latency-sensitive applications, calling OpenAI/Anthropic directly is faster. OpenRouter's value is in routing, fallback, and simplicity — not raw speed.
• Pricing volatility: Provider pricing changes frequently. Don't hardcode cost calculations — query the OpenRouter API for current pricing before making routing decisions.