OpenRouter: One API for Hundreds of LLMs (OpenAI, Anthropic, Google, Meta, Mistral)
Hey everyone, it's me again, Ruby Abdullah. This time I want to talk about a tool that I think you absolutely need to know if you're serious about building applications on top of large language models. It's called OpenRouter. For me personally, OpenRouter is like a universal power outlet for every large language model out there right now. Think about it, normally if you want to use GPT from OpenAI you have to sign up with OpenAI, get your own API key, use their own SDK. Then if you want to try Claude from Anthropic, you have to sign up again with Anthropic, another API key, another SDK. Want to try Gemini from Google? Same thing, sign up again. Not to mention if you want to try open source models like Llama from Meta or Mistral. It gets messy fast, right?
Well, OpenRouter comes in to solve exactly that problem. With one API key and one endpoint whose format is compatible with OpenAI, you can access hundreds of models from various providers. So your code doesn't change, you just swap the model name and you can hop from GPT to Claude to Gemini to Llama as easily as changing a string. This is a real game changer for me, especially when you're experimenting to find which model fits a particular use case, or when you want to build a system that has automatic fallback if one model goes down.
In this tutorial I'll walk you from zero. Starting from how to get an API key, how to use it via the OpenAI SDK, how to pick a model using slugs, streaming, routing and fallback between models, provider preferences, cost and rate limit tracking, all the way to comparing model prices. Every example is in Python so it's easy to follow. Let's get started.
Introduction
Before we get into the technical stuff, I want you to understand first why OpenRouter matters and what it actually does behind the scenes. So here's the deal, OpenRouter sits as a gateway or intermediary. When you send a request to OpenRouter, it forwards that request to the model provider you chose, waits for the answer, then sends it back to you. All of this happens with a uniform format, namely the OpenAI Chat Completions format that has become the de facto standard in the industry.
Why does the OpenAI format matter? Because almost every library, framework, and tool in the LLM ecosystem right now already supports that format. So if you already have code running with the OpenAI SDK, you just swap the baseurl and apikey, and your code can immediately access hundreds of models through OpenRouter without needing to rewrite anything. That's what makes migration super easy.
There are several concrete benefits I felt when using OpenRouter. First, one bill for everything. You don't need to juggle billing in many places, just top up your balance on OpenRouter, and from there you can use any model. Second, you can compare model prices and performance easily because everything is in one place. Third, there are routing and fallback features that make your app more resilient. If one provider is erroring out or rate limiting, OpenRouter can automatically switch to another provider. Fourth, you get access to models that might be hard to sign up for directly, including open source models hosted by various infrastructure providers.
For me, the use case that hits hardest is when building a prototype. I very often want to try out, for example, whether this summarization task is better handled by Claude or GPT or a cheaper model like Llama. Without OpenRouter, I'd have to set up three accounts, three API keys, and maybe write three versions of code. With OpenRouter, I just swap one model name string and I can immediately compare the results. Saves time, saves energy, saves brain cells.
One thing you should know, OpenRouter isn't free in the sense that you still pay per token according to the price of the model you use. OpenRouter takes a small margin or sometimes they pass through the original price. But there are also some models that have a free tier with certain rate limits, perfect for early experimentation. So you can start without spending money first just to learn.
Instalasi
Okay now let's get into the hands-on part. The first thing you need to do is create an account on OpenRouter. It's super easy. Open the site https://openrouter.ai, then sign up. You can register with a Google account, GitHub, or a regular email. Once you're in, look for the Keys menu or go straight to https://openrouter.ai/keys. There you click the button to create a new API key. Give it a clear name, for example "local-experiment" or "production-app-A", so it's easy to manage when you later have many keys. Once created, copy the key and store it carefully. The format usually starts with sk-or-v1- followed by a long string. Remember, this key is only shown once, so if you lose it you have to create a new one.
Before you can use paid models, you need to top up your balance first. Go to the Credits or Billing page, then fill your balance using a credit card or another available payment method. You don't need a lot, for learning just a few dollars can go a long way because the price per token is really cheap. If you just want to try free models, you can skip this top up step for now, but the rate limits are tighter.
Now, because OpenRouter is compatible with OpenAI, we just install the official OpenAI library. Open your terminal and run this command.
pip install openai
If you want to use streaming or other features, this library is enough. But sometimes I also install requests for cases where I want to access OpenRouter endpoints outside the chat completions format, for example to check the list of models or check the balance. So might as well.
pip install openai requests
After that, the best practice I always follow is storing the API key in an environment variable, not hardcoding it directly in the code. This is really important so your key doesn't accidentally get committed to Git and become visible to others. You can create a .env file or export directly in the terminal.
export OPENROUTERAPIKEY="sk-or-v1-xxxxxxxxxxxxxxxxxxxx"
If you want to use a .env file, also install python-dotenv.
pip install python-dotenv
Then in your code you just load it like this.
import os
from dotenv import loaddotenv
loaddotenv()
apikey = os.environ["OPENROUTERAPIKEY"]
print("API key loaded:", apikey[:12], "...")
If the output shows the beginning of your key, that means the setup is done and we're ready to move on to the fun part.
Basic Usage
Now the most anticipated part, how to call a model through OpenRouter. Because it's OpenAI compatible, we use the OpenAI class from the library, but we point its baseurl to the OpenRouter endpoint, namely https://openrouter.ai/api/v1. This is the main key. Take a look at this most basic code.
import os
from openai import OpenAI
client = OpenAI(
baseurl="https://openrouter.ai/api/v1",
apikey=os.environ["OPENROUTERAPIKEY"],
)
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a friendly assistant who answers clearly."},
{"role": "user", "content": "Explain what OpenRouter is in one sentence."},
],
)
print(response.choices[0].message.content)
Pay close attention to the model part. There I wrote openai/gpt-4o-mini. Now, this is what we call the model slug in OpenRouter. The format is always provider/model-name. So if you want to use Claude, you write anthropic/claude-3.5-sonnet. Want Gemini, write google/gemini-2.0-flash-001. Want Llama, write meta-llama/llama-3.3-70b-instruct. Want Mistral, write mistralai/mistral-large. The concept is uniform, you just need to know the correct slug for the model you want.
To find out which slugs are available, you can open the Models page on the OpenRouter site, or you can query it via the API. I like the second way because it's more programmatic. Try running this.
import requests
resp = requests.get("https://openrouter.ai/api/v1/models")
data = resp.json()
for m in data["data"][:10]:
print(m["id"], "->", m.get("name"))
This models endpoint doesn't need authentication just to view the list, and it returns hundreds of models complete with pricing and context length information. We'll discuss pricing in more detail later on.
Now let's swap the earlier model to Claude, just to show how easy it is to hop between providers. The code is exactly the same, only the model string differs.
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[
{"role": "user", "content": "Write a short poem about learning to code."},
],
)
print(response.choices[0].message.content)
See that? I didn't change the client, didn't change the apikey, didn't change anything except the model name. This is the true power of OpenRouter. For me this genuinely changed the way I experiment because I get to focus on the application logic, not on integrating each provider.
There's one more thing I recommend you add when using OpenRouter, namely optional headers to identify your application. These headers aren't required, but they're useful if you want your app to appear on the OpenRouter leaderboard or just for tracking. You can add them via the extraheaders parameter.
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Hi, how are you?"}],
extraheaders={
"HTTP-Referer": "https://myapp.com",
"X-Title": "Ruby Learning App",
},
)
print(response.choices[0].message.content)
Now we can already call basic models. Next we discuss the more advanced features that make OpenRouter more than just an ordinary proxy.
Advanced Usage
In this section I'll cover the features that in my opinion make OpenRouter truly worth using in production. Starting from streaming, routing and fallback, provider preferences, all the way to cost and rate limit tracking. Let's go through them one by one.
Streaming Response
If you build a chat application, you definitely want the answer to appear word by word like ChatGPT, not sit silent then suddenly show everything. Well, that's called streaming. In OpenRouter, streaming is super easy because again it's OpenAI compatible. You just add the stream=True parameter.
stream = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[
{"role": "user", "content": "Tell the brief history of the Python language in 3 paragraphs."},
],
stream=True,
)
for chunk in stream:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content, end="", flush=True)
print()
With streaming, each chunk of text is printed as soon as it's received. This makes the user experience much nicer because they don't have to wait long. For a responsive app, streaming is almost mandatory in my opinion.
Model Routing and Fallback
This is my favorite feature in OpenRouter. Imagine this scenario, you use one main model, but sometimes that model is down, hitting a rate limit, or being slow. Instead of your request failing entirely, OpenRouter can automatically try another model as a backup. The way is using the models (plural) parameter which contains a list of models ordered by priority. Because this is an OpenRouter specific parameter outside the standard OpenAI spec, we send it via extrabody.
response = client.chat.completions.create(
model="openai/gpt-4o",
messages=[
{"role": "user", "content": "Summarize the main idea of the microservices concept."},
],
extrabody={
"models": [
"openai/gpt-4o",
"anthropic/claude-3.5-sonnet",
"google/gemini-2.0-flash-001",
],
},
)
print("Model used:", response.model)
print(response.choices[0].message.content)
Here OpenRouter will try gpt-4o first. If it fails, it moves to claude-3.5-sonnet, then if it still fails, to gemini-2.0-flash-001. Notice I print response.model so I know which model was eventually used. This is really useful for debugging and for making sure the fallback works. For a production app that needs high uptime, this feature is genuinely a lifesaver.
Besides manual fallback, OpenRouter also has a special model called openrouter/auto. If you use this model, OpenRouter will automatically pick the model that best fits your prompt based on their heuristics. Good if you're lazy to think and want it to decide.
response = client.chat.completions.create(
model="openrouter/auto",
messages=[
{"role": "user", "content": "Write a Python function to check for a prime number."},
],
)
print("Auto picked:", response.model)
print(response.choices[0].message.content)
Provider Preferences
A particular model, for example meta-llama/llama-3.3-70b-instruct, can be hosted by several different infrastructure providers. Each provider can have different pricing, speed, and privacy policies. Well, OpenRouter gives you control to pick which provider to use via the provider parameter in extrabody. For example you want to prioritize the cheapest, or the fastest, or you want to exclude a certain provider.
response = client.chat.completions.create(
model="meta-llama/llama-3.3-70b-instruct",
messages=[
{"role": "user", "content": "Explain what overfitting is in machine learning."},
],
extrabody={
"provider": {
"sort": "price",
"order": ["DeepInfra", "Together"],
"allowfallbacks": True,
},
},
)
print("Provider chosen via model:", response.model)
print(response.choices[0].message.content)
In this example, sort is set to price meaning OpenRouter will order providers from the cheapest. order gives a preference for a specific provider order, and allowfallbacks lets OpenRouter switch to another provider if the listed ones aren't available. You can also use sort: "throughput" if your priority is speed, or sort: "latency" if you want the fastest response to start coming out. There's also a datacollection option to control whether you want a provider that stores data or not, important if you care about privacy.
Cost and Usage Tracking
One thing I like about OpenRouter is its transparency about cost. Every response from the API already carries information about how many tokens were used. You can access it via response.usage.
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "What is the capital of Indonesia?"}],
)
usage = response.usage
print("Prompt tokens:", usage.prompttokens)
print("Completion tokens:", usage.completiontokens)
print("Total tokens:", usage.totaltokens)
But if you want to know the actual cost in dollars for one request, you can ask OpenRouter to give you more complete usage details via the usage parameter in extrabody. This will give you the real cost number.
response = client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Name the 3 planets closest to the sun."}],
extrabody={"usage": {"include": True}},
)
print("Answer:", response.choices[0].message.content)
Cost detail is in response.usage, including a cost field when available
print("Full usage:", response.usage)
Besides that, you can also check your remaining balance and total usage of your account via a special endpoint. This is useful for monitoring, for example you want to set an alert when the balance is about to run out.
import requests
import os
headers = {"Authorization": f"Bearer {os.environ['OPENROUTERAPIKEY']}"}
resp = requests.get("https://openrouter.ai/api/v1/credits", headers=headers)
data = resp.json()
print("Balance data:", data)
This /credits endpoint returns the total credits you've bought and the total you've used, so you can compute the remaining balance easily. I usually use this to build a simple dashboard that monitors daily spending.
Comparing Model Prices
Now this is interesting. Because the earlier models endpoint carries pricing info, we can build a script to automatically compare prices between models. Prices on OpenRouter are usually in dollars per token, so the numbers are really tiny. To make it readable, I like to convert to dollars per one million tokens.
import requests
resp = requests.get("https://openrouter.ai/api/v1/models")
models = resp.json()["data"]
target = ["openai/gpt-4o", "anthropic/claude-3.5-sonnet",
"google/gemini-2.0-flash-001", "meta-llama/llama-3.3-70b-instruct"]
print(f"{'Model':<45} {'Input/1M':>10} {'Output/1M':>10}")
for m in models:
if m["id"] in target:
p = m["pricing"]
inp = float(p["prompt"]) 1000000
out = float(p["completion"]) 1000000
print(f"{m['id']:<45} ${inp:>9.3f} ${out:>9.3f}")
This script pulls pricing data for the models you selected and shows the cost per one million input and output tokens. From here you can immediately see, for example, how much cheaper models like Gemini Flash or Llama are compared to GPT-4o for high volume use cases. I often use comparisons like this to decide which model is worth it for a particular feature. If the task is simple like classification or data extraction, I pick the cheap one. If it needs heavy reasoning, only then I bring out the premium model.
Combining All Features
To close the advanced section, I want to give an example that combines several features at once, namely fallback, provider preferences, and usage tracking in one ready-to-use function. This is similar to the pattern I use in real projects.
import os
from openai import OpenAI
client = OpenAI(
baseurl="https://openrouter.ai/api/v1",
apikey=os.environ["OPENROUTERAPIKEY"],
)
def askllm(prompt, mainmodel="openai/gpt-4o-mini"):
response = client.chat.completions.create(
model=mainmodel,
messages=[{"role": "user", "content": prompt}],
extrabody={
"models": [mainmodel, "anthropic/claude-3.5-sonnet", "google/gemini-2.0-flash-001"],
"provider": {"sort": "throughput", "allowfallbacks": True},
"usage": {"include": True},
},
extraheaders={"X-Title": "Ruby LLM Helper"},
)
return {
"answer": response.choices[0].message.content,
"modelused": response.model,
"totaltokens": response.usage.totaltokens,
}
result = askllm("Explain the difference between a list and a tuple in Python.")
print("Model:", result["modelused"])
print("Tokens:", result["totaltokens"])
print(result["answer"])
This askllm function already carries fallback to three models, provider priority based on throughput, and token tracking, all in one neat package. You just call it with any prompt and you get the answer plus useful metadata. Patterns like this make your code more robust and easier to monitor.
Best Practices
After using OpenRouter for a while, there are several practices I really want to share with you so you don't repeat the mistakes I once made. The first and most important, never hardcode your API key in your code. Always use an environment variable or a secret manager. I already touched on this in the installation section, but this is genuinely crucial. I've seen people accidentally commit their API key to a public repo and their balance got drained within hours. So please, be really careful with your keys.
Second, always use explicit model slugs in production, don't rely on openrouter/auto for critical things. auto is nice for experimentation, but because it can pick different models each time, the results become less predictable. For production, I prefer setting a clear main model then giving a clear fallback too, so I know exactly the behavior and the cost.
Third, take advantage of the fallback feature for apps that need high reliability. The combination of a main model plus two or three backups makes your app far more resilient when a provider is having issues. But be careful too, pick backup models whose quality isn't too far off from the main model, so the user experience stays consistent. Don't let your main model be GPT-4o but the fallback is a tiny model whose answers are far worse.
Fourth, always monitor costs. OpenRouter gives you usage data per request and a credits endpoint to check balance. Use those to build monitoring. I recommend you log every request along with the tokens used and the model chosen. With this data you can analyze which feature is the most token hungry and optimize from there. Sometimes just by shortening the system prompt or switching to a cheaper model for light tasks, you can save significant cost.
Fifth, handle rate limits and error handling well. Even though OpenRouter already has fallback, there's still a chance all models fail or you hit a rate limit on OpenRouter's own side. So wrap your calls with try-except and add retry with backoff if needed. Here's a simple example.
import time
from openai import OpenAI, APIError
def askwithretry(client, prompt, model, maxretry=3):
for attempt in range(maxretry):
try:
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
)
return resp.choices[0].message.content
except APIError as e:
wait = 2 ** attempt
print(f"Error: {e}. Retrying in {wait} seconds...")
time.sleep(wait)
return "Failed after several attempts."
A retry pattern with exponential backoff like this keeps your app from giving up the moment there's a momentary disturbance. The wait time doubles each attempt, so you don't spam the server while it's having problems.
Sixth, consider the context length of each model. Different models have different context limits. Some are only eight thousand tokens, some go up to millions of tokens. If you send a prompt that's too long to a model with a small context, your request will fail. This context length info is in the earlier models endpoint, so you can check first before sending. For long documents, pick a model that actually has a large context.
Seventh, pay attention to data privacy policies via provider settings. If you process sensitive data, use the datacollection option to make sure your data isn't stored by the provider. This is really important if you're in an industry that's strict about compliance like healthcare or finance. OpenRouter gives you this control, so use it.
Lastly, get into the habit of reading the official OpenRouter documentation regularly. This LLM ecosystem moves really fast, new models appear every week, prices change, new features get added. By diligently checking, you'll always stay up to date with the best and cheapest models available. I myself often find new models that are better and cheaper just by casually checking their models page.
Conclusion
Okay everyone, we've reached the end of this tutorial. We've covered OpenRouter fairly thoroughly starting from its basic concept as a universal gateway for hundreds of LLMs, how to get an API key, how to use it via the OpenAI SDK by just swapping the baseurl, how to pick a model using slugs in the provider/model-name format, streaming, routing and fallback between models, provider preferences, cost and rate limit tracking, all the way to comparing model prices automatically. All of that you can do with one API key and one uniform format, and in my opinion that genuinely simplifies the life of developers working with LLMs.
For me personally, OpenRouter has become a standard tool every time I start a new project involving LLMs. The ease of hopping between models without changing code makes the experimentation phase far faster, and the fallback feature makes me feel at ease once I'm in production. If you frequently switch providers or you want to build an app that's resilient, I highly recommend you try OpenRouter.
My advice, just go straight to practice. Take one code example from this tutorial, run it on your machine, then start exploring. Try swapping models, try the fallback feature, try comparing prices. From there you'll quickly understand and find the pattern that best fits your needs. Don't be afraid to experiment because the cost is cheap and you can start from free models first if you want.
That's it from me for now. Hopefully this tutorial is useful and makes you more confident building LLM based applications. If you have questions or you find a cool trick while using OpenRouter, don't hesitate to share. Happy coding, and see you in the next tutorial. Cheers from me, Ruby Abdullah.