Complete Aisuite Tutorial: Unified Multi-LLM Interface by Andrew Ng
What is Aisuite?
Aisuite is an open-source Python library developed by Andrew Ng's team that enables developers to interact with multiple Large Language Model (LLM) providers through a single, unified interface. With aisuite, you can switch between OpenAI, Anthropic, Google, Mistral, AWS Bedrock, Azure, and other providers simply by changing a single string identifier, without modifying your core code.
Imagine you're building an AI application and want to compare the performance between GPT-4o, Claude Sonnet, and Gemini for a specific use case. Without aisuite, you'd need to write different code for each provider's SDK. With aisuite, just change the model identifier string and all your other code stays the same.
The library uses an interface compatible with the OpenAI Chat Completions format, so if you're already familiar with the OpenAI SDK, transitioning to aisuite is straightforward.
Why Use Aisuite?
There are several compelling reasons to consider aisuite in your AI workflow:
openai:gpt-4o to anthropic:claude-sonnet-4-6Installation
Basic Installation
To install aisuite, use pip:
pip install aisuite
Provider-Specific Installation
Aisuite supports selective per-provider installation. This is useful so you don't need to install all provider SDKs when you only need a few:
# Install with OpenAI support only
pip install "aisuite[openai]"
Install with Anthropic support only
pip install "aisuite[anthropic]"
Install with multiple providers
pip install "aisuite[openai,anthropic,google]"
Install all supported providers
pip install "aisuite[all]"
Supported Providers
Here's a list of providers supported by aisuite:
| Provider | Identifier | Package |
|----------|-----------|---------|
| OpenAI | openai | openai |
| Anthropic | anthropic | anthropic |
| Google (Gemini) | google | google-generativeai |
| Mistral | mistral | mistralai |
| AWS Bedrock | aws | boto3 |
| Azure OpenAI | azure | openai |
| Groq | groq | groq |
| Cohere | cohere | cohere |
| Fireworks AI | fireworks | fireworks-ai |
| Together AI | together | together |
| Sambanova | sambanova | openai |
| Watsonx | watsonx | ibm-watsonx-ai |
API Key Configuration
Aisuite reads API keys from environment variables. Each provider has a standard environment variable name:
# Set environment variables
export OPENAIAPIKEY="sk-..."
export ANTHROPICAPIKEY="sk-ant-..."
export GOOGLEAPIKEY="AIza..."
export MISTRALAPIKEY="..."
export GROQAPIKEY="gsk..."
You can also use a .env file with python-dotenv:
from dotenv import loaddotenv
loaddotenv()
Or set API keys directly in code through provider configuration parameters:
import aisuite as ai
client = ai.Client()
client.configure({"openai": {"apikey": "sk-..."}})
Basic Usage
Simple Chat Completion
Here's a basic example of using aisuite to create a chat completion:
import aisuite as ai
client = ai.Client()
messages = [
{"role": "system", "content": "You are a helpful assistant that answers concisely."},
{"role": "user", "content": "Explain what machine learning is in 2 sentences."}
]
Using OpenAI GPT-4o
response = client.chat.completions.create(
model="openai:gpt-4o",
messages=messages
)
print("GPT-4o:", response.choices[0].message.content)
Note the model identifier format: provider:modelname. This is aisuite's convention for specifying the provider and model to use.
Comparing Multiple Models
One of aisuite's main advantages is the ability to easily compare responses from different models:
import aisuite as ai
client = ai.Client()
models = [
"openai:gpt-4o",
"anthropic:claude-sonnet-4-6",
"google:gemini-2.0-flash",
"mistral:mistral-large-latest"
]
messages = [
{"role": "system", "content": "You are an experienced data scientist."},
{"role": "user", "content": "What are the first 3 steps to start a machine learning project?"}
]
for model in models:
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=0.7
)
providername = model.split(":")[0]
print(f"\n{'='50}")
print(f"Provider: {providername}")
print(f"Model: {model}")
print(f"{'='50}")
print(response.choices[0].message.content)
Using Additional Parameters
Aisuite supports standard parameters from the OpenAI Chat Completions API:
import aisuite as ai
client = ai.Client()
messages = [
{"role": "user", "content": "Write a short poem about Python programming."}
]
response = client.chat.completions.create(
model="anthropic:claude-sonnet-4-6",
messages=messages,
temperature=0.9,
maxtokens=500,
topp=0.95
)
print(response.choices[0].message.content)
Advanced Usage
Building a Model Selection Pipeline
In production, you may want to select models based on specific criteria such as cost, speed, or quality. Here's an example implementation:
import aisuite as ai
import time
client = ai.Client()
MODELTIERS = {
"fast": "groq:llama-3.3-70b-versatile",
"balanced": "openai:gpt-4o-mini",
"quality": "anthropic:claude-sonnet-4-6",
"premium": "openai:gpt-4o"
}
def smartcompletion(messages, tier="balanced", kwargs):
"""Select model based on the desired tier."""
model = MODELTIERS.get(tier, MODELTIERS["balanced"])
starttime = time.time()
response = client.chat.completions.create(
model=model,
messages=messages,
kwargs
)
elapsed = time.time() - starttime
return {
"content": response.choices[0].message.content,
"model": model,
"tier": tier,
"latencyseconds": round(elapsed, 2)
}
Example usage
messages = [{"role": "user", "content": "What is gradient descent?"}]
For quick, simple queries
resultfast = smartcompletion(messages, tier="fast")
print(f"[{resultfast['tier']}] {resultfast['latencyseconds']}s - {resultfast['model']}")
For queries requiring high quality
resultquality = smartcompletion(messages, tier="quality")
print(f"[{resultquality['tier']}] {resultquality['latencyseconds']}s - {resultquality['model']}")
Fallback and Retry Pattern
Implementing a fallback pattern when a provider experiences an error:
import aisuite as ai
import time
client = ai.Client()
FALLBACKCHAIN = [
"anthropic:claude-sonnet-4-6",
"openai:gpt-4o",
"google:gemini-2.0-flash",
"groq:llama-3.3-70b-versatile"
]
def resilientcompletion(messages, maxretries=2, kwargs):
"""Try multiple providers sequentially if the primary one fails."""
for model in FALLBACKCHAIN:
for attempt in range(maxretries):
try:
response = client.chat.completions.create(
model=model,
messages=messages,
kwargs
)
return {
"content": response.choices[0].message.content,
"model": model,
"attempts": attempt + 1
}
except Exception as e:
print(f"Error with {model} (attempt {attempt + 1}): {e}")
time.sleep(1)
continue
print(f"All retries failed for {model}, trying next provider...")
raise Exception("All providers failed after all retry attempts.")
Example usage
messages = [{"role": "user", "content": "Explain the attention mechanism concept."}]
result = resilientcompletion(messages, temperature=0.3)
print(f"Succeeded with model: {result['model']}")
print(f"Number of attempts: {result['attempts']}")
print(result['content'])
Batch Processing with Multi-Model
Processing multiple prompts simultaneously using various models:
import aisuite as ai
from concurrent.futures import ThreadPoolExecutor, ascompleted
client = ai.Client()
def processsingle(model, messages, taskid):
"""Process a single request to one model."""
try:
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=0.3
)
return {
"taskid": taskid,
"model": model,
"content": response.choices[0].message.content,
"success": True
}
except Exception as e:
return {
"taskid": taskid,
"model": model,
"content": str(e),
"success": False
}
List of tasks to process
tasks = [
{"id": 1, "prompt": "What is a neural network?"},
{"id": 2, "prompt": "Explain random forest."},
{"id": 3, "prompt": "What is transfer learning?"},
]
models = ["openai:gpt-4o-mini", "anthropic:claude-sonnet-4-6"]
Run in parallel
results = []
with ThreadPoolExecutor(maxworkers=6) as executor:
futures = []
for task in tasks:
messages = [{"role": "user", "content": task["prompt"]}]
for model in models:
future = executor.submit(processsingle, model, messages, task["id"])
futures.append(future)
for future in ascompleted(futures):
result = future.result()
results.append(result)
status = "OK" if result["success"] else "FAIL"
print(f"[{status}] Task {result['taskid']} - {result['model']}")
Summary
successcount = sum(1 for r in results if r["success"])
print(f"\nTotal: {successcount}/{len(results)} succeeded")
Custom Provider Configuration
Some providers require additional configuration. Here are examples for AWS Bedrock and Azure:
import aisuite as ai
client = ai.Client()
Configuration for AWS Bedrock
client.configure({
"aws": {
"regionname": "us-east-1",
"awsaccesskeyid": "AKIA...",
"awssecretaccesskey": "..."
}
})
Using a model from AWS Bedrock
response = client.chat.completions.create(
model="aws:anthropic.claude-sonnet-4-6-20250514-v1:0",
messages=[{"role": "user", "content": "Hello from AWS Bedrock!"}]
)
print(response.choices[0].message.content)
Configuration for Azure OpenAI
client.configure({
"azure": {
"apikey": "...",
"baseurl": "https://your-resource.openai.azure.com/openai/deployments/your-deployment"
}
})
Building a Simple Evaluation Framework
Use aisuite to build a simple evaluation framework that compares output quality from different models:
import aisuite as ai
import json
client = ai.Client()
def evaluatemodels(testcases, models, judgemodel="openai:gpt-4o"):
"""Evaluate multiple models using the LLM-as-judge pattern."""
results = {}
for model in models:
results[model] = {"scores": [], "responses": []}
for test in testcases:
# Get response from the model being evaluated
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": test["prompt"]}],
temperature=0.3
)
answer = response.choices[0].message.content
# Use judge model to evaluate
judgeprompt = f"""Rate the following response based on these criteria:
Accuracy (1-5)
Completeness (1-5)
Clarity (1-5)
Question: {test['prompt']}
Response being evaluated: {answer}
Reference answer: {test.get('reference', 'None provided')}
Provide scores in JSON format:
{{"accuracy": X, "completeness": X, "clarity": X, "total": X}}
Only provide JSON, no other explanation."""
judgeresponse = client.chat.completions.create(
model=judgemodel,
messages=[{"role": "user", "content": judgeprompt}],
temperature=0
)
try:
scores = json.loads(judgeresponse.choices[0].message.content)
results[model]["scores"].append(scores)
except json.JSONDecodeError:
results[model]["scores"].append({"total": 0})
results[model]["responses"].append(answer)
# Calculate average scores
for model in models:
scores = results[model]["scores"]
if scores:
avgtotal = sum(s.get("total", 0) for s in scores) / len(scores)
results[model]["avgscore"] = round(avgtotal, 2)
return results
Example usage
testcases = [
{
"prompt": "Explain the difference between supervised and unsupervised learning.",
"reference": "Supervised learning uses labeled data to train models, while unsupervised learning discovers patterns from unlabeled data."
},
{
"prompt": "What is overfitting and how do you prevent it?",
"reference": "Overfitting occurs when a model fits too closely to training data. It can be addressed with regularization, cross-validation, and adding more data."
}
]
modelstoevaluate = [
"openai:gpt-4o-mini",
"anthropic:claude-sonnet-4-6",
"google:gemini-2.0-flash"
]
results = evaluatemodels(testcases, modelstoevaluate)
Display results
print("\n=== Evaluation Results ===")
for model, data in sorted(results.items(), key=lambda x: x[1].get("avgscore", 0), reverse=True):
print(f"{model}: Average Score = {data.get('avgscore', 'N/A')}")
Best Practices
1. Use Environment Variables for API Keys
Never hardcode API keys in source code. Always use environment variables or a secret manager:
import os
from dotenv import loaddotenv
loaddotenv()
Validate that API keys are available
requiredkeys = ["OPENAIAPIKEY", "ANTHROPICAPIKEY"]
for key in requiredkeys:
if not os.getenv(key):
raise ValueError(f"Environment variable {key} not found")
2. Implement Proper Error Handling
Every call to an LLM provider can fail. Make sure your code handles errors gracefully:
import aisuite as ai
client = ai.Client()
def safecompletion(model, messages, kwargs):
"""Wrapper with proper error handling."""
try:
response = client.chat.completions.create(
model=model,
messages=messages,
kwargs
)
return response.choices[0].message.content
except ConnectionError:
print(f"Connection failed to {model}")
return None
except TimeoutError:
print(f"Timeout for {model}")
return None
except Exception as e:
print(f"Unexpected error with {model}: {e}")
return None
3. Use Caching for Cost Efficiency
Avoid making repeated API calls for the same input:
import hashlib
import json
cache = {}
def cached
completion(client, model, messages, kwargs):
"""Completion with simple in-memory cache."""
cachekey = hashlib.md5(
json.dumps({"model": model, "messages": messages,
kwargs}, sortkeys=True).encode()
).hexdigest()
if cachekey in cache:
return cache[cachekey]
response = client.chat.completions.create(
model=model,
messages=messages,
kwargs
)
result = response.choices[0].message.content
cache[cachekey] = result
return result
4. Logging and Monitoring
Log every API call for monitoring and debugging:
import logging
import time
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("aisuite")
def loggedcompletion(client, model, messages, kwargs):
"""Completion with comprehensive logging."""
start = time.time()
logger.info(f"Request to {model} - {len(messages)} messages")
try:
response = client.chat.completions.create(
model=model, messages=messages, kwargs
)
elapsed = time.time() - start
content = response.choices[0].message.content
logger.info(f"Response from {model} - {len(content)} chars - {elapsed:.2f}s")
return content
except Exception as e:
elapsed = time.time() - start
logger.error(f"Error from {model} after {elapsed:.2f}s: {e}")
raise
5. Limit Concurrency
When batch processing, limit the number of parallel requests to avoid exceeding rate limits:
from concurrent.futures import ThreadPoolExecutor
import time
MAXCONCURRENT = 5
RATELIMITDELAY = 0.5
def batchprocess(client, prompts, model, maxworkers=MAXCONCURRENT):
"""Batch processing with rate limiting."""
results = []
with ThreadPoolExecutor(maxworkers=maxworkers) as executor:
futures = []
for i, prompt in enumerate(prompts):
messages = [{"role": "user", "content": prompt}]
future = executor.submit(
client.chat.completions.create,
model=model,
messages=messages
)
futures.append((i, future))
time.sleep(RATELIMITDELAY)
for i, future in futures:
try:
response = future.result(timeout=60)
results.append({
"index": i,
"content": response.choices[0].message.content,
"success": True
})
except Exception as e:
results.append({
"index": i,
"content": str(e),
"success": False
})
return sorted(results, key=lambda x: x["index"])
Case Study: Building a Multi-LLM Chatbot
Here's a complete example of building a simple chatbot that supports dynamic model selection:
import aisuite as ai
client = ai.Client()
AVAILABLEMODELS = {
"1": ("openai:gpt-4o", "GPT-4o (OpenAI)"),
"2": ("anthropic:claude-sonnet-4-6", "Claude Sonnet 4.6 (Anthropic)"),
"3": ("google:gemini-2.0-flash", "Gemini 2.0 Flash (Google)"),
"4": ("groq:llama-3.3-70b-versatile", "Llama 3.3 70B (Groq)")
}
def runchatbot():
print("=== Multi-LLM Chatbot ===")
print("Select a model:")
for key, (, name) in AVAILABLEMODELS.items():
print(f" {key}. {name}")
choice = input("\nChoice (1-4): ").strip()
model, modelname = AVAILABLEMODELS.get(choice, AVAILABLEMODELS["1"])
print(f"\nUsing: {modelname}")
print("Type 'quit' to exit, 'switch' to change model.\n")
conversation = [
{"role": "system", "content": "You are a helpful AI assistant."}
]
while True:
userinput = input("You: ").strip()
if userinput.lower() == "quit":
break
if userinput.lower() == "switch":
print("\nSelect a model:")
for key, (, name) in AVAILABLEMODELS.items():
print(f" {key}. {name}")
choice = input("Choice: ").strip()
model, modelname = AVAILABLEMODELS.get(choice, (model, modelname))
print(f"Switched to: {modelname}\n")
continue
conversation.append({"role": "user", "content": userinput})
try:
response = client.chat.completions.create(
model=model,
messages=conversation,
temperature=0.7
)
assistantmessage = response.choices[0].message.content
conversation.append({"role": "assistant", "content": assistantmessage})
print(f"\n{modelname}: {assistantmessage}\n")
except Exception as e:
print(f"\nError: {e}\n")
if name == "main":
runchatbot()
Additional Tips
Choosing the Right Model
Each model has different strengths:
- GPT-4o: Great for general tasks, coding, and reasoning
- Claude Sonnet: Excellent for writing, long analysis, and instruction following
- Gemini Flash: Fast and efficient for tasks requiring short responses
- Llama (via Groq): Low latency, good for real-time applications
- Mistral Large*: Strong performance for multilingual tasks
Cost Estimation
Use aisuite to estimate costs before running large batches:
COSTPER1KTOKENS = {
"openai:gpt-4o": {"input": 0.0025, "output": 0.01},
"openai:gpt-4o-mini": {"input": 0.00015, "output": 0.0006},
"anthropic:claude-sonnet-4-6": {"input": 0.003, "output": 0.015},
"google:gemini-2.0-flash": {"input": 0.0001, "output": 0.0004},
}
def estimate
cost(model, numrequests, avginputtokens=500, avgoutputtokens=300):
costs = COST
PER1KTOKENS.get(model, {"input": 0, "output": 0})
totalinput = (numrequests avginputtokens / 1000) costs["input"]
totaloutput = (numrequests avgoutputtokens / 1000) * costs["output"]
return round(totalinput + totaloutput, 4)
Example: estimate cost for 1000 requests
for model in COSTPER1KTOKENS:
cost = estimatecost(model, 1000)
print(f"{model}: ~${cost} for 1000 requests")
Conclusion
Aisuite is an elegant solution to a common problem faced by AI developers: managing integrations with multiple LLM providers. With a unified interface that follows the OpenAI Chat Completions standard, this library makes work more efficient without sacrificing flexibility.
Key takeaways:
- Aisuite simplifies multi-LLM usage through a single unified interface
- The
provider:model_namemodel identifier format makes switching between providers effortless - The library supports more than 10 providers including OpenAI, Anthropic, Google, and AWS
- It's ideal for model evaluation, fallback patterns, and building resilient AI applications
- Follow best practices: use environment variables, implement error handling, and mind rate limits
For more information, visit the official aisuite repository on GitHub: github.com/andrewyng/aisuite.