Opik Tutorial: LLM Evaluation and Observability Platform

Opik is an open-source platform developed by Comet for evaluating, testing, and monitoring Large Language Model (LLM) applications. In the modern AI ecosystem, building LLM applications is only half the battle. You need to ensure that model responses are consistent, accurate, and meet expectations. Opik provides a complete toolkit for tracing, automated evaluation, and production monitoring that helps AI teams develop LLM applications with confidence.

This tutorial covers how to use Opik from installation, tracing LLM requests, creating automated evaluations, to monitoring applications in production. All code examples can be run directly in your local Python environment.

Why Opik?

When building LLM applications, there are several key challenges:

Opik addresses all these challenges by providing:

• Tracing: Records every step in your LLM pipeline in detail
• Evaluation: Automated evaluation framework with built-in and custom metrics
• Dataset Management: Manage test datasets for reproducible evaluations
• Monitoring Dashboard: Real-time visualization of LLM application performance

Installation

Install via pip

pip install opik

Configure Opik

There are two ways to use Opik: using Comet Cloud (hosted) or self-hosted.

Option 1: Comet Cloud (Recommended to get started)

opik configure

This command will prompt for your Comet API key. Sign up for free at comet.com if you do not have an account yet.

Option 2: Self-Hosted with Docker

# Clone the repository
git clone https://github.com/comet-ml/opik.git
cd opik

Run with Docker Compose
docker compose up -d

Once running, the Opik dashboard is available at http://localhost:5173.

Configure the client for self-hosted:

import opik

opik.configure(uselocal=True)

import opik

client = opik.Opik()
print("Opik connected successfully!")
print(f"Workspace: {client.workspace}")

import opik
from openai import OpenAI

client = OpenAI()

@opik.track
def generateresponse(prompt: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

@opik.track
def processquery(userquery: str) -> dict:
    enhancedprompt = f"Answer the following question clearly and concisely:\n{userquery}"
    response = generateresponse(enhancedprompt)
    return {
        "query": userquery,

        "response": response
    }

result = processquery("What is machine learning?")
print(result["response"])

Every function call decorated with @opik.track is automatically recorded as a span in the trace. Nested functions automatically form a parent-child hierarchy.

Tracing with Metadata

You can add additional metadata to traces for richer context:

import opik
from opik import opikcontext


@opik.track
def classifytext(text: str, categories: list) -> str:
    opikcontext.updatecurrenttrace(

        tags=["classification", "production"],
        metadata={
            "model": "gpt-4o-mini",
            "numcategories": len(categories),
            "textlength": len(text)

        }
    )

    prompt = f"""Classify the following text into one of these categories: {', '.join(categories)}

Text: {text}

Category:"""

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}]
    )

    result = response.choices[0].message.content.strip()

    opikcontext.updatecurrentspan(
        metadata={"resultcategory": result}

    )

    return result

result = classifytext(
    "BBCA stock price rose 2% today",
    ["finance", "technology", "sports", "politics"]
)
print(f"Category: {result}")

OpenAI Integration

Opik provides direct integration with OpenAI so all API calls are automatically traced:

from opik.integrations.openai import trackopenai
from openai import OpenAI

client = OpenAI()
client = trackopenai(client)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the concept of RAG in 2 sentences."}
    ]
)

print(response.choices[0].message.content)

LangChain Integration

If you use LangChain, Opik provides a callback handler:

from opik.integrations.langchain import OpikTracer
from langchainopenai import ChatOpenAI

from langchaincore.prompts import ChatPromptTemplate

opiktracer = OpikTracer()


llm = ChatOpenAI(model="gpt-4o-mini")

prompt = ChatPromptTemplate.frommessages([
    ("system", "You are an expert in {topic}."),
    ("user", "{question}")
])

chain = prompt | llm

response = chain.invoke(
    {"topic": "data science", "question": "What is the difference between supervised and unsupervised learning?"},
    config={"callbacks": [opiktracer]}

)

print(response.content)

import opik
from openai import OpenAI

client = OpenAI()

@opik.track(name="embedquery")
def embedquery(query: str) -> list:

    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=query
    )
    return response.data[0].embedding

@opik.track(name="retrievedocuments")
def retrievedocuments(queryembedding: list, topk: int = 3) -> list:

    documents = [
        {"id": 1, "text": "Python is a popular programming language for data science.", "score": 0.92},
        {"id": 2, "text": "Machine learning is a branch of artificial intelligence.", "score": 0.87},
        {"id": 3, "text": "Data preprocessing is an important step before training a model.", "score": 0.83},
    ]
    return documents[:topk]

@opik.track(name="generateanswer")

def generateanswer(query: str, contextdocs: list) -> str:

    context = "\n".join([doc["text"] for doc in contextdocs])

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Answer based on the following context:\n{context}"},
            {"role": "user", "content": query}
        ]
    )
    return response.choices[0].message.content

@opik.track(name="ragpipeline")

def ragpipeline(query: str) -> dict:
    embedding = embedquery(query)

    documents = retrievedocuments(embedding)
    answer = generateanswer(query, documents)


    return {
        "query": query,
        "numdocsretrieved": len(documents),

        "answer": answer
    }

result = ragpipeline("How do I get started with data science?")
print(result["answer"])

In the Opik dashboard, you will see the complete trace with hierarchy: ragpipeline -> embedquery -> retrievedocuments -> generateanswer, complete with inputs and outputs for each step.

Automated Evaluation

Evaluation is the feature that makes Opik truly powerful. You can automatically evaluate LLM outputs using built-in or custom metrics.

Creating a Dataset

First, create a dataset for evaluation:

import opik

client = opik.Opik()

dataset = client.getorcreatedataset(name="qa-evaluation-dataset")


dataset.insert([
    {
        "input": {"question": "What is Python?"},
        "expectedoutput": "Python is a high-level programming language popular for data science, web development, and scripting."
    },
    {
        "input": {"question": "What is pandas used for?"},
        "expectedoutput": "Pandas is a Python library for data manipulation and analysis, especially tabular data."

    },
    {
        "input": {"question": "Explain what a neural network is."},
        "expectedoutput": "A neural network is a computational model inspired by the human brain, consisting of interconnected layers of neurons that process information."
    },
    {
        "input": {"question": "What is the difference between SQL and NoSQL?"},
        "expectedoutput": "SQL uses a relational schema with structured tables, while NoSQL uses flexible data models like documents, key-value pairs, or graphs."

    },
    {
        "input": {"question": "What is an API?"},
        "expectedoutput": "An API (Application Programming Interface) is an interface that allows two applications to communicate with each other."
    }
])

print(f"Dataset '{dataset.name}' created with {len(dataset)} items")

Using Built-in Metrics

Opik provides several built-in metrics for LLM evaluation:

from opik.evaluation.metrics import Hallucination, AnswerRelevance, ContextRecall
from opik.evaluation import evaluate

@opik.track
def qatask(inputdata: dict) -> dict:
    question = inputdata["question"]

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}]
    )
    answer = response.choices[0].message.content
    return {
        "output": answer,
        "context": [f"Related question: {question}"]
    }

hallucinationmetric = Hallucination()
relevancemetric = AnswerRelevance()


results = evaluate(
    dataset=dataset,
    task=qatask,
    scoringmetrics=[hallucinationmetric, relevancemetric],

    experimentname="qa-evaluation-v1"
)

print(f"Evaluation complete!")
print(f"Average Hallucination Score: {results.averagescores.get('hallucination', 'N/A')}")

print(f"Average Relevance Score: {results.averagescores.get('answerrelevance', 'N/A')}")

from opik.evaluation.metrics import basemetric, scoreresult

class ResponseLengthMetric(basemetric.BaseMetric):
    def init(self, minlength: int = 50, maxlength: int = 500):
        self.name = "responselength"

        self.minlength = minlength

        self.maxlength = maxlength


    def score(self, output: str, kwargs) -> scoreresult.ScoreResult:

        length = len(output)

        if self.minlength <= length <= self.maxlength:
            scorevalue = 1.0

            reason = f"Response length ({length}) is within acceptable range [{self.minlength}, {self.maxlength}]"

        elif length < self.minlength:
            scorevalue = length / self.minlength
            reason = f"Response too short ({length} < {self.minlength})"

        else:
            scorevalue = max(0, 1 - (length - self.maxlength) / self.maxlength)
            reason = f"Response too long ({length} > {self.maxlength})"


        return scoreresult.ScoreResult(
            name=self.name,
            value=scorevalue,

            reason=reason
        )

class KeywordPresenceMetric(basemetric.BaseMetric):
    def init(self, requiredkeywords: list):

        self.name = "keywordpresence"
        self.requiredkeywords = [kw.lower() for kw in requiredkeywords]

    def score(self, output: str, kwargs) -> scoreresult.ScoreResult:

        outputlower = output.lower()
        found = [kw for kw in self.requiredkeywords if kw in outputlower]
        scorevalue = len(found) / len(self.requiredkeywords) if self.requiredkeywords else 1.0


        return scoreresult.ScoreResult(
            name=self.name,
            value=scorevalue,

            reason=f"Found {len(found)}/{len(self.requiredkeywords)} keywords: {found}"
        )

lengthmetric = ResponseLengthMetric(minlength=30, maxlength=300)

keywordmetric = KeywordPresenceMetric(requiredkeywords=["python", "programming"])


results = evaluate(
    dataset=dataset,
    task=qatask,
    scoringmetrics=[lengthmetric, keywordmetric],

    experimentname="qa-custom-metrics-v1"
)

Evaluation with LLM-as-Judge

Opik supports using an LLM as an evaluator, which is very useful for semantically assessing response quality:

from opik.evaluation.metrics import LLMJudge

judgemetric = LLMJudge(

    name="qualityjudge",
    model="gpt-4o-mini",
    prompttemplate="""Evaluate the quality of the following answer based on the given question.


Question: {input}
Answer: {output}
Expected Answer: {expectedoutput}

Score from 0-1 based on these criteria:
Accuracy: Is the answer correct?
Completeness: Does the answer cover key points?
Clarity: Is the answer easy to understand?

Score (0-1):""",
)

results = evaluate(
    dataset=dataset,
    task=qatask,

    scoringmetrics=[judgemetric],

    experimentname="qa-llm-judge-v1"
)

Comparing Experiments

One of Opik's strengths is the ability to compare results across experiments:

@opik.track
def qataskdetailed(inputdata: dict) -> dict:

    question = inputdata["question"]
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Answer in detail and provide examples when possible. Use clear and accessible language."},
            {"role": "user", "content": question}
        ],
        temperature=0.3
    )
    return {"output": response.choices[0].message.content}

resultsv2 = evaluate(

    dataset=dataset,
    task=qataskdetailed,

    scoringmetrics=[hallucinationmetric, relevancemetric, lengthmetric],

    experimentname="qa-evaluation-v2-detailed"
)

In the Opik dashboard, you can compare qa-evaluation-v1 and qa-evaluation-v2-detailed side-by-side to see the impact of prompt changes.

Production Monitoring

Opik can also be used for monitoring LLM applications in production.

Logging User Feedback

import opik

client = opik.Opik()

@opik.track(name="productionquery", projectname="my-llm-app-production")
def handleuserquery(query: str, userid: str) -> str:

    from opik import opikcontext

    opikcontext.updatecurrenttrace(

        tags=["production"],
        metadata={
            "userid": userid,

            "environment": "production"
        }
    )

    response = openaiclient.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": query}]
    )

    answer = response.choices[0].message.content

    traceid = opikcontext.getcurrenttracedata().id


    return answer, traceid

def submitfeedback(traceid: str, score: float, reason: str = ""):
    client.logtracesfeedback(
        traceids=[traceid],
        scores=[score],
        reasons=[reason],
        name="usersatisfaction"

    )

answer, traceid = handleuserquery(
    query="How do I use Docker?",
    userid="user123"
)

submitfeedback(

    traceid=traceid,

    score=0.9,
    reason="Complete and easy to understand answer"
)

client.createonlinescoringrule(
    projectname="my-llm-app-production",

    name="auto-hallucination-check",
    metric=Hallucination(),
    samplingrate=0.1
)

This will automatically evaluate 10% of all production traces using the hallucination metric.

Prompt Management

Opik also provides prompt management features for versioning and tracking prompts:

import opik

client = opik.Opik()

prompt = client.createprompt(

    name="qa-system-prompt",
    prompt="You are an AI assistant that helps answer questions about {topic}. "
           "Answer clearly, accurately, and concisely. "
           "If you do not know the answer, say so honestly."
)

print(f"Prompt version: {prompt.commit}")

retrievedprompt = client.getprompt(name="qa-system-prompt")


formatted = retrievedprompt.format(topic="machine learning")
print(formatted)

Prompt Versioning

promptv2 = client.createprompt(
    name="qa-system-prompt",
    prompt="You are an AI assistant specializing in {topic}. "
           "Provide structured answers with bullet points when appropriate. "
           "Include practical examples when possible. "
           "If you do not know the answer, direct the user to relevant resources."
)

print(f"Updated prompt version: {promptv2.commit}")


history = client.getpromptversions(name="qa-system-prompt")

for version in history:
    print(f"Version {version.commit}: created at {version.createdat}")

Best Practices

1. Good Tracing Structure

Give descriptive names to every trace and span:

@opik.track(name="customersupportpipeline")
def handlesupportticket(ticket: dict):
    intent = classifyintent(ticket["message"])

    context = retrieveknowledge(intent, ticket["message"])
    response = generateresponse(ticket["message"], context)

    return response

@opik.track
def processrequest(request: dict):
    from opik import opikcontext


    opikcontext.updatecurrenttrace(
        tags=[
            f"model:{request.get('model', 'default')}",
            f"env:{request.get('environment', 'dev')}",
            f"priority:{request.get('priority', 'normal')}"
        ]
    )

3. Evaluate Regularly

Run evaluations regularly, especially when:

• Changing prompts or system messages
• Upgrading model versions
• Adding new data to the knowledge base
• Before and after deploying changes

4. Separate Projects per Environment

@opik.track(projectname="my-app-development")
def devpipeline(query: str):
    pass

@opik.track(projectname="my-app-staging")

def stagingpipeline(query: str):
    pass

@opik.track(projectname="my-app-production")

def prodpipeline(query: str):
    pass

5. Save Evaluation Results as Baselines

Whenever you achieve good evaluation results, save them as a baseline:

results = evaluate(
    dataset=dataset,
    task=qatask,

    scoringmetrics=[hallucinationmetric, relevancemetric],
    experimentname=f"baseline-v1-{datetime.now().strftime('%Y%m%d')}"

)

• LlamaIndex: from opik.integrations.llamaindex import LlamaIndexCallbackHandler
• Anthropic: from opik.integrations.anthropic import trackanthropic
• Google Gemini: from opik.integrations.gemini import trackgemini
• Ollama: Through OpenAI-compatible integration
• LiteLLM: from opik.integrations.litellm import tracklitellm
• Haystack: from opik.integrations.haystack import OpikConnector
• CrewAI: from opik.integrations.crewai import trackcrewai

Example integration with LlamaIndex:

from opik.integrations.llamaindex import LlamaIndexCallbackHandler
from llamaindex.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llamaindex.core.callbacks import CallbackManager


opikhandler = LlamaIndexCallbackHandler()
Settings.callbackmanager = CallbackManager([opikhandler])

documents = SimpleDirectoryReader("./data").loaddata()

index = VectorStoreIndex.fromdocuments(documents)

queryengine = index.asqueryengine()

response = queryengine.query("What is a vector database?")
print(response)

Conclusion

Opik provides a complete toolkit for managing the lifecycle of LLM applications from development to production. With tracing, you gain full visibility into your LLM pipeline. The evaluation features enable consistent and reproducible automated testing. And monitoring ensures your application continues to perform as expected in production.

The main advantages of Opik compared to similar tools are its open-source nature, easy setup (can be self-hosted or cloud-based), and extensive integrations with various popular AI frameworks. For teams seriously building production-grade LLM applications, Opik is a worthy addition to your AI observability stack.

Recommended next steps:

• Explore the Opik dashboard for trace and evaluation visualization
• Implement tracing in your existing LLM pipelines
• Create evaluation datasets from real production data
• Set up monitoring rules for early detection of quality issues