Complete Guide to TruLens: LLM Application Evaluation and Observability

Author: Ruby Abdullah

Building Large Language Model (LLM) applications is one thing. Knowing whether they actually work well is another challenge entirely. As LLM-powered applications move from prototypes to production, the need for systematic evaluation and observability becomes critical. This is where TruLens comes in: an open-source framework built by TruEra that gives you the tools to evaluate, track, and improve your LLM applications with quantitative feedback functions and an intuitive dashboard.

In this tutorial, you will learn how to install and configure TruLens, instrument a Retrieval-Augmented Generation (RAG) pipeline, define and run feedback functions such as groundedness and relevance, visualize results on the TruLens dashboard, build custom metrics, and compare different model configurations side by side. Every code example in this article is designed to be runnable so you can follow along in your own environment.

Why LLM Evaluation Matters

Traditional software testing relies on deterministic inputs and outputs. You write a unit test, assert that add(2, 3) == 5, and move on. LLM applications break this pattern fundamentally. The same prompt can yield different outputs across runs, and judging whether an output is "correct" often requires nuanced, context-dependent reasoning.

Without proper evaluation, you face several risks. First, hallucinations can go undetected. A RAG application might generate confident-sounding answers that have no basis in the retrieved documents. Second, relevance drift can occur silently. Your retrieval pipeline might start pulling in tangentially related documents, degrading answer quality over time. Third, regression becomes invisible. A change to your prompt template, embedding model, or chunk size might hurt performance in subtle ways that only show up after careful measurement.

TruLens addresses these challenges by providing feedback functions that automatically score your LLM application's outputs along dimensions like groundedness, relevance, and sentiment. It records every interaction so you can trace exactly what happened at each step, and it gives you a dashboard to visualize trends and compare experiments.

Installation and Setup

TruLens has been restructured into modular packages. The core package is trulens-core, and you install additional provider packages depending on which LLM you want to use for evaluation. Let us start with a complete installation that covers the most common use cases.

# Install TruLens core and the OpenAI provider for feedback functions
pip install trulens-core trulens-providers-openai trulens-dashboard

If you plan to instrument LangChain applications
pip install trulens-apps-langchain

If you plan to instrument LlamaIndex applications
pip install trulens-apps-llamaindex

For using local/open-source models via LiteLLM
pip install trulens-providers-litellm

After installation, set up your API key. TruLens uses an LLM (typically OpenAI) to power its feedback functions, so you need a valid API key.

import os
os.environ["OPENAIAPIKEY"] = "sk-your-openai-key-here"

Now initialize the TruLens session. This creates a local SQLite database to store all your evaluation records.

from trulenscore import TruSession

session = TruSession()

Optional: reset the database if you want a fresh start
session.resetdatabase()

Basic Usage: Evaluating a RAG Application

Let us build a simple RAG application and then instrument it with TruLens. We will use LangChain for the RAG pipeline, but the same concepts apply to LlamaIndex or any custom framework.

Step 1: Build a Simple RAG Pipeline

from langchainopenai import ChatOpenAI, OpenAIEmbeddings
from langchain.textsplitter import RecursiveCharacterTextSplitter
from langchaincommunity.vectorstores import FAISS

from langchain.chains import RetrievalQA
from langchain.schema import Document

Create some sample documents about machine learning
documents = [
    Document(pagecontent="""
    Random forests are an ensemble learning method that operates by constructing
    multiple decision trees during training. The output of a random forest is the
    class selected by most trees for classification, or the mean prediction of
    individual trees for regression. Random forests correct for the tendency of
    individual decision trees to overfit to their training data.
    """),
    Document(pagecontent="""

    Gradient boosting is a machine learning technique used for regression and
    classification problems. It produces a prediction model in the form of an
    ensemble of weak prediction models, typically decision trees. Gradient boosting
    builds trees one at a time, where each new tree helps to correct errors made
    by the previously trained tree. This sequential approach often leads to better
    performance than random forests on structured data.
    """),
    Document(pagecontent="""
    Neural networks are computing systems inspired by biological neural networks
    in the brain. They consist of layers of interconnected nodes or neurons that
    process information using connectionist approaches. Deep learning refers to
    neural networks with multiple hidden layers. Neural networks excel at tasks
    like image recognition, natural language processing, and speech recognition.
    """),
    Document(pagecontent="""

    Support Vector Machines (SVM) are supervised learning models used for
    classification and regression analysis. An SVM model represents examples as
    points in space, mapped so that the examples of separate categories are divided
    by a clear gap that is as wide as possible. SVMs are effective in
    high-dimensional spaces and are memory efficient because they use a subset of
    training points called support vectors.
    """),
]

Split and embed documents
textsplitter = RecursiveCharacterTextSplitter(chunksize=500, chunkoverlap=50)
splits = textsplitter.splitdocuments(documents)

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.fromdocuments(splits, embeddings)


Create the RAG chain
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
ragchain = RetrievalQA.fromchaintype(
    llm=llm,
    chaintype="stuff",

    retriever=vectorstore.asretriever(searchkwargs={"k": 2}),

    returnsourcedocuments=True,

)

Step 2: Define Feedback Functions

Feedback functions are the heart of TruLens. They evaluate your application's behavior along specific dimensions. The most important set of feedback functions for RAG applications is known as the RAG Triad: Answer Relevance, Context Relevance, and Groundedness.

from trulenscore import Feedback
from trulensprovidersopenai import OpenAI as fOpenAI
import numpy as np

Initialize the OpenAI-based feedback provider
provider = fOpenAI()

1. Answer Relevance: Is the answer relevant to the question?
fanswerrelevance = (
    Feedback(provider.relevancewithcotreasons, name="Answer Relevance")

    .oninput()
    .onoutput()

)

2. Context Relevance: Are the retrieved documents relevant to the question?
fcontextrelevance = (

    Feedback(provider.contextrelevancewithcotreasons, name="Context Relevance")

    .oninput()
    .on(TruChain.selectcontext())

    .aggregate(np.mean)
)

3. Groundedness: Is the answer grounded in (supported by) the retrieved context?
from trulenscore import Select

fgroundedness = (

    Feedback(provider.groundednessmeasurewithcotreasons, name="Groundedness")

    .on(TruChain.selectcontext().collect())
    .onoutput()

)

4. Sentiment: What is the sentiment of the response? (bonus metric)
fsentiment = Feedback(provider.sentiment, name="Sentiment").onoutput()

Let us break down what each of these does. Answer Relevance measures whether the final answer actually addresses what the user asked. If someone asks about random forests and gets an answer about neural networks, this score will be low. Context Relevance checks whether the documents pulled from the vector store are relevant to the query. Poor retrieval is one of the most common failure modes in RAG applications. Groundedness verifies that the generated answer is actually supported by the retrieved context, rather than being a hallucination fabricated by the LLM. Sentiment provides a measure of the emotional tone of the response, which can be useful for customer-facing applications.

Step 3: Instrument and Run the Application

Now we wrap our RAG chain with TruLens instrumentation. Every call to the wrapped application will be recorded and evaluated.

from trulensappslangchain import TruChain

Wrap the chain with TruLens
trurag = TruChain(
    ragchain,

    appname="RAGTutorial",

    appversion="v1",
    feedbacks=[fanswerrelevance, fcontextrelevance, fgroundedness, fsentiment],
)

Run some test queries
testquestions = [

    "What is a random forest and how does it work?",
    "How does gradient boosting differ from random forests?",
    "What are neural networks used for?",
    "Explain support vector machines.",
    "What is the capital of France?",  # Out-of-domain question to test behavior
]

with trurag as recording:
    for question in testquestions:

        response = ragchain.invoke({"query": question})
        print(f"Q: {question}")
        print(f"A: {response['result']}\n")

Notice the last question about the capital of France. This is deliberately out of domain. A well-behaved RAG application should either refuse to answer or indicate that it does not have relevant information. The TruLens feedback functions will help you identify this kind of failure automatically.

Step 4: Launch the Dashboard

After running your evaluations, launch the TruLens dashboard to visualize the results.

from trulensdashboard import rundashboard

rundashboard(session)

This starts a Streamlit application, typically on http://localhost:8501. The dashboard shows you a leaderboard of all your app versions, individual record details with feedback scores, and a trace view that lets you inspect each step of your pipeline.

You can also retrieve results programmatically without the dashboard.

# Get a summary of all records
recordsdf = session.getrecordsandfeedback()[0]

print(recordsdf[["input", "output", "Answer Relevance", "Context Relevance", "Groundedness"]])

Advanced Usage

Custom Feedback Functions

While the built-in feedback functions cover the most common evaluation needs, you will often want to create custom metrics specific to your application. TruLens makes this straightforward.

from trulenscore import Feedback

Custom feedback function: Check if the response includes a disclaimer
when the model is uncertain
def uncertaintydisclosure(response: str) -> float:
    """
    Scores whether the response appropriately discloses uncertainty.
    Returns 1.0 if uncertainty language is present when needed, 0.0 otherwise.
    """
    uncertaintyphrases = [

        "i'm not sure",
        "i don't have enough information",
        "based on the available context",
        "i cannot determine",
        "the provided documents don't",
        "i don't know",
    ]
    responselower = response.lower()
    hasdisclaimer = any(phrase in responselower for phrase in uncertaintyphrases)

    return 1.0 if hasdisclaimer else 0.5

funcertainty = Feedback(uncertaintydisclosure, name="Uncertainty Disclosure").onoutput()

You can also create LLM-based custom feedback functions that leverage the full reasoning capability of a language model.

from trulensprovidersopenai import OpenAI as fOpenAI

provider = fOpenAI()

Custom LLM-based feedback: Technical accuracy check
def technicalaccuracy(question: str, response: str) -> float:
    """
    Uses an LLM to judge whether the response is technically accurate
    for machine learning topics.
    """
    prompt = f"""You are an expert machine learning engineer. Evaluate whether
    the following response to the given question is technically accurate.

    Question: {question}
    Response: {response}

    Score the technical accuracy from 0.0 to 1.0 where:
1.0 means completely accurate with no errors
0.5 means partially accurate with some minor errors
0.0 means mostly inaccurate or contains major errors

    Respond with only a single float number."""

    result = provider.endpoint.client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
    )
    try:
        return float(result.choices[0].message.content.strip())
    except ValueError:
        return 0.5

ftechnical = (

    Feedback(technicalaccuracy, name="Technical Accuracy")
    .oninput()

    .onoutput()
)

Comparing Models and Configurations

One of the most powerful features of TruLens is the ability to compare different versions of your application. Let us say you want to test how different chunk sizes or different LLMs affect your RAG pipeline's quality.

from langchainopenai import ChatOpenAI

Configuration 1: GPT-4o-mini with small chunks
llmmini = ChatOpenAI(model="gpt-4o-mini", temperature=0)
ragchainmini = RetrievalQA.fromchaintype(
    llm=llmmini,

    chaintype="stuff",
    retriever=vectorstore.asretriever(searchkwargs={"k": 2}),
    returnsourcedocuments=True,
)

trumini = TruChain(

    ragchainmini,

    appname="RAGTutorial",

    appversion="gpt4o-mini-k2",
    feedbacks=[fanswerrelevance, fcontextrelevance, fgroundedness],

)

Configuration 2: GPT-4o with more retrieved documents
llmfull = ChatOpenAI(model="gpt-4o", temperature=0)
ragchainfull = RetrievalQA.fromchaintype(
    llm=llmfull,

    chaintype="stuff",
    retriever=vectorstore.asretriever(searchkwargs={"k": 4}),
    returnsourcedocuments=True,
)

trufull = TruChain(

    ragchainfull,

    appname="RAGTutorial",

    appversion="gpt4o-k4",
    feedbacks=[fanswerrelevance, fcontextrelevance, fgroundedness],

)

Run the same test suite against both configurations
for truapp, chain in [(trumini, ragchainmini), (trufull, ragchainfull)]:
    with truapp as recording:

        for question in testquestions:
            chain.invoke({"query": question})

Compare results on the dashboard
rundashboard(session)

The TruLens dashboard leaderboard will now show both versions side by side, making it easy to see which configuration performs better on each metric.

Integration with LlamaIndex

If you are using LlamaIndex instead of LangChain, TruLens integrates just as smoothly.

from llamaindex.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llamaindex.llms.openai import OpenAI

from llamaindex.embeddings.openai import OpenAIEmbedding
from trulensappsllamaindex import TruLlama

Configure LlamaIndex
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
Settings.embedmodel = OpenAIEmbedding()


Load documents and create index
documents = SimpleDirectoryReader("./data").loaddata()
index = VectorStoreIndex.fromdocuments(documents)

queryengine = index.asqueryengine(similaritytopk=3)

Define feedback functions using LlamaIndex selectors
from trulenscore import Feedback

from trulensprovidersopenai import OpenAI as fOpenAI


provider = fOpenAI()

fgroundednessllama = (

    Feedback(provider.groundednessmeasurewithcotreasons, name="Groundedness")

    .on(TruLlama.selectcontext().collect())
    .onoutput()

)

frelevancellama = (

    Feedback(provider.relevancewithcotreasons, name="Answer Relevance")
    .oninput()

    .onoutput()
)

fcontextllama = (
    Feedback(provider.contextrelevancewithcotreasons, name="Context Relevance")
    .oninput()

    .on(TruLlama.selectcontext())
    .aggregate(np.mean)
)

Instrument the query engine
truqueryengine = TruLlama(
    queryengine,

    appname="LlamaIndexRAG",

    appversion="v1",
    feedbacks=[fgroundednessllama, frelevancellama, fcontextllama],
)

Run queries
with truqueryengine as recording:
    response = queryengine.query("What is machine learning?")

    print(response)

Instrumenting Custom Applications

You do not need LangChain or LlamaIndex to use TruLens. You can instrument any Python function or class using the TruCustomApp wrapper.

from trulenscore import TruCustomApp, instrument
import openai

class SimpleQA:
    """A minimal question-answering application."""

    def init(self):
        self.client = openai.OpenAI()

    @instrument
    def retrieve(self, query: str) -> str:
        """Simulate retrieval by returning a fixed context."""
        # In a real app, this would query a vector database
        context = (
            "Machine learning is a subset of artificial intelligence that "
            "enables systems to learn and improve from experience without "
            "being explicitly programmed. It focuses on developing algorithms "
            "that can access data and use it to learn for themselves."
        )
        return context

    @instrument
    def generate(self, query: str, context: str) -> str:
        """Generate an answer based on the query and context."""
        response = self.client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[
                {"role": "system", "content": f"Answer based on this context: {context}"},
                {"role": "user", "content": query},
            ],
            temperature=0,
        )
        return response.choices[0].message.content

    @instrument
    def respond(self, query: str) -> str:
        """Main entry point: retrieve context and generate response."""
        context = self.retrieve(query)
        answer = self.generate(query, context)
        return answer

Create and instrument the app
qaapp = SimpleQA()


truqa = TruCustomApp(
    qaapp,

    appname="CustomQA",

    appversion="v1",
    feedbacks=[fanswerrelevance, fsentiment],

    mainmethod=qaapp.respond,

)

with truqa as recording:
    result = qaapp.respond("What is machine learning?")

    print(result)

The @instrument decorator tells TruLens to trace calls to that method, so you get full visibility into each step of your pipeline in the dashboard.

Deferred Feedback Evaluation

By default, TruLens evaluates feedback functions synchronously, which can slow down your application. For production use, you can switch to deferred evaluation mode where feedback functions run in the background.

from trulenscore import FeedbackMode

trudeferred = TruChain(

    ragchain,
    appname="RAGTutorial",
    appversion="v1deferred",
    feedbacks=[fanswerrelevance, fcontextrelevance, fgroundedness],

    feedbackmode=FeedbackMode.DEFERRED,
)

Queries return immediately without waiting for feedback evaluation
with trudeferred as recording:

    for question in testquestions:
        ragchain.invoke({"query": question})


Start the feedback evaluator in the background
session.startevaluator()

When you are done, stop the evaluator
session.stopevaluator()

This approach is particularly useful when you want to evaluate every production query without adding latency to your user-facing application.

Guardrails with TruLens

TruLens feedback functions can also serve as guardrails to filter or block low-quality responses before they reach the user.

from trulenscore import TruCustomApp, Select

Use context relevance as a guardrail on retrieved context
from trulenscore.guardrails.base import contextfilter

class GuardedRAG(SimpleQA):

    @instrument
    @contextfilter(

        feedback=fcontextrelevance,

        threshold=0.5,
        keywordforprompt="query",

    )
    def retrieve(self, query: str) -> list:
        """Retrieve documents, filtering out irrelevant ones."""
        # In production, this would return multiple documents
        docs = [
            "Random forests use multiple decision trees for prediction.",
            "The weather in Paris is typically mild in spring.",
            "Ensemble methods combine multiple models for better accuracy.",
        ]
        return docs

With this guardrail in place, any retrieved document scoring below 0.5 on context relevance will be filtered out before being passed to the generation step, reducing the chance of irrelevant information contaminating the response.

Best Practices

After working with TruLens across multiple projects, several best practices have emerged that will help you get the most out of the framework.

Start with the RAG Triad. If you are building a RAG application, always begin with the three core feedback functions: Answer Relevance, Context Relevance, and Groundedness. These three metrics together give you a comprehensive view of your pipeline's health. Low context relevance points to retrieval problems. Low groundedness suggests hallucination issues. Low answer relevance indicates that something in the overall pipeline is failing to connect the user's question to a useful answer. Use chain-of-thought reasoning in feedback functions. The withcotreasons variants of feedback functions (like relevancewithcotreasons) not only return a score but also provide an explanation of why that score was assigned. This is invaluable for debugging. When you see a low score, the reasoning tells you exactly what went wrong. Version your experiments systematically. Use meaningful appversion strings that encode the key parameters of each configuration. For example, gpt4o-mini-k3-chunk500 tells you the model, retrieval count, and chunk size at a glance. This makes the dashboard leaderboard much more informative. Test with adversarial and edge-case queries. Include questions that are out of domain, ambiguous, or designed to trigger hallucinations in your test suite. These edge cases are where evaluation provides the most value, because they reveal failure modes that are hard to catch through manual testing alone. Run evaluations on a representative sample. In production, you may not want to evaluate every single query due to cost constraints. Use TruLens with a sampling strategy: evaluate a random percentage of queries, or focus evaluation on queries where the model's confidence is low. Monitor feedback score distributions, not just averages. An average groundedness score of 0.85 might look good, but if the distribution is bimodal with scores clustering at 0.99 and 0.3, you have a serious problem that the average hides. The TruLens dashboard histograms help you catch this. Use deferred evaluation in production. Synchronous feedback evaluation adds latency and cost to every request. Switch to deferred mode for production deployments so that evaluation happens in the background without affecting user experience. Integrate into your CI/CD pipeline. Create a test suite of representative queries with expected minimum feedback scores. Run TruLens evaluation as part of your continuous integration pipeline, and fail the build if scores drop below acceptable thresholds.

# Example CI/CD quality gate
recordsdf = session.getrecordsandfeedback()[0]

avggroundedness = recordsdf["Groundedness"].mean()
avgrelevance = recordsdf["Answer Relevance"].mean()
avgcontext = recordsdf["Context Relevance"].mean()

assert avggroundedness >= 0.7, f"Groundedness too low: {avggroundedness:.2f}"
assert avgrelevance >= 0.7, f"Answer Relevance too low: {avgrelevance:.2f}"
assert avgcontext >= 0.6, f"Context Relevance too low: {avgcontext:.2f}"

print("All quality gates passed.")

Keep your evaluation LLM consistent. If you use GPT-4o for feedback functions in development, do not switch to a different model in production without recalibrating your thresholds. Different models may score the same output differently, which can lead to confusing comparisons.

Understanding the TruLens Architecture

TruLens follows a modular architecture that is worth understanding. At the core, there is a session that manages the database connection and stores all records. The Feedback class defines evaluation functions that can be composed and targeted at specific parts of your application's input/output chain using selectors. Provider classes like OpenAI and LiteLLM implement the actual evaluation logic using different LLM backends. App wrappers like TruChain, TruLlama, and TruCustomApp handle the instrumentation of different framework types, intercepting calls and recording inputs, outputs, and intermediate steps.

The data flows as follows: your application receives a query and produces a response. The TruLens wrapper intercepts this interaction, records the full trace (including intermediate steps like retrieval), and then runs each feedback function against the appropriate parts of the trace. The results are stored in the database and can be viewed through the dashboard or queried programmatically.

Conclusion

TruLens fills a critical gap in the LLM application development workflow. While building an LLM application is increasingly straightforward thanks to frameworks like LangChain and LlamaIndex, knowing whether your application actually works well, and continuing to work well over time, requires systematic evaluation. TruLens provides exactly this with its feedback functions, tracing capabilities, and visualization dashboard.

The RAG Triad of Answer Relevance, Context Relevance, and Groundedness gives you a solid foundation for evaluating retrieval-augmented generation applications. Custom feedback functions let you extend this to cover domain-specific requirements. The ability to compare different configurations side by side makes it practical to iterate on your pipeline with confidence.

Whether you are building a prototype or running a production LLM application, integrating TruLens into your workflow will give you the visibility and confidence you need to ship reliable AI-powered features. Start with the basic feedback functions, instrument your pipeline, and let the data guide your optimization decisions.

For more information, visit the official TruLens documentation at https://www.trulens.org/ and explore the GitHub repository at https://github.com/truera/trulens for the latest updates and examples.