LangFuse: Open-Source Platform for LLM Application Observability

As more companies adopt Large Language Models (LLMs) in production applications, the need for observability becomes critical. How do you know the cost of each request? How do you track the performance of different prompts? How do you systematically evaluate the quality of LLM outputs?

LangFuse is an open-source solution that answers all these questions. It provides a comprehensive observability platform for tracing, monitoring, evaluating, and managing prompts in your LLM applications.

What Is LangFuse?

LangFuse is an open-source LLM engineering platform that helps development teams with:

• Tracing: Track every step of LLM application execution, from input to output
• Monitoring: Monitor latency, cost, and error rates in real-time
• Evaluation: Score LLM outputs both manually and automatically
• Prompt Management: Centralized prompt versioning and deployment
• Analytics: Dashboards for analyzing usage and performance

LangFuse supports various LLM providers (OpenAI, Anthropic, Azure, etc.) and integrates with popular frameworks like LangChain, LlamaIndex, and Haystack.

Deployment Options

1. LangFuse Cloud (Managed)

The easiest way to get started is using LangFuse Cloud at cloud.langfuse.com.

# No installation needed - just sign up and get API keys
LANGFUSEPUBLICKEY=pk-lf-...
LANGFUSESECRETKEY=sk-lf-...
LANGFUSEHOST=https://cloud.langfuse.com

2. Self-Hosted with Docker

For full control over your data, you can deploy LangFuse yourself:

# docker-compose.yml
version: '3.8'

services:
  langfuse:
    image: langfuse/langfuse:2
    ports:
"3000:3000"
    environment:

DATABASEURL=postgresql://postgres:postgres@db:5432/langfuse
NEXTAUTHSECRET=mysecret

SALT=mysalt
NEXTAUTHURL=http://localhost:3000
    dependson:

db

  db:
    image: postgres:15
    environment:
POSTGRESUSER=postgres
POSTGRESPASSWORD=postgres

POSTGRESDB=langfuse
    volumes:

pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

# Start LangFuse
docker-compose up -d

Access at http://localhost:3000
Create the first admin account through the UI

Python SDK Installation

pip install langfuse

Environment Configuration

import os

os.environ["LANGFUSEPUBLICKEY"] = "pk-lf-..."
os.environ["LANGFUSESECRETKEY"] = "sk-lf-..."
os.environ["LANGFUSEHOST"] = "https://cloud.langfuse.com"  # or self-hosted URL

Client Initialization

from langfuse import Langfuse

langfuse = Langfuse(
    publickey="pk-lf-...",
    secretkey="sk-lf-...",

    host="https://cloud.langfuse.com"
)

Verify connection
langfuse.authcheck()
print("LangFuse connected!")

Tracing: Tracking LLM Application Execution

Tracing is LangFuse's core feature. Every interaction with your LLM application is recorded as a trace consisting of several components:

Core Tracing Concepts

• Trace: The top-level unit representing one end-to-end execution
• Span: A sub-unit representing a single step in the trace (e.g., retrieval, preprocessing)
• Generation: A special span for LLM calls, automatically capturing token usage and cost

Creating a Simple Trace

from langfuse import Langfuse

langfuse = Langfuse()

Create a new trace
trace = langfuse.trace(
    name="customer-support-chat",
    userid="user-123",

    metadata={"session": "abc-456"},
    tags=["production", "customer-support"]
)

Add a span for preprocessing step
span = trace.span(
    name="preprocess-input",
    input={"rawmessage": "How do I reset my password?"}
)