Helicone: How I Monitor and Control Every LLM Call in Production

Folks, once you start getting serious about building LLM-powered applications, whether it's a chatbot, an agent, or an AI feature inside a product, sooner or later you will hit the same wall: you have no idea what is actually happening behind each call to the model. How many tokens did that cost? What is my total bill this month? Why is this one request so painfully slow? Which user is hammering the API the most? Which prompt version is quietly blowing up my costs? All of these questions are hard to answer if you just fire off openai.chat.completions.create() and forget about it.

In this tutorial I want to introduce you to Helicone. Helicone is an open-source observability platform built specifically for LLMs. Its core job is simple but crucial: it logs, monitors, and gives you full control over every LLM call your application makes. What I love about it is that using it is not complicated at all. You just change the baseurl to the Helicone proxy and add one authentication header, and suddenly all your requests show up neatly in a dashboard. No need to rebuild your architecture, no heavy SDK to install, no changes to your business logic.

Throughout this tutorial I will walk you through everything from scratch: from account setup and the most basic integration, to request logging, custom properties, per-user tracking, caching to save money, rate limiting to keep users honest, all the way to sessions and tracing for agents with many steps. Every example is written in Python using the OpenAI SDK, because that is the most common combination out there. Let's get started.

Introduction

Before we dive into the code, I want you to understand why observability matters and how Helicone solves it.

While your LLM app is still a prototype, everything looks fine. You type a prompt, you get an answer, you are happy. But once it hits production with hundreds or thousands of users, the situation changes completely. API costs can spiral out of control because of an overly long prompt. Requests fail silently and users complain. Latency spikes during peak hours and you have no idea why. Without observability, you are flying blind.

Helicone solves this with two integration approaches:

• Proxy (Gateway): You route your LLM requests through the Helicone proxy. This is the fastest way, you just change the baseurl. Because requests go through Helicone, it can add features like caching and rate limiting at the network level.
• Async Logging: If you would rather not route requests through a proxy (for latency or security policy reasons), you can send logs asynchronously to Helicone after the request completes. The LLM call itself goes directly to OpenAI, and the log is sent separately.

For most cases, the proxy approach is the most practical, and that is what I will focus on here. The reason is that with the proxy you get all the features at once without writing much extra code.

What makes Helicone attractive compared to just storing your own logs in a database:

• Cost and latency dashboards that come ready to use. You can see total spend, breakdown per model, per user, per feature.
• Custom properties to tag every request so you can filter and analyze it later.
• Built-in caching that can dramatically cut costs for repeated requests.
• Rate limiting per user or per property, without having to build your own system.
• Sessions and tracing for agents with many steps, so you can see the full execution flow from start to finish.

And because it is open-source, you can self-host it if you really want full control over your data. But to get started, the free cloud version is more than enough.

Installation

The good news is that to use Helicone with the proxy approach, you actually don't need to install the Helicone package at all. You just need the OpenAI SDK you are already using. But for completeness, let me explain everything you need.

Step 1: Create an Account and Get an API Key

First, sign up at helicone.ai with your email or a GitHub account. Once you're in, open the Settings menu then the API Keys section. Generate a new key, which usually starts with sk-helicone-. Keep this key safe and never leak it into a public repo.

Step 2: Install Dependencies

The only requirement is the OpenAI SDK:

pip install openai

If you want to use the dedicated Helicone Python SDK (for manual async logging or certain advanced features), install this too:

pip install helicone-helpers

But for most of this tutorial, openai alone is enough because we are using the proxy.

Step 3: Store Credentials in Environment Variables

Never hardcode API keys in your code. I always store them in environment variables or a .env file. Create a .env file:

OPENAIAPIKEY=sk-proj-xxxxxxxxxxxxxxxx
HELICONEAPIKEY=sk-helicone-xxxxxxxxxxxxxxxx

Then to read this .env file in Python, install python-dotenv:

pip install python-dotenv

Now you're ready. Nothing complicated, right? Let's move on to the basic usage.

Basic Usage

In this section I'll show you the most fundamental way to use Helicone: routing the OpenAI SDK through the Helicone proxy so every request is logged automatically.

The Most Basic Proxy Integration

The key is just two things: change the baseurl to the Helicone proxy endpoint, and add the Helicone-Auth header. Look at the following code:

import os
from dotenv import loaddotenv
from openai import OpenAI

loaddotenv()


client = OpenAI(
    apikey=os.environ["OPENAIAPIKEY"],
    baseurl="https://oai.helicone.ai/v1",

    defaultheaders={
        "Helicone-Auth": f"Bearer {os.environ['HELICONEAPIKEY']}",
    },
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a friendly and concise assistant."},
        {"role": "user", "content": "Explain what observability is in one sentence."},
    ],
)

print(response.choices[0].message.content)

Notice that the only difference from regular OpenAI code is just two lines: baseurl and defaultheaders. Everything else is exactly what you normally write. Once this code runs, open the Helicone dashboard and you'll see the request appear complete with the prompt, response, token count, cost, and latency. This is what I mean by "super easy". You don't change any logic, you just tell the OpenAI SDK to go through Helicone first.

Understanding What Gets Logged

Every request that goes through the proxy automatically stores this information in the dashboard:

• Request and response body in full, so you can see the actual prompt and answer.
• Model used.
• Token usage: prompt tokens, completion tokens, total.
• Cost: Helicone automatically calculates the cost based on model pricing.
• Latency: how long the request took.
• Status: success or error, plus the error code if it failed.

You get all of this for free without writing a single line of manual logging code.

Alternative: Async Logging Without a Proxy

If you have a reason not to route through the proxy, you can use async logging. Here the LLM call still goes directly to OpenAI, and the log is sent separately to Helicone. Here is a simple example using a manual HTTP request to the Helicone logging endpoint:

import os
import time
import requests
from dotenv import loaddotenv

from openai import OpenAI

loaddotenv()

LLM call directly to OpenAI, no proxy
client = OpenAI(apikey=os.environ["OPENAIAPIKEY"])


start = time.time()
messages = [{"role": "user", "content": "What is the capital of Indonesia?"}]
response = client.chat.completions.create(model="gpt-4o-mini", messages=messages)

Send the log to Helicone separately
logpayload = {
    "providerRequest": {
        "url": "https://api.openai.com/v1/chat/completions",
        "json": {"model": "gpt-4o-mini", "messages": messages},
        "meta": {},
    },
    "providerResponse": {
        "json": response.modeldump(),

        "status": 200,
        "headers": {},
    },
    "timing": {"startTime": {"seconds": int(start)}, "endTime": {"seconds": int(time.time())}},
}

requests.post(
    "https://api.helicone.ai/oai/v1/log",
    headers={"Authorization": f"Bearer {os.environ['HELICONEAPIKEY']}"},

    json=logpayload,
    timeout=10,
)

print(response.choices[0].message.content)

Honestly, this async approach is more cumbersome and I usually only use it when there is a specific security or latency requirement. For almost every case, the proxy is far simpler. So from now on I will keep using the proxy approach.

Simplifying with a Helper Function

To avoid writing the client configuration over and over, I like to make one helper function that builds a Helicone client:

import os
from openai import OpenAI

def getheliconeclient(extraheaders=None):

    headers = {
        "Helicone-Auth": f"Bearer {os.environ['HELICONEAPIKEY']}",

    }
    if extraheaders:
        headers.update(extraheaders)

    return OpenAI(
        apikey=os.environ["OPENAIAPIKEY"],
        baseurl="https://oai.helicone.ai/v1",

        defaultheaders=headers,
    )

client = getheliconeclient()

With this pattern, I can easily add extra headers like custom properties or a user id, which we'll cover in a moment.

Advanced Usage

Alright folks, so far you can already do basic logging. Now we get into the part that makes Helicone truly powerful. All the features below are enabled via headers, so the pattern is consistent and easy to remember.

Custom Properties for Tagging and Segmentation

Custom properties are labels you attach to each request. Their purpose is to filter and analyze data in the dashboard. For example, you want to know which feature is eating up the most cost, or which environment (production vs staging) generates the most requests. You just send a header with the Helicone-Property- prefix.

import os
from openai import OpenAI

client = OpenAI(
    apikey=os.environ["OPENAIAPIKEY"],

    baseurl="https://oai.helicone.ai/v1",
    defaultheaders={

        "Helicone-Auth": f"Bearer {os.environ['HELICONEAPIKEY']}",

    },
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Summarize this article into 3 points."}],
    extraheaders={
        "Helicone-Property-Feature": "summarizer",
        "Helicone-Property-Environment": "production",
        "Helicone-Property-Version": "v2",
    },
)

print(response.choices[0].message.content)

Notice I use extraheaders at the call level, not on the client. This matters, because properties are usually different per request. Helicone-Property-Feature becomes a "Feature" column in the dashboard that you can filter. The name after Helicone-Property- is yours to define. I usually stick with a few standard properties like Feature, Environment, and Version across all requests so the analysis stays tidy.

User Tracking

If your application has many users, you definitely want to know who is most active and how much each user costs. Helicone has a dedicated header for this, Helicone-User-Id.

def chatforuser(client, userid, prompt):
    return client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        extraheaders={

            "Helicone-User-Id": userid,
        },
    )

Example usage
resp = chatforuser(client, "user12345", "Help me write an Instagram caption.")

print(resp.choices[0].message.content)

With Helicone-User-Id, the dashboard gets a dedicated page per user. You can see total requests, total cost, and usage patterns for each user. This is incredibly useful if you sell a usage-based product, because you can compute the margin per user accurately. I recommend using a stable, anonymous id, such as the user id from your database, rather than the email directly, for privacy reasons.

Caching to Save Money

This is one of my favorite features. Many applications send the same request over and over. For example, if 100 users ask the exact same FAQ question, why would you pay OpenAI 100 times? Helicone can cache the response at the proxy level. Enable it with the Helicone-Cache-Enabled header.

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is machine learning?"}],
    extraheaders={
        "Helicone-Cache-Enabled": "true",
        "Helicone-Cache-Bucket-Max-Size": "3",
        "Cache-Control": "max-age=3600",
    },
)

print(response.choices[0].message.content)

Explanation of the headers above:

• Helicone-Cache-Enabled: true enables caching for this request.
• Helicone-Cache-Bucket-Max-Size: 3 sets how many different responses are stored per bucket. Useful if you want a bit of answer variety.
• Cache-Control: max-age=3600 sets how long the cache is valid, in seconds. Here 3600 means one hour.

The first request runs normally to OpenAI. The second request with an identical prompt within the cache period is served directly from cache, much faster and free. In the dashboard, you can see how many "cache hits" you got and how much money you saved. For applications with many repeated queries, the savings can be significant. But be careful, don't use caching for requests that must always be fresh, such as those that depend on time or real-time data.

Rate Limiting

Sometimes a user, whether on purpose or not, sends too many requests and makes your costs explode. Helicone can rate limit at the proxy level without you building your own system. You configure it via the Helicone-RateLimit-Policy header.

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hi!"}],
    extraheaders={

        "Helicone-User-Id": "user12345",
        # Maximum 100 requests per 3600 seconds (1 hour), counted per user
        "Helicone-RateLimit-Policy": "100;w=3600;s=user",
    },
)

The policy format is [quota];w=[window seconds];s=[segment]. In this example it means a maximum of 100 requests per hour per user. The s=user segment makes the limit counted per Helicone-User-Id. You can also segment by a custom property. If the limit is exceeded, Helicone returns a response with status 429 (Too Many Requests), and you just handle that in your code.

from openai import RateLimitError

try:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hi again!"}],
        extraheaders={

            "Helicone-User-Id": "user12345",
            "Helicone-RateLimit-Policy": "100;w=3600;s=user",
        },
    )
    print(response.choices[0].message.content)
except RateLimitError:
    print("You've reached your usage limit. Please try again later.")

Cost and Latency Dashboard

All the features above flow into the Helicone dashboard, and this is where you really feel the benefit. Without writing any code, you get visualizations of:

• Total cost per day, week, or month, with a breakdown per model.
• Latency distribution: median, p95, p99, so you know how bad the slowest user experience is.
• Request volume over time, to see traffic patterns.
• Filter by custom property or user, so you can answer questions like "how much did the summarizer feature in production cost this month?"

What I love is that all these metrics can be filtered in combination. For example "show me the p95 latency for the chat feature in the production environment for premium users". Getting that level of detail without building your own analytics pipeline is a huge saving in engineering time.

Sessions and Tracing for Agents

Now this is an important part if you're building agents. An agent usually makes many sequential LLM calls: think, call a tool, think again, answer. If each call is logged separately, it's hard to see the full flow. Helicone has a concept of sessions that groups all related requests into a single trace.

You use three headers:

• Helicone-Session-Id: a unique id for one conversation session or one run of the agent.
• Helicone-Session-Name: a name for the session so it's easy to recognize.
• Helicone-Session-Path: shows the request's position in the trace hierarchy, like a folder path.

import uuid

sessionid = str(uuid.uuid4())


def agentstep(client, sessionid, path, prompt):

    return client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        extraheaders={
            "Helicone-Session-Id": sessionid,

            "Helicone-Session-Name": "Research Agent",
            "Helicone-Session-Path": path,
        },
    )

Step 1: the agent plans
plan = agentstep(client, sessionid, "/plan",

                  "Create a plan to research AI trends in 2026.")

Step 2: the agent executes one sub-task
research = agentstep(client, sessionid, "/plan/research",

                      "Find the 3 most important AI trends in 2026.")

Step 3: the agent composes a conclusion
summary = agentstep(client, sessionid, "/plan/summary",

                     "Summarize the research findings into a short paragraph.")

print(summary.choices[0].message.content)

Because all three calls use the same Helicone-Session-Id, the dashboard displays them as a single trace tree. The Helicone-Session-Path values shaped like /plan, /plan/research, /plan/summary let Helicone draw the hierarchy of which is a child of which. This makes debugging agents far more pleasant. If an agent produces a weird output, you can trace step by step exactly where the logic went off, complete with the prompt and response of each step.

Best Practices

After using Helicone for a while across several projects, there are a few things I've learned that I want to share so you don't fall into the same holes I did.

Be Consistent with Your Custom Properties Schema

Define one set of standard custom properties from the start and use them consistently across the whole application. I usually use at least Feature, Environment, and Version. If every developer invents their own property names, your dashboard becomes a mess and hard to filter. Build one centralized helper function that automatically adds these mandatory properties to every request so nothing gets forgotten.

Don't Cache Carelessly

Caching is powerful for saving money, but don't just turn it on for every request. Requests that depend on time, specific user data, or need creative variety should not be cached. I usually only cache deterministic, repeated requests, such as FAQs, classification, or extraction where the input is often identical. Always set Cache-Control with a reasonable duration based on how quickly your data changes.

Always Set a User Id in Production

Always send Helicone-User-Id in production environments. Without it, you lose the ability to analyze cost and usage per user, and that's very valuable data for business decisions. Use a stable, anonymous id from your database, not an email or personal data, for user privacy.

Use Sessions for All Multi-Step Flows

Whenever you have a flow that involves more than one related LLM call, wrap it with the same session id. This isn't just for agents, but also for multi-turn chatbot conversations or pipelines with several stages. A clean trace will save your debugging time many times over in the future.

Handle Errors Gracefully

Because requests go through a proxy, there's a small chance the proxy has a problem or a rate limit is hit. Always wrap LLM calls in a try-except that handles RateLimitError and network errors. For truly critical applications, I sometimes prepare a fallback that goes directly to OpenAI without the proxy if the Helicone proxy is down, so the service keeps running even if observability is temporarily off.

Secure Your API Keys

This is basic but often forgotten. Never hardcode HELICONEAPIKEY or OPENAIAPIKEY in code that goes into a repo. Always use environment variables. If a key leaks, rotate it immediately from the Helicone and OpenAI dashboards.

Consider Self-Hosting for Sensitive Data

If you work with highly sensitive data or are under strict regulations, remember that Helicone is open-source and can be self-hosted. That way all logs stay in your own infrastructure. For most startups, the cloud version is safe and practical, but the self-host option is there if you need it.

Conclusion

Here we are, folks, we've gone quite a long way. We started with understanding why LLM observability matters, then set up Helicone which turned out to only require changing the baseurl and adding one authentication header. From there we learned basic logging, attaching custom properties for segmentation, per-user tracking, caching to save money, rate limiting to stay safe from abuse, reading the cost and latency dashboard, all the way to sessions and tracing for debugging agents with many steps.

What I hope you take home from this tutorial is the awareness that LLM applications in production need eyes and ears. You can't manage what you don't measure. Helicone gives you all those measuring tools with minimal integration effort. For me personally, the ratio between how easy the setup is and how much insight I get is one of the best among similar tools.

My advice, start small first. Enable the proxy and basic logging on one feature, look at the data in the dashboard, then gradually add custom properties, user ids, and caching as your needs grow. You don't need to use all the features at once. What matters is that you start having visibility into what your application is doing behind the scenes.

Good luck, and may your AI applications become more cost-efficient, faster, and more controlled. Once you've played with Helicone, try exploring the other features in its dashboard that I didn't cover here, because there's still plenty to dig into. See you in the next tutorial, folks.