DSPy: Stop Hand-Tuning Prompts, Let the Compiler Optimize Them

Hey everyone, today I want to introduce a library whose whole mental model is different from the other LLM tooling we usually talk about. It is called DSPy. If your current workflow is writing a long prompt, tweaking it a bit, testing, not liking the result, tweaking again, adding "you are an expert", adding one more example, and repeating until you are exhausted, DSPy is here to tell you there is a more systematic way.

The core idea is this: DSPy treats prompts not as text you write by hand, but as something that can be compiled and optimized automatically from data. You only declare what the task is (what goes in, what comes out), not how to talk to the model. The DSPy optimizer then searches for the best instructions and few-shot examples based on a metric you define.

In this tutorial we start from zero: installation, model configuration, the Signature and Module concepts, building a small RAG pipeline, and then the most interesting part, the optimizers that can raise your program's accuracy without you retyping a single word of prompt. Grab a coffee, this is a bit mind-bending at first but addictive afterwards.

Introduction

Before we write code, I want you to understand why DSPy exists.

The Problem With Manual Prompt Engineering

Manual prompt engineering has a few diseases that people rarely talk about.

First, it is brittle. A prompt that shines on GPT-4o may fall apart on Claude or on an open source model like Llama. The moment you switch models, much of your hand tuning has to be redone.

Second, it is unmeasured. When you add "think step by step", do you know how many percentage points of accuracy that bought you? Usually not. It is a gut feeling. We rarely keep a clean baseline and a metric.

Third, it does not scale to bigger pipelines. If your app has five chained LLM steps (rewrite the query, retrieve, rerank, answer, verify), hand tuning five prompts at once is a nightmare. Changing step one can quietly break step four.

How DSPy Thinks

DSPy borrows its worldview from PyTorch. In PyTorch you do not write gradients by hand, you define the architecture and the optimizer takes care of the weights. In DSPy you define the structure of your LLM program and the optimizer takes care of the "weights", which here are the prompt instructions and the few-shot demonstrations.

There are three core concepts to master.

A Signature declares the inputs and outputs of one LLM step, for example question -> answer or context, question -> answer. You are not writing the prompt, you are writing the contract.

A Module is the strategy used to execute a Signature, for example dspy.Predict (answer directly), dspy.ChainOfThought (reason first, then answer), or dspy.ReAct (reason while calling tools).

An Optimizer (formerly called a Teleprompter) is an algorithm that takes your program, a small dataset, and a metric, then automatically searches for the instructions and examples that maximize that metric.

So the flow is: declare a Signature, wrap it in a Module, compose a program, prepare data plus a metric, then compile.

Installation

DSPy needs Python 3.9 or newer. I always recommend a virtual environment so dependencies do not collide with your other projects.

python -m venv venv
source venv/bin/activate   # Linux / Mac
venv\Scripts\activate    # Windows

pip install -U dspy

If you want to follow the RAG section below, also install a lightweight embedding library:

pip install sentence-transformers numpy

Modern DSPy uses LiteLLM under the hood, so nearly every provider works through the same string format: provider/model-name. That is very convenient because switching providers is a one line change.

Configuring the Model

The first step of every DSPy program is telling DSPy which model to use.

import os
import dspy

Example with OpenAI
os.environ["OPENAIAPIKEY"] = "sk-..."
lm = dspy.LM("openai/gpt-4o-mini", maxtokens=1000, temperature=0.0)


dspy.configure(lm=lm)

lm = dspy.LM("ollamachat/llama3.1", apibase="http://localhost:11434", apikey="")
dspy.configure(lm=lm)

Notice that I set temperature=0.0. When you are measuring program quality and running an optimizer, you want results to be as consistent as possible so that comparisons between prompt candidates are fair.

Signature: Declaration, Not Prompt

The fastest way to build a Signature is with a short string.

import dspy

qa = dspy.Predict("question -> answer")
result = qa(question="What is the capital of Indonesia and why is it being moved?")
print(result.answer)

One line, and DSPy assembles the actual prompt behind the scenes, complete with a parseable output format. You can inspect exactly what was sent with:

dspy.inspecthistory(n=1)

import dspy

class TicketClassification(dspy.Signature):
    """Classify a customer support ticket into the correct category."""

    ticket: str = dspy.InputField(desc="the customer message body")
    category: str = dspy.OutputField(desc="one of: billing, technical, account, other")
    urgency: str = dspy.OutputField(desc="low, medium, or high")
    reason: str = dspy.OutputField(desc="one short sentence of justification")

classifier = dspy.Predict(TicketClassification)
out = classifier(ticket="I paid three days ago but my account is still locked and my client deadline is tomorrow!")

print(out.category)   # billing
print(out.urgency)    # high
print(out.reason)

from typing import Literal
import dspy

class Sentiment(dspy.Signature):
    """Analyze the sentiment of a product review."""

    review: str = dspy.InputField()
    label: Literal["positive", "negative", "neutral"] = dspy.OutputField()
    score: float = dspy.OutputField(desc="confidence between 0.0 and 1.0")

analyzer = dspy.Predict(Sentiment)
r = analyzer(review="The product is fine, but shipping took a whole week.")
print(r.label, r.score)

cot = dspy.ChainOfThought("mathproblem -> answer")
result = cot(mathproblem="A store gives 20 percent off, then another 10 percent off the discounted price. What is the effective total discount?")


print(result.reasoning)   # step by step reasoning
print(result.answer)      # 28 percent

import ast
import operator

OPS = {ast.Add: operator.add, ast.Sub: operator.sub,
       ast.Mult: operator.mul, ast.Div: operator.truediv}

def evaluate(node):
    if isinstance(node, ast.Constant):
        return node.value
    if isinstance(node, ast.BinOp):
        return OPStype(node.op), evaluate(node.right))
    raise ValueError("unsupported expression")

def calculator(expression: str) -> float:
    """Evaluate a simple math expression, for example '15 * 1.11'."""
    return evaluate(ast.parse(expression, mode="eval").body)


def lookupprice(productname: str) -> str:

    """Look up a product price in the internal catalog."""
    catalog = {"laptop": 12000000, "mouse": 250000, "monitor": 2100000}
    return str(catalog.get(productname.lower(), "not found"))

agent = dspy.ReAct("question -> answer", tools=[calculator, lookupprice])

print(agent(question="If I buy 3 monitors, what is the total after 11 percent tax?").answer)

import dspy

class SummarizeAndCritique(dspy.Module):
    def init(self):
        super().init()
        self.summarize = dspy.ChainOfThought("article -> summary")
        self.critique = dspy.ChainOfThought("article, summary -> critique, finalsummary")

    def forward(self, article):
        s = self.summarize(article=article)
        c = self.critique(article=article, summary=s.summary)
        return dspy.Prediction(
            draftsummary=s.summary,

            critique=c.critique,
            summary=c.finalsummary,
        )

program = SummarizeAndCritique()
text = "The government announced a new policy on electric vehicle subsidies ..."
result = program(article=text)
print(result.summary)

Both steps above carry their own prompt, and the optimizer will later tune both at once while looking at the final metric. That is the thing you simply cannot do cleanly by hand.

Case Study: A Small RAG Pipeline

Let us build a mini RAG so this becomes concrete. I deliberately use a very simple homemade retriever so the focus stays on DSPy rather than on a vector database.

import numpy as np
import dspy
from sentencetransformers import SentenceTransformer


DOCS = [
    "Permanent employees get 12 working days of annual leave per year, forfeited if unused within 18 months.",
    "Reimbursement must be filed within 30 days of the transaction date, otherwise the system rejects it automatically.",
    "Employees may work from home up to 8 days per month with direct manager approval.",
    "Salaries are paid on the 25th; if the 25th is a holiday, payment moves to the previous working day.",
    "The annual bonus is 60 percent individual performance and 40 percent company performance.",
]

encoder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
EMB = encoder.encode(DOCS, normalizeembeddings=True)

def retrieve(query: str, k: int = 2):
    q = encoder.encode([query], normalizeembeddings=True)[0]

    scores = EMB @ q
    idx = np.argsort(-scores)[:k]
    return [DOCS[i] for i in idx]


class RAG(dspy.Module):
    def init(self, k=2):
        super().init()
        self.k = k
        self.answer = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        context = retrieve(question, self.k)
        out = self.answer(context="\n".join(context), question=question)
        return dspy.Prediction(context=context, answer=out.answer)


rag = RAG()
print(rag(question="What if I submit a reimbursement two months late?").answer)

import dspy

trainset = [
    dspy.Example(
        question="How many days of annual leave do I get?",
        answer="12 working days per year, forfeited if unused within 18 months.",
    ).withinputs("question"),
    dspy.Example(
        question="I bought something on the 1st, today is the 20th, can I still claim it?",
        answer="Yes, the limit is 30 days after the transaction date.",
    ).withinputs("question"),

    dspy.Example(
        question="Payday is the 25th but that is a Sunday, when do I get paid?",
        answer="It moves to the previous working day.",
    ).withinputs("question"),
    dspy.Example(
        question="How is the annual bonus calculated?",
        answer="60 percent individual performance and 40 percent company performance.",
    ).withinputs("question"),

    dspy.Example(
        question="What is the monthly work from home limit?",
        answer="8 days per month with direct manager approval.",
    ).withinputs("question"),
]

def answermetric(example, pred, trace=None):
    """Check whether the prediction covers the core of the reference answer."""
    gold = example.answer.lower()
    predicted = pred.answer.lower()
    keywords = [w for w in gold.split() if len(w) > 4]
    if not keywords:
        return 0.0
    hits = sum(1 for w in keywords if w in predicted)
    return hits / len(keywords) >= 0.5

For production you can build something smarter, even using an LLM as a judge:

class JudgeAnswer(dspy.Signature):
    """Judge whether the predicted answer is factually equivalent to the reference."""
    question: str = dspy.InputField()
    reference: str = dspy.InputField()
    prediction: str = dspy.InputField()
    equivalent: bool = dspy.OutputField()

judge = dspy.Predict(JudgeAnswer)

def llmmetric(example, pred, trace=None):

    out = judge(question=example.question, reference=example.answer, prediction=pred.answer)
    return bool(out.equivalent)

from dspy.teleprompt import BootstrapFewShot

optimizer = BootstrapFewShot(metric=answermetric, maxbootstrappeddemos=4, maxlabeleddemos=4)
ragoptimized = optimizer.compile(RAG(), trainset=trainset)


print(ragoptimized(question="What if I submit a reimbursement two months late?").answer)

What just happened? DSPy ran your program over the training data, collected the execution traces that satisfied your metric, and picked the best of those traces as few-shot demonstrations that get injected into the prompt automatically. You never typed a single example into the prompt, yet the prompt is now much richer.

Stronger Optimizers

from dspy.teleprompt import MIPROv2

optimizer = MIPROv2(metric=answermetric, auto="light")

ragmipro = optimizer.compile(RAG(), trainset=trainset, requirespermissiontorun=False)

from dspy.evaluate import Evaluate

devset = [
    dspy.Example(question="When does my leave expire?",
                 answer="If it is not used within 18 months.").withinputs("question"),
    dspy.Example(question="Can I work from home if my manager disagrees?",
                 answer="No, working from home requires direct manager approval.").withinputs("question"),

]

evaluator = Evaluate(devset=devset, metric=answermetric, numthreads=4, displayprogress=True)

print("Before optimization:", evaluator(RAG()))
print("After optimization:", evaluator(ragoptimized))

ragoptimized.save("raghr.json")

fresh = RAG()
fresh.load("raghr.json")
print(fresh(question="How long do I have to file a reimbursement?").answer)

That JSON holds the optimized instructions and demonstrations, not model weights. It is small, it fits in git, and your team can review it in a pull request.

Tips and Best Practices

Start simple. Do not begin with a seven step program. Write one dspy.Predict, measure it, and only add complexity when the metric demands it.

Invest most of your time in the metric, not the prompt. In DSPy the metric is your steering wheel. A sloppy metric makes the optimizer chase the wrong thing very diligently.

Separate trainset, devset, and testset. Optimize on the trainset, pick configurations on the devset, and report the final number on a testset you never touched.

Use a cheap model for execution and a smarter one for optimization. DSPy lets you set different models per module, which saves a meaningful amount of money.

Always check dspy.inspect_history() when something looks off. Nine out of ten problems become obvious the moment you look at the real prompt.

Keep caching on while experimenting so identical calls are not billed twice. DSPy does this by default through LiteLLM, and it is a lifesaver when you run optimizers repeatedly.

Never use raw eval() for a calculator tool like many internet examples do. Use a restricted AST parser as shown above, because your tools will be called with strings the model made up.

Conclusion

DSPy turns LLM application development from the art of arranging sentences into an engineering discipline you can measure. The key takeaways:

You declare a Signature (inputs and outputs) instead of writing a prompt, which makes your program portable across models.

Modules define the thinking strategy, from Predict that answers directly, to ChainOfThought that reasons, to ReAct that calls tools.

Multi-step programs are composed like nn.Module in PyTorch, and every step gets optimized together.

Optimizers such as BootstrapFewShot and MIPROv2 automatically search for the best demonstrations and instructions against your metric, something no human can do consistently by hand.

Your metric and your data are the real investment. The prompt becomes a compiled artifact rather than the product of an all-nighter of guessing.

Try it on one small use case at work, maybe ticket classification or field extraction from documents. Collect 30 labeled examples, write a simple metric, and compile. The moment you watch accuracy climb without touching a prompt, you will understand why I was so excited to write this tutorial. Happy building.