Outlines: Structured LLM Generation with Constrained Decoding

One of the biggest challenges when working with Large Language Models (LLMs) is consistently getting structured and valid output. When you need valid JSON, an enum from specific choices, or other specific formats, LLMs often produce output that does not match the required format. Outlines solves this problem with a constrained decoding approach at the token level, guaranteeing that output is always structurally valid.

How Constrained Decoding Works

Before diving into Outlines, it is important to understand the difference between two approaches to structured generation:

Retry-Based Approach (like Instructor)

Libraries like Instructor use a "generate-then-validate" approach:

The LLM generates free-form output

The output is parsed and validated

If validation fails, the prompt is modified and the LLM is called again

Repeat until output is valid or the retry limit is reached

The problem: API costs increase, latency is unpredictable, and there is no guarantee of convergence.

Constrained Decoding Approach (Outlines)

Outlines uses a fundamentally different approach:

Before generation, Outlines builds a finite-state machine (FSM) or regular expression automaton from the desired schema

At each step of token generation, Outlines calculates which tokens are valid based on the current state

The probabilities of invalid tokens are set to zero (masked)

The LLM can only choose from valid tokens

The result: output is guaranteed to be 100% valid without retries, with minimal computational overhead.

Installation

Installing Outlines is straightforward:

pip install outlines

To use with local transformers models:

pip install outlines[transformers]

For integration with llama.cpp:

pip install outlines[llamacpp]

For integration with vLLM:

pip install outlines[vllm]

JSON Generation with Pydantic Models

The most popular feature of Outlines is the ability to generate valid JSON based on Pydantic models.

Basic Example

import outlines
from pydantic import BaseModel, Field
from typing import List, Optional
from enum import Enum

Define schema with Pydantic
class Address(BaseModel):
    street: str = Field(description="Street name and number")
    city: str = Field(description="City name")
    state: str = Field(description="State or province")
    zipcode: str = Field(description="Postal code")


class Employee(BaseModel):
    name: str = Field(description="Full name of the employee")
    age: int = Field(ge=18, le=65, description="Employee age")
    email: str = Field(description="Email address")
    department: str = Field(description="Work department")
    salary: float = Field(gt=0, description="Monthly salary")
    address: Address
    skills: List[str] = Field(description="List of skills")

Load model
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

Create generator with JSON schema
generator = outlines.generate.json(model, Employee)

Generate structured data
prompt = """Create a fictional employee record for a technology company
in San Francisco. This employee is a senior data engineer."""

result = generator(prompt)
print(type(result))  # 
print(result.name)
print(result.department)
print(result.modeldumpjson(indent=2))

Nested and Complex Schemas

from pydantic import BaseModel, Field
from typing import List, Optional
from enum import Enum
import outlines

class Priority(str, Enum):
    HIGH = "high"
    MEDIUM = "medium"
    LOW = "low"

class Status(str, Enum):
    TODO = "todo"
    INPROGRESS = "inprogress"

    REVIEW = "review"
    DONE = "done"

class SubTask(BaseModel):
    title: str
    completed: bool

class Task(BaseModel):
    id: int
    title: str = Field(maxlength=100)
    description: str
    priority: Priority
    status: Status
    assignee: str
    estimatedhours: float = Field(gt=0)

    subtasks: List[SubTask]
    tags: List[str]

class SprintPlan(BaseModel):
    sprintname: str

    durationdays: int = Field(ge=7, le=30)
    goal: str
    tasks: List[Task]

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
generator = outlines.generate.json(model, SprintPlan)

prompt = """Create a sprint plan for a development team working on a
product recommendation feature using machine learning. The sprint is
2 weeks long with 3 main tasks."""

sprint = generator(prompt)
print(sprint.modeldumpjson(indent=2))

Regex-Guided Generation

Outlines allows you to control output format using regular expressions:

import outlines

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

Generate US phone number
phonepattern = r"\+1[0-9]{10}"

generator = outlines.generate.regex(model, phonepattern)
result = generator("Provide a US phone number: ")
print(result)  # example: +12025551234

Generate ISO date format
datepattern = r"[0-9]{4}-[0-1][0-9]-[0-3][0-9]"

generator = outlines.generate.regex(model, datepattern)
result = generator("Today's date in ISO format: ")
print(result)  # example: 2026-04-30

Generate email address
emailpattern = r"[a-z0-9.%+-]+@[a-z0-9.-]+\.[a-z]{2,}"
generator = outlines.generate.regex(model, emailpattern)

result = generator("Customer service contact email: ")
print(result)

Generate custom format: product code
productcodepattern = r"[A-Z]{3}-[0-9]{4}-[A-Z]{2}"

generator = outlines.generate.regex(model, productcodepattern)

result = generator("Product code for gaming laptop: ")
print(result)  # example: LAP-2026-GM

Generate IPv4 address
ippattern = r"((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)"
generator = outlines.generate.regex(model, ippattern)

result = generator("Database server IP address: ")
print(result)

Choice and Enum Generation

When you want to restrict output to specific choices:

import outlines

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

Sentiment analysis with limited choices
sentiments = ["positive", "negative", "neutral"]
generator = outlines.generate.choice(model, sentiments)

reviews = [
    "This product is amazing! The quality is outstanding.",
    "Terrible service, shipping was extremely slow.",
    "The product is okay, you get what you pay for.",
]

for review in reviews:
    prompt = f"Analyze the sentiment of this review: '{review}'\nSentiment: "
    sentiment = generator(prompt)
    print(f"Review: {review[:50]}... -> {sentiment}")

Support ticket classification
categories = [
    "billing",
    "technicalissue",
    "featurerequest",

    "accountmanagement",
    "generalinquiry"

]
generator = outlines.generate.choice(model, categories)

ticket = "I cannot log into my account after resetting my password"
prompt = f"Classify the following support ticket: '{ticket}'\nCategory: "
category = generator(prompt)
print(f"Category: {category}")

Language detection
languages = ["english", "spanish", "french", "german", "italian"]
generator = outlines.generate.choice(model, languages)

text = "Bonjour, comment allez-vous aujourd'hui?"
prompt = f"Detect the language of the following text: '{text}'\nLanguage: "
language = generator(prompt)
print(f"Detected language: {language}")

Type-Guided Generation

Outlines also supports generation based on Python data types:

import outlines

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")

Generate integer
intgenerator = outlines.generate.format(model, int)
result = intgenerator("How many planets are in our solar system? ")

print(f"Result (int): {result}, type: {type(result)}")

Generate float
floatgenerator = outlines.generate.format(model, float)
result = floatgenerator("What is the value of pi to 5 decimal places? ")

print(f"Result (float): {result}, type: {type(result)}")

Generate boolean
boolgenerator = outlines.generate.format(model, bool)
result = boolgenerator("Is Python a compiled language? ")

print(f"Result (bool): {result}, type: {type(result)}")

Integration with Various Models

Using Transformers (Hugging Face)

import outlines

Model from Hugging Face
model = outlines.models.transformers(
    "microsoft/Phi-3-mini-4k-instruct",
    device="cuda"  # or "cpu"
)

With sampling configuration
generator = outlines.generate.json(model, MySchema)
result = generator(
    prompt,
    maxtokens=1024,
    temperature=0.7,
    topp=0.9

)

Using llama.cpp

import outlines

Local GGUF model via llama.cpp
model = outlines.models.llamacpp(
    "path/to/model.gguf",
    nctx=4096,
    ngpulayers=35
)

generator = outlines.generate.json(model, MySchema)
result = generator(prompt)

Using vLLM

import outlines

Model with vLLM for fast inference
model = outlines.models.vllm(
    "microsoft/Phi-3-mini-4k-instruct",
    tensorparallelsize=1
)

generator = outlines.generate.json(model, MySchema)

Batch generation - very efficient with vLLM
prompts = [
    "Extract information from: Alice, 30 years old, New York",
    "Extract information from: Bob, 25 years old, London",
    "Extract information from: Charlie, 35 years old, Tokyo",
]
results = generator(prompts)
for r in results:
    print(r.modeldumpjson())

Practical Example: Data Extraction Pipeline with Guaranteed Valid Output

Here is a complete example of building a data extraction pipeline that guarantees output is always valid:

import outlines
from pydantic import BaseModel, Field, validator
from typing import List, Optional
from enum import Enum
import json

Schema for invoice information extraction
class Currency(str, Enum):
    USD = "USD"
    EUR = "EUR"
    GBP = "GBP"
    JPY = "JPY"

class InvoiceItem(BaseModel):
    itemname: str = Field(description="Name of the product or service")

    quantity: int = Field(ge=1, description="Number of items")
    unitprice: float = Field(gt=0, description="Price per unit")
    total: float = Field(gt=0, description="Total price for this item")

class CompanyInfo(BaseModel):
    name: str = Field(description="Company name")
    address: str = Field(description="Company address")
    taxid: Optional[str] = Field(None, description="Tax identification number if available")


class Invoice(BaseModel):
    invoicenumber: str = Field(description="Invoice number")
    date: str = Field(description="Invoice date in YYYY-MM-DD format")
    issuer: CompanyInfo
    recipient: CompanyInfo
    items: List[InvoiceItem]
    subtotal: float = Field(gt=0)
    taxrate: float = Field(ge=0, le=100)

    taxamount: float = Field(ge=0)
    grandtotal: float = Field(gt=0)

    currency: Currency
    notes: Optional[str] = None

Initialize model and generator
model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
invoicegenerator = outlines.generate.json(model, Invoice)

Invoice text to extract
invoicetext = """

INVOICE #INV-2026-0042

From:
TechForward Solutions Inc.
123 Innovation Drive, San Francisco, CA 94105
Tax ID: 12-3456789

To:
DataFlow Analytics LLC
456 Market Street, Suite 200, New York, NY 10001

Date: April 15, 2026

Items:
IT Consulting Services - 40 hours @ $150.00 = $6,000.00
Enterprise Software License - 5 units @ $500.00 = $2,500.00
Employee Training Sessions - 2 sessions @ $1,200.00 = $2,400.00

Subtotal: $10,900.00
Sales Tax (8.5%): $926.50
Total: $11,826.50

Note: Payment due within 30 days of invoice date.
"""

Extract data - guaranteed to produce valid output
prompt = f"""Extract all information from the following invoice into a structured format:

{invoicetext}

Invoice data:"""

invoicedata = invoicegenerator(prompt)

Output is guaranteed to be valid and typed as Invoice
print(f"Invoice Number: {invoicedata.invoicenumber}")
print(f"Issuer: {invoicedata.issuer.name}")

print(f"Recipient: {invoicedata.recipient.name}")
print(f"Number of items: {len(invoicedata.items)}")

print(f"Grand Total: {invoicedata.currency.value} {invoicedata.grandtotal:,.2f}")
print(f"\nFull JSON:")
print(invoicedata.modeldumpjson(indent=2))



Batch processing pipeline
class ExtractionResult(BaseModel):
    name: str
    title: Optional[str] = None
    company: Optional[str] = None
    email: Optional[str] = None
    phone: Optional[str] = None
    location: Optional[str] = None

persongenerator = outlines.generate.json(model, ExtractionResult)

Business cards to extract
businesscards = [

    """Dr. Emily Watson, Ph.D.
    Chief Technology Officer
    NeuralNet Dynamics Inc.
    emily.watson@neuralnet.com
    +1 (415) 555-0123
    San Francisco, CA""",

    """James Chen, MBA
    Regional Sales Director
    GlobalTech Solutions Pte Ltd
    james.chen@globaltech.sg
    +65 9123 4567
    Singapore""",

    """Maria Garcia
    Senior Data Scientist
    FinData Analytics GmbH
    m.garcia@findata.de
    +49 30 1234567
    Berlin, Germany""",
]

print("\n=== Batch Extraction Results ===")
for i, card in enumerate(businesscards):
    prompt = f"Extract contact information from this business card:\n{card}\n\nData:"
    result = persongenerator(prompt)

    print(f"\n--- Business Card {i+1} ---")
    print(f"Name: {result.name}")
    print(f"Title: {result.title}")
    print(f"Company: {result.company}")
    print(f"Email: {result.email}")
    print(f"Phone: {result.phone}")
    print(f"Location: {result.location}")

Document Classification Pipeline

import outlines
from pydantic import BaseModel, Field
from typing import List, Optional
from enum import Enum

class DocumentCategory(str, Enum):
    CONTRACT = "contract"
    INVOICE = "invoice"
    PROPOSAL = "proposal"
    REPORT = "report"
    LETTER = "letter"
    MEMO = "memo"

class UrgencyLevel(str, Enum):
    URGENT = "urgent"
    NORMAL = "normal"
    LOW = "low"

class DocumentClassification(BaseModel):
    category: DocumentCategory
    urgency: UrgencyLevel
    summary: str = Field(maxlength=200)
    keywords: List[str] = Field(maxlength=5)

    targetdepartment: str
    actionrequired: bool

    suggestedaction: Optional[str] = None

model = outlines.models.transformers("microsoft/Phi-3-mini-4k-instruct")
classifier = outlines.generate.json(model, DocumentClassification)

document = """
INTERNAL MEMO
Subject: Emergency Budget Request for Server Repair

To all management personnel,

We are writing to inform you that the main data center server experienced
critical hardware failure on April 28, 2026. Immediate replacement of
3 enterprise SSDs and 2 ECC RAM modules is required.

Estimated cost: $12,500
Deadline: Maximum 3 business days

We kindly request immediate approval given the impact on company operations.

Regards,
IT Infrastructure Team
"""

prompt = f"Classify the following document:\n\n{document}\n\nClassification:"
result = classifier(prompt)
print(result.modeldumpjson(indent=2))

Tips and Best Practices

1. Design Good Schemas

# Use Field for documentation and validation
class CustomerData(BaseModel):
    name: str = Field(
        description="Full name of the customer",
        minlength=2,

        maxlength=100
    )
    age: int = Field(
        ge=0, le=150,
        description="Age in years"
    )
    email: str = Field(
        description="Valid email address"
    )

2. Limit Output Size

# Use maxtokens for efficiency
generator = outlines.generate.json(model, MySchema)
result = generator(prompt, maxtokens=512)

3. Choose the Right Model

For structured generation, smaller models are often sufficient because Outlines helps "guide" the output. Models with 3B-7B parameters typically provide good results for simple extraction tasks.

4. Use Enums for Limited Choices

# Better to use Enum than free-form string
class Sentiment(str, Enum):
    POSITIVE = "positive"
    NEGATIVE = "negative"
    NEUTRAL = "neutral"

Instead of
class AnalysisResult(BaseModel):
    sentiment: str  # can produce any value

5. Consider Performance

# Cache the generator for repeated use
model = outlines.models.transformers("model-name")
generator = outlines.generate.json(model, MySchema)

The generator builds the FSM during initialization
Subsequent calls are faster because the FSM is cached
for prompt in prompts:
    result = generator(prompt)

Conclusion

Outlines transforms how we interact with LLMs to produce structured output. With its constrained decoding approach, you get:

• 100% guarantee that output is always valid according to the schema
• No retries, reducing cost and latency
• Flexible integration with various models and output formats
• Optimal performance with minimal computational overhead

For projects that require structured output from LLMs, such as data extraction pipelines, document classification, or structured data generation, Outlines is a highly recommended choice over traditional retry-based approaches.