Distilabel Tutorial: Synthetic Data Generation Framework for LLM Fine-Tuning

In the world of Large Language Model (LLM) development, training data quality is the single most important factor determining model success. Distilabel is an open-source framework from Argilla that enables you to build pipelines for generating high-quality synthetic data, evaluating models, and preparing datasets for fine-tuning. This tutorial covers how to use Distilabel from installation to building complex production pipelines.

What Is Distilabel?

Distilabel is a Python framework developed by the Argilla team for creating synthetic data generation and LLM evaluation pipelines. It is designed to address the primary challenge in AI development: obtaining large amounts of high-quality training data without the expensive costs of manual annotation.

Key features of Distilabel include:

• DAG-based pipelines: Directed Acyclic Graph architecture enabling parallel and efficient data processing
• Multi-provider LLM support: Integration with OpenAI, Anthropic, Google, Hugging Face, vLLM, Ollama, and more
• Advanced generation techniques: Built-in implementations for Self-Instruct, Evol-Instruct, UltraFeedback, Magpie, and other techniques
• Scalability: Ability to run pipelines on Ray clusters for large-scale data processing
• Argilla and Hugging Face Hub integration: Direct data export to Argilla for human validation or to Hugging Face Hub

Installation

Prerequisites

Ensure you have Python 3.9 or newer installed on your system.

Basic Installation

pip install distilabel

Installation with Specific Providers

Distilabel uses an extras system to install LLM provider dependencies:

# For OpenAI
pip install "distilabel[openai]"

For Anthropic
pip install "distilabel[anthropic]"

For vLLM (local)
pip install "distilabel[vllm]"

For Hugging Face Inference Endpoints
pip install "distilabel[hf-inference-endpoints]"

For Ollama
pip install "distilabel[ollama]"

Full installation with all providers
pip install "distilabel[openai,anthropic,vllm,ollama]"

Installation with Ray for Scalability

pip install "distilabel[ray,openai]"

Verify Installation

import distilabel
print(distilabel.version)

Configure API Keys

Before using cloud-based LLM providers, configure your API keys:

export OPENAIAPIKEY="sk-your-api-key"
export ANTHROPICAPIKEY="your-anthropic-key"
export HFTOKEN="hfyour-token"

Basic Usage

Core Concepts

Distilabel is built upon three core concepts:

Running a Simple LLM Call

from distilabel.llms import OpenAILLM

llm = OpenAILLM(model="gpt-4o-mini")
llm.load()

result = llm.generateoutputs(

    inputs=[
        [
            {"role": "user", "content": "Explain what machine learning is in 2 sentences."}
        ]
    ]
)

print(result[0][0])

from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import TextGeneration
from distilabel.steps import LoadDataFromDicts

data = [
    {"instruction": "Explain the concept of neural networks for beginners."},
    {"instruction": "What is the difference between supervised and unsupervised learning?"},
    {"instruction": "How does gradient descent work?"},
]

with Pipeline(name="basic-tutorial") as pipeline:
    loaddata = LoadDataFromDicts(data=data)

    textgen = TextGeneration(

        llm=OpenAILLM(model="gpt-4o-mini"),
        numgenerations=1,
    )

    loaddata >> textgen

distiset = pipeline.run()

df = distiset["default"]["train"].topandas()

print(df[["instruction", "generation"]].tostring())

Using Multiple LLMs

One of Distilabel's strengths is the ability to use multiple LLMs within a single pipeline:

from distilabel.llms import OpenAILLM, AnthropicLLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import TextGeneration
from distilabel.steps import LoadDataFromDicts, CombineOutputs

data = [
    {"instruction": "What is transfer learning?"},
    {"instruction": "Explain the attention mechanism concept."},
]

with Pipeline(name="multi-llm") as pipeline:
    loaddata = LoadDataFromDicts(data=data)


    genopenai = TextGeneration(
        name="genopenai",

        llm=OpenAILLM(model="gpt-4o-mini"),
        numgenerations=1,
        outputmappings={"generation": "generationopenai"},
    )

    genanthropic = TextGeneration(

        name="genanthropic",
        llm=AnthropicLLM(model="claude-sonnet-4-6"),
        numgenerations=1,

        outputmappings={"generation": "generationanthropic"},

    )

    combine = CombineOutputs()

    loaddata >> [genopenai, genanthropic] >> combine

distiset = pipeline.run()

Advanced Usage

Self-Instruct: Instruction Dataset Generation

Self-Instruct is a technique that allows LLMs to generate new training instructions based on seed instructions. This is extremely useful for expanding fine-tuning datasets:

from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import SelfInstruct
from distilabel.steps import LoadDataFromDicts

seeddata = [

    {
        "input": "You are an AI assistant specialized in data science and machine learning. "
                 "Your task is to help users understand technical concepts in an accessible way."
    }
]

with Pipeline(name="self-instruct-pipeline") as pipeline:
    loaddata = LoadDataFromDicts(data=seeddata)


    selfinstruct = SelfInstruct(
        llm=OpenAILLM(model="gpt-4o-mini"),
        numinstructions=5,

        criteriaforquerygeneration=(
            "Instructions should relate to data science, machine learning, or AI. "
            "Instructions should vary in difficulty level. "
            "Instructions should be practical and answerable concretely."
        ),
    )

    loaddata >> selfinstruct

distiset = pipeline.run()

df = distiset["default"]["train"].topandas()

for idx, row in df.iterrows():
    print(f"\nGenerated instructions:")
    for instruction in row["instructions"]:
        print(f"  - {instruction}")

from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import EvolInstruct
from distilabel.steps import LoadDataFromDicts

data = [
    {"instruction": "Calculate the average of a list."},
    {"instruction": "Create a sorting function."},
    {"instruction": "Explain what an API is."},
]

with Pipeline(name="evol-instruct-pipeline") as pipeline:
    loaddata = LoadDataFromDicts(data=data)

    evolinstruct = EvolInstruct(

        llm=OpenAILLM(model="gpt-4o-mini"),
        numevolutions=3,
        storeevolutions=True,

        includeoriginalinstruction=True,

    )

    loaddata >> evolinstruct


distiset = pipeline.run()

df = distiset["default"]["train"].topandas()
for idx, row in df.iterrows():
    print(f"\nOriginal: {row['instruction']}")
    print(f"Evolved: {row['evolvedinstruction']}")

from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import TextGeneration, UltraFeedback
from distilabel.steps import LoadDataFromDicts, CombineOutputs, FormatTextGenerationDPO

data = [
    {"instruction": "Explain the difference between a list and tuple in Python."},
    {"instruction": "How do you optimize a slow SQL query?"},
]

with Pipeline(name="ultrafeedback-pipeline") as pipeline:
    loaddata = LoadDataFromDicts(data=data)

    gen1 = TextGeneration(

        name="genmodela",

        llm=OpenAILLM(model="gpt-4o-mini"),
        outputmappings={"generation": "generationa"},

    )

    gen2 = TextGeneration(
        name="genmodelb",
        llm=OpenAILLM(model="gpt-4o"),
        outputmappings={"generation": "generationb"},
    )

    combine = CombineOutputs()

    feedback = UltraFeedback(
        llm=OpenAILLM(model="gpt-4o"),
        aspect="overall-quality",
    )

    formatdpo = FormatTextGenerationDPO()


    loaddata >> [gen1, gen2] >> combine >> feedback >> formatdpo


distiset = pipeline.run()

from distilabel.llms import OpenAILLM
from distilabel.pipeline import Pipeline
from distilabel.steps.tasks import Magpie

with Pipeline(name="magpie-pipeline") as pipeline:
    magpie = Magpie(
        llm=OpenAILLM(model="gpt-4o-mini"),
        nturns=1,
        numrows=20,

        systemprompt=(
            "You are an AI assistant specialized in Python programming. "
            "Provide detailed answers with code examples."
        ),
    )

distiset = pipeline.run()

df = distiset["default"]["train"].topandas()

for idx, row in df.iterrows():
    print(f"\n{'='*60}")
    print(f"Instruction: {row['instruction'][:100]}...")
    print(f"Response: {row['generation'][:100]}...")

from typing import Any
from distilabel.steps.tasks import Task
from distilabel.steps.tasks.typing import ChatType

class CodeReviewTask(Task):
    systemprompt: str = (
        "You are a senior code reviewer with extensive experience. "
        "Review the following code and provide constructive feedback."
    )

    @property
    def inputs(self) -> list[str]:
        return ["code"]

    @property
    def outputs(self) -> list[str]:
        return ["review", "score", "suggestions"]

    def formatinput(self, input: dict[str, Any]) -> ChatType:

        return [
            {"role": "system", "content": self.systemprompt},
            {
                "role": "user",
                "content": (
                    f"Review the following code:\n\n

Structured Generation with JSON Schema

Distilabel supports structured generation using JSON schemas:

Pipeline with Caching and Checkpoints

Distilabel automatically saves cache to avoid reprocessing:

Exporting to Hugging Face Hub and Argilla

Best Practices

1. Design Modular Pipelines

Separate each processing stage into distinct steps. This makes debugging, reuse, and testing easier:

2. Use Appropriate Batch Sizes

Adjust inputbatchsize based on API capacity and memory requirements:

For large datasets, use smaller batch sizes to avoid rate limiting. For small datasets, larger batch sizes improve throughput.

3. Implement Error Handling

Use the retries parameter and handle failures gracefully:

4. Validate Data Quality

Always validate output before using it for fine-tuning:

5. Use Specific System Prompts

Detailed system prompts produce more consistent and higher-quality data:

6. Diversify Data Sources

Combine multiple techniques for richer datasets:

7. Monitor Token Usage

Track token consumption to control costs:

8. Use Local LLMs for Fast Iteration

When developing pipelines, use local models via Ollama to save costs:

``

Conclusion

Distilabel is a powerful framework for building synthetic data generation and LLM evaluation pipelines. With its flexible pipeline-based architecture, multi-provider support, and built-in implementations of advanced techniques like Self-Instruct, Evol-Instruct, and UltraFeedback, Distilabel streamlines the process of creating high-quality datasets for model fine-tuning.

Key takeaways to remember:

• Start simple: Build a basic pipeline first, then add complexity incrementally
• Always validate: Never use synthetic data without quality validation
• Iterate with local models: Use Ollama or vLLM for development, then switch to production APIs
• Leverage integrations: Use Argilla for human-in-the-loop validation and Hugging Face Hub for dataset sharing
• Watch costs: Monitor token usage and use caching to avoid reprocessing

By incorporating Distilabel into your AI development workflow, you can significantly reduce the cost and time required to prepare high-quality training data while maintaining full control over the data generation process.