Torchtune Tutorial: Fine-Tuning LLMs with PyTorch Native

Introduction

Torchtune is PyTorch's official library for fine-tuning Large Language Models (LLMs). It is designed as a lightweight, modular, and highly customizable native PyTorch solution. Unlike other fine-tuning frameworks that often come with complex and hidden abstractions, Torchtune gives developers full control while providing ready-to-use components for common workflows.

Torchtune supports popular models including Llama 3, Mistral, Gemma, Phi, and Qwen. It offers various fine-tuning techniques ranging from full fine-tuning to parameter-efficient methods like LoRA and QLoRA. With direct integration into the PyTorch ecosystem, Torchtune is ideal for practitioners already familiar with PyTorch who want to fine-tune LLMs without relying on heavy third-party frameworks.

In this tutorial, we will learn how to use Torchtune from installation through fine-tuning models with various techniques, including configuring custom datasets and deploying fine-tuned models.

Why Torchtune?

Before diving into the implementation, here are several reasons why Torchtune deserves consideration:

Native PyTorch: Built entirely on top of PyTorch without additional abstraction layers

Modular: Every component (model, dataset, optimizer, loss) can be swapped and customized

Memory Efficient: Supports memory-saving techniques like QLoRA, gradient checkpointing, and mixed precision

Config-Driven: Workflows are managed through readable and modifiable YAML files

Multi-GPU: Supports distributed training via FSDP2 (Fully Sharded Data Parallel)

Ecosystem: Direct integration with Hugging Face Hub, Weights & Biases, and GGUF format

Installation

Prerequisites

Make sure you have Python 3.10+ and PyTorch 2.4+ installed.

# Check Python version
python --version

Check PyTorch version
python -c "import torch; print(torch.version)"

Installing Torchtune

# Install via pip (stable release) pip install torchtune Or install from source for the latest version pip install git+https://github.com/pytorch/torchtune.git Install with all features pip install torchtune[all]

Verify Installation

# Check torchtune version tune --version List available recipes tune ls

Output from tune ls will display all available fine-tuning recipes:

RECIPE CONFIG fullfinetunesingledevice llama32/1Bfullsingledevice fullfinetunedistributed llama32/3Bfull lorafinetunesingledevice llama32/1Blorasingledevice lorafinetunedistributed llama32/3Blora qlorafinetunesingledevice llama32/1Bqlorasingledevice knowledgedistillation llama32/1Bkdsingledevice ...

Downloading Models

Before fine-tuning, we need to download a base model. Torchtune provides the tune download command to fetch models from Hugging Face Hub.

# Download Llama 3.2 1B (small model, good for experiments) tune download meta-llama/Llama-3.2-1B-Instruct \ --output-dir ./models/Llama-3.2-1B-Instruct \ --hf-token YOURHFTOKEN Download Mistral 7B tune download mistralai/Mistral-7B-Instruct-v0.3 \ --output-dir ./models/Mistral-7B-Instruct \ --hf-token YOURHFTOKEN Download Qwen 2.5 0.5B (very lightweight) tune download Qwen/Qwen2.5-0.5B-Instruct \ --output-dir ./models/Qwen2.5-0.5B-Instruct

You will need a Hugging Face token for models that require special access like Llama. Get your token at huggingface.co/settings/tokens.

Basic Usage: LoRA Fine-Tuning

LoRA (Low-Rank Adaptation) is the most popular fine-tuning technique because it is memory efficient and fast. Let us start by fine-tuning Llama 3.2 1B using LoRA.

Viewing Default Configuration

# View the default configuration for LoRA single device tune cp llama32/1Blorasingledevice ./myloraconfig.yaml

Understanding the Configuration File

# myloraconfig.yaml Model model: component: torchtune.models.llama32.lorallama321b loraattnmodules: ['qproj', 'vproj', 'kproj', 'outputproj'] applyloratomlp: True applyloratooutput: False lorarank: 8 loraalpha: 16 Tokenizer tokenizer: component: torchtune.models.llama3.llama3tokenizer path: ./models/Llama-3.2-1B-Instruct/original/tokenizer.model maxseqlen: 2048 Checkpoint checkpointer: component: torchtune.training.FullModelHFCheckpointer checkpointdir: ./models/Llama-3.2-1B-Instruct checkpointfiles: [model.safetensors] outputdir: ./output/lora-llama3.2-1b modeltype: LLAMA32 Dataset dataset: component: torchtune.datasets.alpacacleaneddataset seed: 42 shuffle: True Training epochs: 1 maxstepsperepoch: null batchsize: 4 gradientaccumulationsteps: 4 optimizer: component: torch.optim.AdamW fused: True lr: 2e-5 weightdecay: 0.01 lrscheduler: component: torchtune.training.lrschedulers.getcosineschedulewithwarmup numwarmupsteps: 100 Hardware device: cuda dtype: bf16 enableactivationcheckpointing: True Logging metriclogger: component: torchtune.training.metriclogging.WandBLogger project: torchtune-tutorial logeverynsteps: 10

Running Fine-Tuning

# Run LoRA fine-tuning with default config
tune run lorafinetunesingledevice \

    --config myloraconfig.yaml


Or override parameters directly from the command line
tune run lorafinetunesingledevice \
    --config myloraconfig.yaml \
    batchsize=2 \

    epochs=3 \
    optimizer.lr=1e-5

Monitoring Training

Torchtune supports various loggers for monitoring:

# Using Weights & Biasesmetriclogger: component: torchtune.training.metriclogging.WandBLogger project: my-finetune-project Using TensorBoard metriclogger: component: torchtune.training.metriclogging.TensorBoardLogger logdir: ./logs Using stdout (no external logger) metriclogger: component: torchtune.training.metriclogging.StdoutLogger

Custom Datasets

One of Torchtune's strengths is its flexibility in working with custom datasets.

Instruct Format

Create a JSON file with the following format:

[ { "instruction": "Explain what machine learning is in one paragraph.", "input": "", "output": "Machine learning is a branch of artificial intelligence that enables computers to learn from data without being explicitly programmed. ML systems use statistical algorithms to find patterns in data, make predictions, and improve their performance as more data is processed." }, { "instruction": "Translate the following text to French.", "input": "I am learning Python programming.", "output": "J'apprends la programmation Python." } ]

Custom Dataset Configuration

# Using a local dataset with instruct format
dataset:
  component: torchtune.datasets.instructdataset

  source: json
  datafiles: ./data/mydataset.json

  split: train
  columnmap:
    instruction: instruction
    input: input
    output: output
  trainoninput: False

Using a dataset from Hugging Face
dataset:
  component: torchtune.datasets.instructdataset

  source: tatsu-lab/alpaca
  split: train
  trainoninput: False

Chat Format (Multi-turn)

For multi-turn conversation datasets:

[ { "messages": [ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "What is Python?"}, {"role": "assistant", "content": "Python is a high-level programming language designed for code readability."}, {"role": "user", "content": "What are its advantages?"}, {"role": "assistant", "content": "Python has clean syntax, extensive libraries, a large community, and supports multiple programming paradigms."} ] } ]

# Configuration for chat dataset dataset:component: torchtune.datasets.chatdataset source: json datafiles: ./data/chatdata.json split: train conversationcolumn: messages conversationstyle: openai trainoninput: False

Building a Custom Dataset Class

For more specific needs, you can create your own dataset builder:

# mydataset.py
from torchtune.data import Message
from torchtune.datasets import SFTDataset
from torchtune.modules.tokenizers import ModelTokenizer

def mycustomdataset(

    tokenizer: ModelTokenizer,
    source: str = "json",
    datafiles: str = "./data/custom.json",
    maxseqlen: int = 2048,
    kwargs,

) -> SFTDataset:

    def transform(sample):
        messages = [
            Message(
                role="user",
                content=sample["question"],
            ),
            Message(
                role="assistant",
                content=sample["answer"],
            ),
        ]
        return {"messages": messages}

    return SFTDataset(
        source=source,
        datafiles=datafiles,
        messagetransform=transform,

        modeltransform=tokenizer,
        maxseqlen=maxseqlen,
        kwargs,
    )

Use it in your configuration:

dataset: component: mydataset.mycustomdataset datafiles: ./data/custom.json maxseqlen: 2048

Advanced Usage

QLoRA Fine-Tuning

QLoRA combines 4-bit quantization with LoRA, enabling fine-tuning of large models on GPUs with limited VRAM.

# Copy QLoRA configuration tune cp llama32/1Bqlorasingledevice ./myqloraconfig.yaml

# myqloraconfig.yaml (differences from LoRA) model: component: torchtune.models.llama32.qlorallama321b loraattnmodules: ['qproj', 'vproj', 'kproj', 'outputproj'] applyloratomlp: True lorarank: 8 loraalpha: 16 QLoRA uses specific optimizer settings optimizer: component: torch.optim.AdamW fused: True lr: 2e-4 weightdecay: 0.01 Quantization is automatically handled by the model component

# Run QLoRA fine-tuning
tune run lorafinetunesingledevice \

    --config myqloraconfig.yaml

Full Fine-Tuning

For the best results (requires larger GPU):

# fullfinetuneconfig.yaml model: component: torchtune.models.llama32.llama321b checkpointer: component: torchtune.training.FullModelHFCheckpointer checkpointdir: ./models/Llama-3.2-1B-Instruct checkpointfiles: [model.safetensors] outputdir: ./output/full-llama3.2-1b modeltype: LLAMA32 optimizer: component: torch.optim.AdamW fused: True lr: 5e-6 weightdecay: 0.01 enableactivationcheckpointing: True compile: True # Use torch.compile for speedup

# Full fine-tuning on single GPU
tune run fullfinetunesingledevice \

    --config fullfinetuneconfig.yaml

Distributed Training (Multi-GPU)

For training on multiple GPUs using FSDP2:

# distributedconfig.yaml model: component: torchtune.models.llama32.lorallama323b loraattnmodules: ['qproj', 'vproj'] lorarank: 16 loraalpha: 32 Distributed-specific settings enableactivationcheckpointing: True fsdpcpuoffload: False # Set True if VRAM is limited

# Run distributed training on 4 GPUs
tune run --nprocpernode 4 lorafinetunedistributed \
    --config distributedconfig.yaml

Knowledge Distillation

Torchtune supports knowledge distillation to transfer knowledge from a large model to a smaller one:

# kdconfig.yaml teachermodel: component: torchtune.models.llama32.llama323b studentmodel: component: torchtune.models.llama32.lorallama321b loraattnmodules: ['qproj', 'vproj'] lorarank: 8 kdloss: component: torchtune.modules.loss.ForwardKLWithChunkedOutputLoss kdratio: 0.5 # Balance between task loss and KD loss

tune run knowledgedistillationsingledevice \ --config kdconfig.yaml

Model Evaluation

Torchtune integrates with EleutherAI's lm-evaluation-harness for evaluation:

# Install evaluation harness
pip install lm-eval

Evaluate the fine-tuned model
tune run eleuthereval \

    --config eleutherevaluation \
    checkpointer.checkpointdir=./output/lora-llama3.2-1b \

    tasks=["hellaswag", "arceasy", "truthfulqamc2"] \

    limit=100

Using the Model Programmatically

import torch
from torchtune.models.llama32 import llama321b
from torchtune.models.llama3 import llama3tokenizer

from torchtune.training import FullModelHFCheckpointer

Load tokenizer
tokenizer = llama3tokenizer(
    path="./models/Llama-3.2-1B-Instruct/original/tokenizer.model"
)

Load model from fine-tuned checkpoint
checkpointer = FullModelHFCheckpointer(
    checkpointdir="./output/lora-llama3.2-1b",

    checkpointfiles=["model-00001-of-00001.safetensors"],
    outputdir="./output/merged",

    modeltype="LLAMA32",

)

model = llama321b()

statedict = checkpointer.loadcheckpoint()

model.loadstatedict(statedict["model"])
model.eval()

Generate text
prompt = "Explain the concept of machine learning:"
tokens = tokenizer.encode(prompt, addbos=True)

inputids = torch.tensor([tokens], dtype=torch.long)

with torch.nograd():

    output = model.generate(inputids, maxnewtokens=256)

response = tokenizer.decode(output[0].tolist())
print(response)

Exporting to GGUF Format

For deployment using llama.cpp:

# Export model to GGUF format
tune run quantize \
    --config quantization \
    checkpointer.checkpointdir=./output/lora-llama3.2-1b \

    quantizer.component=torchtune.training.quantization.Int8DynActInt4WeightQuantizer

Best Practices

1. Hyperparameter Selection

# Recommended hyperparameters by model size Small models (1B-3B) LoRA rank: 8-16, lr: 2e-5 to 5e-5 Effective batch size: 16-32 Medium models (7B-13B) LoRA rank: 16-32, lr: 1e-5 to 3e-5 Effective batch size: 8-16 Large models (70B+) LoRA rank: 32-64, lr: 5e-6 to 1e-5 Effective batch size: 4-8

2. Memory Management

# Memory-saving strategies (ordered from most to least savings) 1. Use QLoRA (4-bit quantization + LoRA) model:component: torchtune.models.llama32.qlorallama321b 2. Enable activation checkpointing enableactivationcheckpointing: True 3. Reduce batch size, increase gradient accumulation batchsize: 1 gradientaccumulationsteps: 16 4. Reduce maxseqlen tokenizer: maxseqlen: 1024 # Default is usually 2048 or 4096 5. Use FSDP CPU offload (multi-GPU) fsdpcpuoffload: True

3. Dataset Quality

Ensure the dataset is clean and consistent in format
Use trainoninput: False for instruction-following so the model only learns to generate outputs
1,000 high-quality samples are better than 100,000 noisy samples
Diversify instructions to avoid overfitting to specific patterns

4. Monitoring and Evaluation

# Log every N steps
logeverynsteps: 10


Save checkpoints periodically
saveeverynsteps: 500

Use WandB for visual monitoring
metriclogger:

  component: torchtune.training.metriclogging.WandBLogger
  project: my-project
  logdir: ./logs

5. Reproducibility

# Always set seed for reproducibility seed: 42 shuffle: True Use deterministic algorithmstorchdeterministic: True

6. Speed Optimization

# Enable torch.compile for speedup compile: True Use fused optimizer optimizer: component: torch.optim.AdamW fused: True Use bf16 for training (faster than fp16) dtype: bf16

Troubleshooting

CUDA Out of Memory Error

# Step-by-step solutions:
1. Reduce batchsize

2. Enable activation checkpointing
3. Reduce maxseqlen

4. Use QLoRA instead of LoRA
5. Use gradient accumulation

Model Not Converging

# Checklist:
1. Check learning rate (too high = diverge, too low = slow)
2. Verify dataset format is correct (inspect a few samples)
3. Increase number of epochs
4. Check whether trainoninput matches your needs

Error Loading Checkpoint

# Ensure modeltype matches the model being used Ensure checkpointfiles includes all shard files Example for multi-shard models: checkpointfiles: [ model-00001-of-00003.safetensors, model-00002-of-00003.safetensors, model-00003-of-00003.safetensors, ]

Comparison with Other Frameworks

|---------|-----------|---------------|---------|---------|

| Native PyTorch | Yes | No | No | No |

| Knowledge Distillation | Yes | No | No | No |

Conclusion

Torchtune is an excellent choice for practitioners who want full control over the LLM fine-tuning process while staying within the familiar PyTorch ecosystem. With its modular architecture and config-driven workflow, Torchtune makes it easy to experiment with various fine-tuning techniques without writing excessive boilerplate code.

Key takeaways to remember:

Start with LoRA or QLoRA for initial experiments before moving to full fine-tuning if needed
Invest time in preparing quality datasets rather than blindly increasing data volume
Use YAML configs to maintain reproducibility and experiment documentation
Leverage integration with WandB or TensorBoard for visual training monitoring
For deployment, export models to GGUF format so they can run with llama.cpp on consumer hardware

With a solid understanding of Torchtune, you can efficiently fine-tune LLMs and produce models tailored to your specific project requirements.