Torchtune Tutorial: Fine-Tuning LLMs with PyTorch Native

Introduction

Torchtune is PyTorch's official library for fine-tuning Large Language Models (LLMs). It is designed as a lightweight, modular, and highly customizable native PyTorch solution. Unlike other fine-tuning frameworks that often come with complex and hidden abstractions, Torchtune gives developers full control while providing ready-to-use components for common workflows.

Torchtune supports popular models including Llama 3, Mistral, Gemma, Phi, and Qwen. It offers various fine-tuning techniques ranging from full fine-tuning to parameter-efficient methods like LoRA and QLoRA. With direct integration into the PyTorch ecosystem, Torchtune is ideal for practitioners already familiar with PyTorch who want to fine-tune LLMs without relying on heavy third-party frameworks.

In this tutorial, we will learn how to use Torchtune from installation through fine-tuning models with various techniques, including configuring custom datasets and deploying fine-tuned models.

Why Torchtune?

Before diving into the implementation, here are several reasons why Torchtune deserves consideration:

Native PyTorch: Built entirely on top of PyTorch without additional abstraction layers

Modular: Every component (model, dataset, optimizer, loss) can be swapped and customized

Memory Efficient: Supports memory-saving techniques like QLoRA, gradient checkpointing, and mixed precision

Config-Driven: Workflows are managed through readable and modifiable YAML files

Multi-GPU: Supports distributed training via FSDP2 (Fully Sharded Data Parallel)

Ecosystem: Direct integration with Hugging Face Hub, Weights & Biases, and GGUF format

Installation

Prerequisites

Make sure you have Python 3.10+ and PyTorch 2.4+ installed.

# Check Python version
python --version

Check PyTorch version
python -c "import torch; print(torch.version)"

Installing Torchtune

# Install via pip (stable release)
pip install torchtune

Or install from source for the latest version
pip install git+https://github.com/pytorch/torchtune.git

Install with all features
pip install torchtune[all]

Verify Installation

# Check torchtune version
tune --version

List available recipes
tune ls

Output from tune ls will display all available fine-tuning recipes:

RECIPE                          CONFIG
fullfinetunesingledevice     llama32/1Bfullsingledevice

fullfinetunedistributed       llama32/3Bfull

lorafinetunesingledevice     llama32/1Blorasingledevice
lorafinetunedistributed       llama32/3Blora
qlorafinetunesingledevice    llama32/1Bqlorasingledevice

knowledgedistillation          llama32/1Bkdsingledevice
...

Downloading Models

Before fine-tuning, we need to download a base model. Torchtune provides the tune download command to fetch models from Hugging Face Hub.

# Download Llama 3.2 1B (small model, good for experiments)
tune download meta-llama/Llama-3.2-1B-Instruct \
    --output-dir ./models/Llama-3.2-1B-Instruct \
    --hf-token YOURHFTOKEN

Download Mistral 7B
tune download mistralai/Mistral-7B-Instruct-v0.3 \
    --output-dir ./models/Mistral-7B-Instruct \
    --hf-token YOURHFTOKEN

Download Qwen 2.5 0.5B (very lightweight)
tune download Qwen/Qwen2.5-0.5B-Instruct \
    --output-dir ./models/Qwen2.5-0.5B-Instruct

You will need a Hugging Face token for models that require special access like Llama. Get your token at huggingface.co/settings/tokens.

Basic Usage: LoRA Fine-Tuning

LoRA (Low-Rank Adaptation) is the most popular fine-tuning technique because it is memory efficient and fast. Let us start by fine-tuning Llama 3.2 1B using LoRA.

Viewing Default Configuration

# View the default configuration for LoRA single device
tune cp llama32/1Blorasingledevice ./myloraconfig.yaml