Complete LlamaFactory Tutorial: All-in-One LLM Fine-Tuning Toolkit

Fine-tuning Large Language Models (LLMs) often involves complex configurations and significant engineering effort. LlamaFactory simplifies this entire process by providing an intuitive interface and support for numerous models and training methods. In this tutorial, we will learn how to use LlamaFactory from installation to deploying your fine-tuned model.

What is LlamaFactory?

LlamaFactory (or LLaMA-Factory) is an open-source toolkit that provides a unified framework for fine-tuning over 100 language models. It supports various training methods including full fine-tuning, LoRA, QLoRA, and other Parameter-Efficient Fine-Tuning (PEFT) approaches. One of LlamaFactory's standout features is LLaMA Board, a Gradio-based web interface that enables fine-tuning without writing any code.

Key Features

• Multi-Model Support: Supports LLaMA, Mistral, Qwen, Phi, Gemma, Yi, DeepSeek, and many more
• Diverse Training Methods: Full fine-tuning, LoRA, QLoRA, DoRA, and GaLore
• LLaMA Board: Web UI for no-code fine-tuning
• Integrated Datasets: Dozens of built-in datasets ready to use
• Automatic Evaluation: Automated benchmarking with MMLU, CMMLU, and C-Eval
• Model Export: Export to GGUF, vLLM, or other formats

Installation

System Requirements

Before installing LlamaFactory, ensure your system meets the following requirements:

• Python 3.8 or newer
• CUDA 11.8 or 12.1 (for NVIDIA GPUs)
• Minimum 16GB RAM (32GB recommended)
• GPU with minimum 8GB VRAM for QLoRA, 16GB for LoRA, 40GB+ for full fine-tuning

Installation from Source

git clone --depth 1 https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory
pip install -e ".[torch,metrics]"

Installation with Docker

docker pull hiyouga/llamafactory:latest

docker run --gpus all \
  -v ./hfcache:/root/.cache/huggingface \

  -v ./data:/app/data \
  -v ./output:/app/output \
  -p 7860:7860 \
  -e GRADIOSHARE=1 \
  hiyouga/llamafactory:latest

Verify Installation

llamafactory-cli version

Expected output:

----------------------------------------------------------
| Welcome to LLaMA Factory, version x.x.x               |
| Project page: https://github.com/hiyouga/LLaMA-Factory |

Optional Dependencies

# For GPTQ quantization
pip install auto-gptq

For AWQ quantization
pip install autoawq

For Flash Attention 2
pip install flash-attn --no-build-isolation

For vLLM inference
pip install vllm

Basic Usage: Fine-Tuning with CLI

Preparing Your Dataset

LlamaFactory uses a specific dataset format. Create a JSON file with the following structure:

[
  {
    "instruction": "Explain what machine learning is in simple terms.",
    "input": "",
    "output": "Machine learning is a branch of artificial intelligence where computers learn from data to make predictions or decisions without being explicitly programmed. Think of it like teaching a child to recognize animals - the more examples they see, the better they become at identifying new animals."
  },
  {
    "instruction": "Write Python code to calculate factorial.",
    "input": "5",
    "output": "def factorial(n):\n    if n <= 1:\n        return 1\n    return n * factorial(n - 1)\n\nresult = factorial(5)\nprint(f'Factorial of 5 is {result}')  # Output: 120"
  }
]

Save this file as data/mydataset.json. Then register it in data/datasetinfo.json:

{
  "mycustomdataset": {
    "filename": "mydataset.json",
    "columns": {
      "prompt": "instruction",
      "query": "input",
      "response": "output"
    }
  }
}

Chat/Conversation Dataset Format

For multi-turn conversation datasets, use this format:

[
  {
    "conversations": [
      {"from": "human", "value": "What is Python?"},