Complete Tutorial: Fine-tuning LLMs with LoRA and PEFT

# Tutorial Lengkap Fine-tuning LLM dengan LoRA dan PEFT Fine-tuning Large Language Models (LLM) secara tradisional membutuhkan GPU memory yang sangat besar dan waktu yang lama. LoRA (Low-Rank Adaptat...

By Ruby Abdullah · · tutorial
LLMLoRAPEFTFine-tuningHugging FaceDeep Learning

Complete Tutorial: Fine-tuning LLMs with LoRA and PEFT

Fine-tuning Large Language Models (LLMs) traditionally requires massive GPU memory and long training times. LoRA (Low-Rank Adaptation) and PEFT (Parameter-Efficient Fine-Tuning) enable you to fine-tune with significantly fewer resources while maintaining competitive performance.

What are LoRA and PEFT?

LoRA (Low-Rank Adaptation)

LoRA is a technique that adds trainable low-rank matrices to frozen model layers. Instead of updating all model parameters:

  • Freeze all original model weights
  • Inject trainable rank decomposition matrices (A and B)
  • Only train these new matrices (< 1% of total parameters)

Benefits:
  • Memory efficient: ~10x smaller than full fine-tuning
  • Faster training: Training is faster
  • No inference latency: Weights can be merged
  • Task switching: Swap LoRA adapters for different tasks

PEFT (Parameter-Efficient Fine-Tuning)

PEFT is a library from Hugging Face that provides various efficient fine-tuning methods:

  • LoRA: Low-rank adaptation
  • QLoRA: LoRA with quantization
  • Prefix Tuning: Prepend trainable tokens
  • Prompt Tuning: Learn soft prompts
  • IA3: Infused Adapter by Inhibiting and Amplifying

Installation

# Install packages

pip install transformers datasets accelerate peft bitsandbytes

pip install trl # For training utilities

pip install wandb # Optional: experiment tracking

For QLoRA (4-bit quantization)

pip install bitsandbytes>=0.41.0

Verify GPU

python -c "import torch; print(torch.cuda.isavailable())"

Quick Start: Fine-tune with LoRA

1. Basic LoRA Fine-tuning

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments

from peft import LoraConfig, getpeftmodel, TaskType

from datasets import loaddataset

from trl import SFTTrainer

Load base model and tokenizer

modelname = "meta-llama/Llama-2-7b-hf" # Or another model

tokenizer = AutoTokenizer.frompretrained(modelname)

tokenizer.padtoken = tokenizer.eostoken

model = AutoModelForCausalLM.frompretrained(

modelname,

torchdtype="auto",

devicemap="auto"

)

LoRA configuration

loraconfig = LoraConfig(

r=8, # Rank of the update matrices

loraalpha=32, # Scaling factor

targetmodules=["qproj", "vproj"], # Modules to apply LoRA

loradropout=0.05,

bias="none",

tasktype=TaskType.CAUSALLM

)

Apply LoRA to model

model = getpeftmodel(model, loraconfig)

model.printtrainableparameters()

Output: trainable params: 4,194,304 || all params: 6,742,609,920 || trainable%: 0.06%

Load dataset

dataset = loaddataset("timdettmers/openassistant-guanaco", split="train")

Training arguments

trainingargs = TrainingArguments(

outputdir="./lora-output",

numtrainepochs=3,

perdevicetrainbatchsize=4,

gradientaccumulationsteps=4,

learningrate=2e-4,

fp16=True,

loggingsteps=10,

savestrategy="epoch",

warmupratio=0.03,

)

Trainer

trainer = SFTTrainer(

model=model,

args=trainingargs,

traindataset=dataset,

tokenizer=tokenizer,

datasettextfield="text",

maxseqlength=512,

)

Train

trainer.train()

Save LoRA weights

model.savepretrained("./lora-adapter")

2. LoRA Config Parameters

from peft import LoraConfig

config = LoraConfig(

# Core parameters

r=8, # Rank: dimension of low-rank matrices

# Higher = more expressive but more params

# Typical: 4, 8, 16, 32, 64

loraalpha=32, # Scaling factor

# Rule: alpha = 2 r for optimal results

# Scaling = alpha / r

# Target modules (varies by model architecture)

targetmodules=[

"qproj", # Query projection

"kproj", # Key projection

"vproj", # Value projection

"oproj", # Output projection

"gateproj", # MLP gate

"upproj", # MLP up

"downproj", # MLP down

],

loradropout=0.05, # Dropout for regularization

bias="none", # "none", "all", "loraonly"

tasktype=TaskType.CAUSALLM, # Or SEQCLS, SEQ2SEQLM, TOKENCLS

# Advanced options

modulestosave=["embedtokens", "lmhead"], # Modules to train fully

layerstotransform=None, # Specific layers to apply LoRA

layerspattern=None, # Pattern for layer selection

)

QLoRA: 4-bit Quantization + LoRA

QLoRA enables fine-tuning large models with minimal memory by using 4-bit quantization.

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

from peft import LoraConfig, getpeftmodel, preparemodelforkbittraining

import torch

Quantization config

bnbconfig = BitsAndBytesConfig(

loadin4bit=True,

bnb4bitquanttype="nf4", # Normal Float 4-bit

bnb4bitcomputedtype=torch.bfloat16,

bnb4bitusedoublequant=True, # Nested quantization

)

Load model with 4-bit quantization

modelname = "meta-llama/Llama-2-7b-hf"

model = AutoModelForCausalLM.frompretrained(

modelname,

quantizationconfig=bnbconfig,

devicemap="auto",

trustremotecode=True,

)

tokenizer = AutoTokenizer.frompretrained(modelname)

tokenizer.padtoken = tokenizer.eostoken

Prepare model for k-bit training

model = preparemodelforkbittraining(model)

LoRA config

loraconfig = LoraConfig(

r=16,

loraalpha=32,

targetmodules=[

"qproj", "kproj", "vproj", "oproj",

"gateproj", "upproj", "downproj"

],

loradropout=0.05,

bias="none",

tasktype="CAUSALLM"

)

Apply LoRA

model = getpeftmodel(model, loraconfig)

model.printtrainableparameters()

Dataset Preparation

1. Instruction Dataset Format

from datasets import Dataset

Format: instruction-input-output

data = [

{

"instruction": "Translate to Spanish",

"input": "Hello, how are you?",

"output": "Hola, como estas?"

},

{

"instruction": "Explain the following concept",

"input": "Machine Learning",

"output": "Machine Learning is a branch of AI..."

}

]

Create formatting function

def formatinstruction(sample):

return f"""### Instruction:

{sample['instruction']}

Input:

{sample['input']}

Response:

{sample['output']}"""

dataset = Dataset.fromlist(data)

dataset = dataset.map(lambda x: {"text": formatinstruction(x)})

2. Chat Format (ChatML)

def formatchat(sample):

messages = sample.get("messages", [])

text = ""

for msg in messages:

role = msg["role"]

content = msg["content"]

if role == "system":

text += f"<|system|>\n{content}\n"

elif role == "user":

text += f"<|user|>\n{content}\n"

elif role == "assistant":

text += f"<|assistant|>\n{content}\n"

return {"text": text}

Data format

data = [

{

"messages": [

{"role": "system", "content": "You are a helpful assistant."},

{"role": "user", "content": "What is Python?"},

{"role": "assistant", "content": "Python is a programming language..."}

]

}

]

3. Custom Dataset from File

import json

from datasets import Dataset

Load from JSON

with open("trainingdata.json", "r") as f:

data = json.load(f)

Or from JSONL

data = []

with open("trainingdata.jsonl", "r") as f:

for line in f:

data.append(json.loads(line))

Create dataset

dataset = Dataset.fromlist(data)

Split train/val

dataset = dataset.traintestsplit(testsize=0.1)

traindataset = dataset["train"]

evaldataset = dataset["test"]

Training with SFTTrainer

from transformers import TrainingArguments

from trl import SFTTrainer, DataCollatorForCompletionOnlyLM

Data collator for completion-only training

responsetemplate = "### Response:"

collator = DataCollatorForCompletionOnlyLM(

responsetemplate=responsetemplate,

tokenizer=tokenizer

)

Training arguments

trainingargs = TrainingArguments(

outputdir="./output",

# Training hyperparameters

numtrainepochs=3,

perdevicetrainbatchsize=4,

perdeviceevalbatchsize=4,

gradientaccumulationsteps=4,

gradientcheckpointing=True, # Memory saving

# Optimizer

learningrate=2e-4,

weightdecay=0.001,

warmupratio=0.03,

lrschedulertype="cosine",

# Precision

fp16=True, # Or bf16=True for Ampere+

# Logging

loggingsteps=10,

loggingdir="./logs",

# Saving

savestrategy="steps",

savesteps=100,

savetotallimit=3,

# Evaluation

evalstrategy="steps",

evalsteps=100,

# Misc

reportto="wandb", # Or "tensorboard"

runname="lora-experiment",

)

Trainer

trainer = SFTTrainer(

model=model,

args=trainingargs,

traindataset=traindataset,

evaldataset=evaldataset,

tokenizer=tokenizer,

datacollator=collator,

maxseqlength=1024,

packing=False, # True for packing multiple samples

)

Train

trainer.train()

Save

trainer.savemodel("./final-model")

model.savepretrained("./lora-adapter")

tokenizer.savepretrained("./lora-adapter")

Inference with LoRA

1. Load and Merge Adapter

from transformers import AutoModelForCausalLM, AutoTokenizer

from peft import PeftModel

Load base model

basemodel = AutoModelForCausalLM.frompretrained(

"meta-llama/Llama-2-7b-hf",

torchdtype="auto",

devicemap="auto"

)

Load LoRA adapter

model = PeftModel.frompretrained(basemodel, "./lora-adapter")

Merge weights (optional, for faster inference)

model = model.mergeandunload()

Save merged model

model.savepretrained("./merged-model")

2. Inference

from transformers import AutoModelForCausalLM, AutoTokenizer

import torch

Load model (merged or with adapter)

model = AutoModelForCausalLM.frompretrained(

"./merged-model",

torchdtype=torch.float16,

devicemap="auto"

)

tokenizer = AutoTokenizer.frompretrained("./merged-model")

def generateresponse(prompt, maxlength=512):

inputs = tokenizer(prompt, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=maxlength,

temperature=0.7,

topp=0.9,

dosample=True,

padtokenid=tokenizer.eostokenid

)

response = tokenizer.decode(outputs[0], skipspecialtokens=True)

return response[len(prompt):]

Test

prompt = """### Instruction:

Explain what machine learning is

Response:

"""

response = generateresponse(prompt)

print(response)

3. Multiple LoRA Adapters

from peft import PeftModel

Load base model

basemodel = AutoModelForCausalLM.frompretrained("base-model")

Load adapter 1

model = PeftModel.frompretrained(basemodel, "./adapter-task1", adaptername="task1")

Load adapter 2

model.loadadapter("./adapter-task2", adaptername="task2")

Switch between adapters

model.setadapter("task1")

output1 = model.generate(...)

model.setadapter("task2")

output2 = model.generate(...)

Disable adapter (use base model)

with model.disableadapter():

baseoutput = model.generate(...)

Fine-tuning for Specific Tasks

1. Text Classification

from transformers import AutoModelForSequenceClassification

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForSequenceClassification.frompretrained(

"bert-base-uncased",

numlabels=3

)

loraconfig = LoraConfig(

r=8,

loraalpha=16,

targetmodules=["query", "value"],

loradropout=0.1,

bias="none",

tasktype=TaskType.SEQCLS

)

model = getpeftmodel(model, loraconfig)

2. Question Answering

from transformers import AutoModelForQuestionAnswering

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForQuestionAnswering.frompretrained("bert-base-uncased")

loraconfig = LoraConfig(

r=16,

loraalpha=32,

targetmodules=["query", "key", "value"],

tasktype=TaskType.QUESTIONANS

)

model = getpeftmodel(model, loraconfig)

3. Summarization

from transformers import AutoModelForSeq2SeqLM

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForSeq2SeqLM.frompretrained("t5-base")

loraconfig = LoraConfig(

r=8,

loraalpha=16,

targetmodules=["q", "v"],

tasktype=TaskType.SEQ2SEQLM

)

model = getpeftmodel(model, loraconfig)

Tips and Best Practices

1. Hyperparameter Tuning

# Start with this config

loraconfig = LoraConfig(

r=8, # Start small, increase if underfitting

loraalpha=16, # 2 r

targetmodules=["qproj", "vproj"], # Minimal

loradropout=0.05,

)

For complex tasks

loraconfig = LoraConfig(

r=32,

loraalpha=64,

targetmodules=[

"qproj", "kproj", "vproj", "oproj",

"gateproj", "upproj", "downproj"

],

loradropout=0.1,

)

2. Memory Optimization

# Gradient checkpointing

trainingargs = TrainingArguments(

gradientcheckpointing=True,

...

)

Mixed precision

trainingargs = TrainingArguments(

fp16=True, # Or bf16=True

...

)

Gradient accumulation

trainingargs = TrainingArguments(

perdevicetrainbatchsize=1,

gradientaccumulationsteps=16, # Effective batch = 16

...

)

3. Debugging

# Check trainable parameters

model.printtrainableparameters()

Check LoRA modules

for name, param in model.namedparameters():

if param.requiresgrad:

print(f"{name}: {param.shape}")

Verify adapter loaded

print(model.peftconfig)

print(model.activeadapter)

4. Common Issues

# Issue: OOM (Out of Memory)

Solution: Reduce batch size, use gradient checkpointing

Issue: Loss not decreasing

Solution: Check learning rate, increase r, add more target modules

Issue: Overfitting

Solution: Increase dropout, reduce epochs, use more data

Issue: Catastrophic forgetting

Solution: Lower learning rate, use fewer epochs

Conclusion

LoRA and PEFT enable LLM fine-tuning with minimal resources:

| Method | Memory | Speed | Quality |

|--------|--------|-------|---------|

| Full Fine-tune | 100% | Slow | Best |

| LoRA | ~10% | Fast | Very Good |

| QLoRA | ~5% | Medium | Good |

Key takeaways:

  • Use QLoRA for limited GPU memory
  • Start with r=8, increase if needed
  • Minimum targets: qproj, vproj
  • Use gradient checkpointing for memory
  • Merge weights for production inference
  • Save adapters separately for flexibility
  • Related Articles

    Complete Google Gemma Tutorial: Open-Source Models from Google DeepMind

    Tutorial Lengkap Google Gemma: Model Open-Source dari Google DeepMind Google Gemma adalah keluarga model bahasa open-sou...

    Complete Guide to Hugging Face AutoTrain: Fine-Tune AI Models Without the Hassle

    Tutorial Lengkap Hugging Face AutoTrain: Fine-Tuning Model AI Tanpa Ribet Hugging Face AutoTrain adalah platform dan lib...

    Unsloth Tutorial: Fast and Memory-Efficient LLM Fine-Tuning

    Fine-Tuning LLM Secara Efisien dengan Unsloth Dahulu, melakukan fine-tuning model bahasa besar membutuhkan server multi-...

    MergeKit Tutorial: Merging LLM Models to Create Custom AI Models

    Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom Model bahasa besar (LLM) open-source terus berk...