Tutorial Lengkap Fine-tuning LLM dengan LoRA dan PEFT

# Tutorial Lengkap Fine-tuning LLM dengan LoRA dan PEFT Fine-tuning Large Language Models (LLM) secara tradisional membutuhkan GPU memory yang sangat besar dan waktu yang lama. LoRA (Low-Rank Adaptat...

By Ruby Abdullah · · tutorial
LLMLoRAPEFTFine-tuningHugging FaceDeep Learning

Tutorial Lengkap Fine-tuning LLM dengan LoRA dan PEFT

Fine-tuning Large Language Models (LLM) secara tradisional membutuhkan GPU memory yang sangat besar dan waktu yang lama. LoRA (Low-Rank Adaptation) dan PEFT (Parameter-Efficient Fine-Tuning) memungkinkan Anda melakukan fine-tuning dengan resource yang jauh lebih kecil sambil mempertahankan performa yang kompetitif.

Apa itu LoRA dan PEFT?

LoRA (Low-Rank Adaptation)

LoRA adalah teknik yang menambahkan trainable low-rank matrices ke layer-layer model yang di-freeze. Daripada mengupdate semua parameter model:

  • Freeze semua weights original model
  • Inject trainable rank decomposition matrices (A dan B)
  • Hanya train matrices baru ini (< 1% dari total parameters)

Keuntungan:
  • Memory efficient: ~10x lebih kecil dari full fine-tuning
  • Faster training: Training lebih cepat
  • No inference latency: Weights bisa di-merge
  • Task switching: Swap LoRA adapters untuk tasks berbeda

PEFT (Parameter-Efficient Fine-Tuning)

PEFT adalah library dari Hugging Face yang menyediakan berbagai metode fine-tuning efisien:

  • LoRA: Low-rank adaptation
  • QLoRA: LoRA dengan quantization
  • Prefix Tuning: Prepend trainable tokens
  • Prompt Tuning: Learn soft prompts
  • IA3: Infused Adapter by Inhibiting and Amplifying

Instalasi

# Install packages

pip install transformers datasets accelerate peft bitsandbytes

pip install trl # Untuk training utilities

pip install wandb # Optional: experiment tracking

Untuk QLoRA (4-bit quantization)

pip install bitsandbytes>=0.41.0

Verify GPU

python -c "import torch; print(torch.cuda.isavailable())"

Quick Start: Fine-tune dengan LoRA

1. Basic LoRA Fine-tuning

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments

from peft import LoraConfig, getpeftmodel, TaskType

from datasets import loaddataset

from trl import SFTTrainer

Load base model dan tokenizer

modelname = "meta-llama/Llama-2-7b-hf" # Atau model lain

tokenizer = AutoTokenizer.frompretrained(modelname)

tokenizer.padtoken = tokenizer.eostoken

model = AutoModelForCausalLM.frompretrained(

modelname,

torchdtype="auto",

devicemap="auto"

)

Konfigurasi LoRA

loraconfig = LoraConfig(

r=8, # Rank of the update matrices

loraalpha=32, # Scaling factor

targetmodules=["qproj", "vproj"], # Modules to apply LoRA

loradropout=0.05,

bias="none",

tasktype=TaskType.CAUSALLM

)

Apply LoRA ke model

model = getpeftmodel(model, loraconfig)

model.printtrainableparameters()

Output: trainable params: 4,194,304 || all params: 6,742,609,920 || trainable%: 0.06%

Load dataset

dataset = loaddataset("timdettmers/openassistant-guanaco", split="train")

Training arguments

trainingargs = TrainingArguments(

outputdir="./lora-output",

numtrainepochs=3,

perdevicetrainbatchsize=4,

gradientaccumulationsteps=4,

learningrate=2e-4,

fp16=True,

loggingsteps=10,

savestrategy="epoch",

warmupratio=0.03,

)

Trainer

trainer = SFTTrainer(

model=model,

args=trainingargs,

traindataset=dataset,

tokenizer=tokenizer,

datasettextfield="text",

maxseqlength=512,

)

Train

trainer.train()

Save LoRA weights

model.savepretrained("./lora-adapter")

2. LoRA Config Parameters

from peft import LoraConfig

config = LoraConfig(

# Core parameters

r=8, # Rank: dimensi dari low-rank matrices

# Lebih tinggi = lebih ekspresif tapi lebih banyak params

# Typical: 4, 8, 16, 32, 64

loraalpha=32, # Scaling factor

# Rule: alpha = 2 r untuk hasil optimal

# Scaling = alpha / r

# Target modules (varies by model architecture)

targetmodules=[

"qproj", # Query projection

"kproj", # Key projection

"vproj", # Value projection

"oproj", # Output projection

"gateproj", # MLP gate

"upproj", # MLP up

"downproj", # MLP down

],

loradropout=0.05, # Dropout untuk regularization

bias="none", # "none", "all", "loraonly"

tasktype=TaskType.CAUSALLM, # Atau SEQCLS, SEQ2SEQLM, TOKENCLS

# Advanced options

modulestosave=["embedtokens", "lmhead"], # Modules to train fully

layerstotransform=None, # Specific layers to apply LoRA

layerspattern=None, # Pattern untuk layer selection

)

QLoRA: 4-bit Quantization + LoRA

QLoRA memungkinkan fine-tuning model besar dengan memory sangat kecil dengan menggunakan 4-bit quantization.

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

from peft import LoraConfig, getpeftmodel, preparemodelforkbittraining

import torch

Quantization config

bnbconfig = BitsAndBytesConfig(

loadin4bit=True,

bnb4bitquanttype="nf4", # Normal Float 4-bit

bnb4bitcomputedtype=torch.bfloat16,

bnb4bitusedoublequant=True, # Nested quantization

)

Load model dengan 4-bit quantization

modelname = "meta-llama/Llama-2-7b-hf"

model = AutoModelForCausalLM.frompretrained(

modelname,

quantizationconfig=bnbconfig,

devicemap="auto",

trustremotecode=True,

)

tokenizer = AutoTokenizer.frompretrained(modelname)

tokenizer.padtoken = tokenizer.eostoken

Prepare model untuk k-bit training

model = preparemodelforkbittraining(model)

LoRA config

loraconfig = LoraConfig(

r=16,

loraalpha=32,

targetmodules=[

"qproj", "kproj", "vproj", "oproj",

"gateproj", "upproj", "downproj"

],

loradropout=0.05,

bias="none",

tasktype="CAUSALLM"

)

Apply LoRA

model = getpeftmodel(model, loraconfig)

model.printtrainableparameters()

Dataset Preparation

1. Instruction Dataset Format

from datasets import Dataset

Format: instruction-input-output

data = [

{

"instruction": "Terjemahkan ke bahasa Indonesia",

"input": "Hello, how are you?",

"output": "Halo, apa kabar?"

},

{

"instruction": "Jelaskan konsep berikut",

"input": "Machine Learning",

"output": "Machine Learning adalah cabang dari AI..."

}

]

Buat formatting function

def formatinstruction(sample):

return f"""### Instruction:

{sample['instruction']}

Input:

{sample['input']}

Response:

{sample['output']}"""

dataset = Dataset.fromlist(data)

dataset = dataset.map(lambda x: {"text": formatinstruction(x)})

2. Chat Format (ChatML)

def formatchat(sample):

messages = sample.get("messages", [])

text = ""

for msg in messages:

role = msg["role"]

content = msg["content"]

if role == "system":

text += f"<|system|>\n{content}\n"

elif role == "user":

text += f"<|user|>\n{content}\n"

elif role == "assistant":

text += f"<|assistant|>\n{content}\n"

return {"text": text}

Data format

data = [

{

"messages": [

{"role": "system", "content": "Kamu adalah asisten yang helpful."},

{"role": "user", "content": "Apa itu Python?"},

{"role": "assistant", "content": "Python adalah bahasa pemrograman..."}

]

}

]

3. Custom Dataset dari File

import json

from datasets import Dataset

Load dari JSON

with open("trainingdata.json", "r") as f:

data = json.load(f)

Atau dari JSONL

data = []

with open("trainingdata.jsonl", "r") as f:

for line in f:

data.append(json.loads(line))

Buat dataset

dataset = Dataset.fromlist(data)

Split train/val

dataset = dataset.traintestsplit(testsize=0.1)

traindataset = dataset["train"]

evaldataset = dataset["test"]

Training dengan SFTTrainer

from transformers import TrainingArguments

from trl import SFTTrainer, DataCollatorForCompletionOnlyLM

Data collator untuk completion-only training

responsetemplate = "### Response:"

collator = DataCollatorForCompletionOnlyLM(

responsetemplate=responsetemplate,

tokenizer=tokenizer

)

Training arguments

trainingargs = TrainingArguments(

outputdir="./output",

# Training hyperparameters

numtrainepochs=3,

perdevicetrainbatchsize=4,

perdeviceevalbatchsize=4,

gradientaccumulationsteps=4,

gradientcheckpointing=True, # Memory saving

# Optimizer

learningrate=2e-4,

weightdecay=0.001,

warmupratio=0.03,

lrschedulertype="cosine",

# Precision

fp16=True, # Atau bf16=True untuk Ampere+

# Logging

loggingsteps=10,

loggingdir="./logs",

# Saving

savestrategy="steps",

savesteps=100,

savetotallimit=3,

# Evaluation

evalstrategy="steps",

evalsteps=100,

# Misc

reportto="wandb", # Atau "tensorboard"

runname="lora-experiment",

)

Trainer

trainer = SFTTrainer(

model=model,

args=trainingargs,

traindataset=traindataset,

evaldataset=evaldataset,

tokenizer=tokenizer,

datacollator=collator,

maxseqlength=1024,

packing=False, # True untuk packing multiple samples

)

Train

trainer.train()

Save

trainer.savemodel("./final-model")

model.savepretrained("./lora-adapter")

tokenizer.savepretrained("./lora-adapter")

Inference dengan LoRA

1. Load dan Merge Adapter

from transformers import AutoModelForCausalLM, AutoTokenizer

from peft import PeftModel

Load base model

basemodel = AutoModelForCausalLM.frompretrained(

"meta-llama/Llama-2-7b-hf",

torchdtype="auto",

devicemap="auto"

)

Load LoRA adapter

model = PeftModel.frompretrained(basemodel, "./lora-adapter")

Merge weights (optional, untuk inference lebih cepat)

model = model.mergeandunload()

Save merged model

model.savepretrained("./merged-model")

2. Inference

from transformers import AutoModelForCausalLM, AutoTokenizer

import torch

Load model (merged atau dengan adapter)

model = AutoModelForCausalLM.frompretrained(

"./merged-model",

torchdtype=torch.float16,

devicemap="auto"

)

tokenizer = AutoTokenizer.frompretrained("./merged-model")

def generateresponse(prompt, maxlength=512):

inputs = tokenizer(prompt, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=maxlength,

temperature=0.7,

topp=0.9,

dosample=True,

padtokenid=tokenizer.eostokenid

)

response = tokenizer.decode(outputs[0], skipspecialtokens=True)

return response[len(prompt):]

Test

prompt = """### Instruction:

Jelaskan apa itu machine learning

Response:

"""

response = generateresponse(prompt)

print(response)

3. Multiple LoRA Adapters

from peft import PeftModel

Load base model

basemodel = AutoModelForCausalLM.frompretrained("base-model")

Load adapter 1

model = PeftModel.frompretrained(basemodel, "./adapter-task1", adaptername="task1")

Load adapter 2

model.loadadapter("./adapter-task2", adaptername="task2")

Switch between adapters

model.setadapter("task1")

output1 = model.generate(...)

model.setadapter("task2")

output2 = model.generate(...)

Disable adapter (use base model)

with model.disableadapter():

baseoutput = model.generate(...)

Fine-tuning untuk Task Spesifik

1. Text Classification

from transformers import AutoModelForSequenceClassification

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForSequenceClassification.frompretrained(

"bert-base-uncased",

numlabels=3

)

loraconfig = LoraConfig(

r=8,

loraalpha=16,

targetmodules=["query", "value"],

loradropout=0.1,

bias="none",

tasktype=TaskType.SEQCLS

)

model = getpeftmodel(model, loraconfig)

2. Question Answering

from transformers import AutoModelForQuestionAnswering

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForQuestionAnswering.frompretrained("bert-base-uncased")

loraconfig = LoraConfig(

r=16,

loraalpha=32,

targetmodules=["query", "key", "value"],

tasktype=TaskType.QUESTIONANS

)

model = getpeftmodel(model, loraconfig)

3. Summarization

from transformers import AutoModelForSeq2SeqLM

from peft import LoraConfig, getpeftmodel, TaskType

model = AutoModelForSeq2SeqLM.frompretrained("t5-base")

loraconfig = LoraConfig(

r=8,

loraalpha=16,

targetmodules=["q", "v"],

tasktype=TaskType.SEQ2SEQLM

)

model = getpeftmodel(model, loraconfig)

Tips dan Best Practices

1. Hyperparameter Tuning

# Start dengan config ini

loraconfig = LoraConfig(

r=8, # Mulai kecil, naikkan jika underfitting

loraalpha=16, # 2 r

targetmodules=["qproj", "vproj"], # Minimal

loradropout=0.05,

)

Untuk task kompleks

loraconfig = LoraConfig(

r=32,

loraalpha=64,

targetmodules=[

"qproj", "kproj", "vproj", "oproj",

"gateproj", "upproj", "downproj"

],

loradropout=0.1,

)

2. Memory Optimization

# Gradient checkpointing

trainingargs = TrainingArguments(

gradientcheckpointing=True,

...

)

Mixed precision

trainingargs = TrainingArguments(

fp16=True, # Atau bf16=True

...

)

Gradient accumulation

trainingargs = TrainingArguments(

perdevicetrainbatchsize=1,

gradientaccumulationsteps=16, # Effective batch = 16

...

)

3. Debugging

# Check trainable parameters

model.printtrainableparameters()

Check LoRA modules

for name, param in model.namedparameters():

if param.requiresgrad:

print(f"{name}: {param.shape}")

Verify adapter loaded

print(model.peftconfig)

print(model.activeadapter)

4. Common Issues

# Issue: OOM (Out of Memory)

Solution: Reduce batch size, use gradient checkpointing

Issue: Loss not decreasing

Solution: Check learning rate, increase r, add more target modules

Issue: Overfitting

Solution: Increase dropout, reduce epochs, use more data

Issue: Catastrophic forgetting

Solution: Lower learning rate, use less epochs

Kesimpulan

LoRA dan PEFT memungkinkan fine-tuning LLM dengan resource minimal:

| Metode | Memory | Speed | Quality |

|--------|--------|-------|---------|

| Full Fine-tune | 100% | Slow | Best |

| LoRA | ~10% | Fast | Very Good |

| QLoRA | ~5% | Medium | Good |

Key takeaways:

  • Gunakan QLoRA untuk GPU memory terbatas
  • Mulai dengan r=8, naikkan jika perlu
  • Target minimal: qproj, vproj
  • Gunakan gradient checkpointing untuk memory
  • Merge weights untuk production inference
  • Simpan adapter terpisah untuk flexibility
  • Artikel Terkait

    Tutorial Lengkap Google Gemma: Model Open-Source dari Google DeepMind

    Tutorial Lengkap Google Gemma: Model Open-Source dari Google DeepMind Google Gemma adalah keluarga model bahasa open-sou...

    Tutorial Lengkap Hugging Face AutoTrain: Fine-Tuning Model AI Tanpa Ribet

    Tutorial Lengkap Hugging Face AutoTrain: Fine-Tuning Model AI Tanpa Ribet Hugging Face AutoTrain adalah platform dan lib...

    Tutorial Unsloth: Fine-Tuning LLM yang Cepat dan Hemat Memori

    Fine-Tuning LLM Secara Efisien dengan Unsloth Dahulu, melakukan fine-tuning model bahasa besar membutuhkan server multi-...

    Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom

    Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom Model bahasa besar (LLM) open-source terus berk...