Complete Google Gemma Tutorial: Open-Source Models from Google DeepMind

# Tutorial Lengkap Google Gemma: Model Open-Source dari Google DeepMind Google Gemma adalah keluarga model bahasa open-source yang dikembangkan oleh Google DeepMind. Dibangun dengan teknologi dan ris...

By Ruby Abdullah · · tutorial
Google GemmaLLMOpen SourceFine-tuningHugging Face

Complete Google Gemma Tutorial: Open-Source Models from Google DeepMind

Google Gemma is a family of open-source language models developed by Google DeepMind. Built with the same technology and research behind the Gemini models, Gemma offers highly competitive performance in lightweight and efficient packages. This tutorial covers how to use Gemma for various purposes, from simple inference to fine-tuning for specific needs.

Why Gemma?

Gemma comes in several size variants (2B, 7B, 9B, 27B parameters) designed to run efficiently on diverse hardware. Key advantages of Gemma compared to other open-source models:

  • High performance per parameter: Gemma consistently outperforms models of similar size across various benchmarks
  • Open license: Can be used for both commercial and research purposes
  • Broad ecosystem support: Compatible with Hugging Face Transformers, vLLM, Ollama, llama.cpp, and many other frameworks
  • Specialized variants: CodeGemma for coding, PaliGemma for vision, and RecurrentGemma for memory efficiency
  • Hardware optimization: Optimized for NVIDIA GPUs, Google TPUs, and even CPUs

Installation and Setup

Prerequisites

Make sure you have Python 3.9 or newer and pip installed.

python --version

pip --version

Library Installation

pip install torch transformers accelerate bitsandbytes

pip install huggingfacehub

To access Gemma models, you need a Hugging Face account and must accept the Gemma license agreement:

pip install huggingfacehub

huggingface-cli login

Enter your Hugging Face token when prompted. You can get your token from Settings > Access Tokens on the Hugging Face website.

import torch

print(f"CUDA available: {torch.cuda.isavailable()}")

if torch.cuda.isavailable():

print(f"GPU: {torch.cuda.getdevicename(0)}")

print(f"Memory: {torch.cuda.getdeviceproperties(0).totalmem / 1e9:.1f} GB")

Basic Usage: Text Generation

Using Pipeline (Easiest Approach)

from transformers import pipeline

generator = pipeline(

"text-generation",

model="google/gemma-2-9b-it",

devicemap="auto",

torchdtype="auto"

)

prompt = "Explain the concept of machine learning in 3 paragraphs:"

output = generator(

prompt,

maxnewtokens=512,

temperature=0.7,

topp=0.9,

dosample=True

)

print(output[0]["generatedtext"])

Using Model and Tokenizer Directly

For more granular control, you can load the model and tokenizer separately:

from transformers import AutoTokenizer, AutoModelForCausalLM

import torch

modelid = "google/gemma-2-9b-it"

tokenizer = AutoTokenizer.frompretrained(modelid)

model = AutoModelForCausalLM.frompretrained(

modelid,

devicemap="auto",

torchdtype=torch.bfloat16

)

prompt = "What is the difference between supervised and unsupervised learning?"

inputs = tokenizer(prompt, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=256,

temperature=0.7,

topp=0.9,

dosample=True

)

response = tokenizer.decode(outputs[0], skipspecialtokens=True)

print(response)

Chat Template Format

Gemma instruction-tuned models (the -it variants) use a specific chat format. Use the built-in chat template for optimal results:

messages = [

{"role": "user", "content": "Write a Python function to calculate fibonacci numbers"},

]

inputtext = tokenizer.applychattemplate(

messages,

tokenize=False,

addgenerationprompt=True

)

inputs = tokenizer(inputtext, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=512,

temperature=0.3

)

response = tokenizer.decode(outputs[0][inputs["inputids"].shape[-1]:], skipspecialtokens=True)

print(response)

Multi-turn Conversation

conversation = [

{"role": "user", "content": "What is a neural network?"},

{"role": "assistant", "content": "A neural network is a computational model inspired by how the human brain works. It consists of layers of artificial neurons that are interconnected and can learn patterns from data."},

{"role": "user", "content": "How do you train one?"},

]

inputtext = tokenizer.applychattemplate(

conversation,

tokenize=False,

addgenerationprompt=True

)

inputs = tokenizer(inputtext, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(inputs, maxnewtokens=512, temperature=0.7)

response = tokenizer.decode(outputs[0][inputs["inputids"].shape[-1]:], skipspecialtokens=True)

print(response)

Quantization: Running Gemma on Limited Hardware

4-bit Quantization with BitsAndBytes

Quantization allows you to run large models on GPUs with limited VRAM:

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

import torch

quantizationconfig = BitsAndBytesConfig(

loadin4bit=True,

bnb4bitcomputedtype=torch.bfloat16,

bnb4bitusedoublequant=True,

bnb4bitquanttype="nf4"

)

modelid = "google/gemma-2-27b-it"

tokenizer = AutoTokenizer.frompretrained(modelid)

model = AutoModelForCausalLM.frompretrained(

modelid,

quantizationconfig=quantizationconfig,

devicemap="auto"

)

print(f"Model loaded. Memory usage: {model.getmemoryfootprint() / 1e9:.2f} GB")

With 4-bit quantization, Gemma 27B which normally requires around 54GB of VRAM can run with only about 16GB of VRAM.

8-bit Quantization

If you want a balance between quality and memory efficiency:

quantizationconfig = BitsAndBytesConfig(

loadin8bit=True

)

model = AutoModelForCausalLM.frompretrained(

"google/gemma-2-9b-it",

quantizationconfig=quantizationconfig,

devicemap="auto"

)

VRAM Requirements Comparison

| Model | FP16 | 8-bit | 4-bit |

|-------|------|-------|-------|

| Gemma 2B | ~4 GB | ~2.5 GB | ~1.5 GB |

| Gemma 7B | ~14 GB | ~8 GB | ~5 GB |

| Gemma 2 9B | ~18 GB | ~10 GB | ~6 GB |

| Gemma 2 27B | ~54 GB | ~28 GB | ~16 GB |

Using Gemma with Ollama

The easiest way to run Gemma locally:

# Install Ollama (Linux)

curl -fsSL https://ollama.com/install.sh | sh

Download and run Gemma

ollama run gemma2:9b

Or a smaller version

ollama run gemma2:2b

Accessing Ollama from Python

import requests

import json

def chatwithgemma(prompt, model="gemma2:9b"):

response = requests.post(

"http://localhost:11434/api/generate",

json={

"model": model,

"prompt": prompt,

"stream": False,

"options": {

"temperature": 0.7,

"topp": 0.9,

"numpredict": 512

}

}

)

return response.json()["response"]

result = chatwithgemma("Explain how transformer architecture works")

print(result)

Ollama with OpenAI-compatible API

from openai import OpenAI

client = OpenAI(

baseurl="http://localhost:11434/v1",

apikey="ollama"

)

response = client.chat.completions.create(

model="gemma2:9b",

messages=[

{"role": "user", "content": "Write a SQL query to calculate average monthly sales"}

],

temperature=0.3

)

print(response.choices[0].message.content)

Advanced Usage: Fine-tuning Gemma with LoRA

Fine-tuning allows you to customize Gemma for specific tasks. Using LoRA (Low-Rank Adaptation) makes this process efficient and affordable.

Additional Dependencies

pip install peft trl datasets

Preparing the Dataset

from datasets import Dataset

data = {

"instruction": [

"Classify the sentiment of the following review",

"Classify the sentiment of the following review",

"Classify the sentiment of the following review",

"Extract entities from the following text",

],

"input": [

"This product is amazing, fast shipping!",

"Poor quality, doesn't match the description.",

"It's okay, but could be better.",

"John bought a laptop from Amazon yesterday.",

],

"output": [

"Positive",

"Negative",

"Neutral",

"Entities: John (Person), Amazon (Organization), yesterday (Time)",

]

}

dataset = Dataset.fromdict(data)

def formatinstruction(example):

if example["input"]:

text = f"""ofturn>user

{example['instruction']}

{example['input']}ofturn>

ofturn>model

{example['output']}ofturn>"""

else:

text = f"""ofturn>user

{example['instruction']}ofturn>

ofturn>model

{example['output']}ofturn>"""

return {"text": text}

dataset = dataset.map(formatinstruction)

print(dataset[0]["text"])

LoRA Configuration and Training

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

from peft import LoraConfig, getpeftmodel, preparemodelforkbittraining

from trl import SFTTrainer, SFTConfig

import torch

modelid = "google/gemma-2-9b-it"

bnbconfig = BitsAndBytesConfig(

loadin4bit=True,

bnb4bitcomputedtype=torch.bfloat16,

bnb4bitusedoublequant=True,

bnb4bitquanttype="nf4"

)

tokenizer = AutoTokenizer.frompretrained(modelid)

tokenizer.padtoken = tokenizer.eostoken

model = AutoModelForCausalLM.frompretrained(

modelid,

quantizationconfig=bnbconfig,

devicemap="auto"

)

model = preparemodelforkbittraining(model)

loraconfig = LoraConfig(

r=16,

loraalpha=32,

targetmodules=["qproj", "kproj", "vproj", "oproj"],

loradropout=0.05,

bias="none",

tasktype="CAUSALLM"

)

model = getpeftmodel(model, loraconfig)

model.printtrainableparameters()

trainingconfig = SFTConfig(

outputdir="./gemma-finetuned",

numtrainepochs=3,

perdevicetrainbatchsize=4,

gradientaccumulationsteps=4,

learningrate=2e-4,

weightdecay=0.01,

warmupratio=0.1,

loggingsteps=10,

savestrategy="epoch",

bf16=True,

maxseqlength=512,

datasettextfield="text",

)

trainer = SFTTrainer(

model=model,

traindataset=dataset,

args=trainingconfig,

tokenizer=tokenizer,

)

trainer.train()

trainer.savemodel("./gemma-finetuned-final")

tokenizer.savepretrained("./gemma-finetuned-final")

Using the Fine-tuned Model

from peft import PeftModel

from transformers import AutoModelForCausalLM, AutoTokenizer

import torch

basemodel = AutoModelForCausalLM.frompretrained(

"google/gemma-2-9b-it",

devicemap="auto",

torchdtype=torch.bfloat16

)

model = PeftModel.frompretrained(basemodel, "./gemma-finetuned-final")

tokenizer = AutoTokenizer.frompretrained("./gemma-finetuned-final")

model = model.mergeandunload()

prompt = """ofturn>user

Classify the sentiment of the following review

The customer service at this store was exceptional, I'll definitely come back!ofturn>

ofturn>model

"""

inputs = tokenizer(prompt, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(inputs, maxnewtokens=50, temperature=0.1)

response = tokenizer.decode(outputs[0][inputs["inputids"].shape[-1]:], skipspecialtokens=True)

print(response)

Using Gemma for Specific Tasks

Text Summarization

def summarize(text, maxlength=256):

messages = [

{

"role": "user",

"content": f"Provide a concise summary of the following text:\n\n{text}"

}

]

inputtext = tokenizer.applychattemplate(

messages, tokenize=False, addgenerationprompt=True

)

inputs = tokenizer(inputtext, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=maxlength,

temperature=0.3,

repetitionpenalty=1.1

)

return tokenizer.decode(

outputs[0][inputs["inputids"].shape[-1]:],

skipspecialtokens=True

)

article = """

The World Health Organization reported a significant breakthrough in malaria

vaccine development. The new vaccine, which completed phase 3 trials across

15 African countries, showed 78% efficacy in preventing severe malaria cases

in children under 5. Health officials project that widespread deployment could

save an estimated 500,000 lives annually.

"""

summary = summarize(article)

print(summary)

Code Generation

def generatecode(taskdescription):

messages = [

{

"role": "user",

"content": f"""Write Python code for: {taskdescription}

Provide clean code with type hints and usage examples."""

}

]

inputtext = tokenizer.applychattemplate(

messages, tokenize=False, addgenerationprompt=True

)

inputs = tokenizer(inputtext, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=1024,

temperature=0.2,

topp=0.95

)

return tokenizer.decode(

outputs[0][inputs["inputids"].shape[-1]:],

skipspecialtokens=True

)

code = generatecode("A REST API endpoint for CRUD operations using FastAPI and SQLAlchemy")

print(code)

Structured Data Extraction

import json

def extractentities(text):

messages = [

{

"role": "user",

"content": f"""Extract entities from the following text in JSON format.

Entity categories: person, organization, location, date, amount.

Text: {text}

Provide output only in JSON format like this:

{{"entities": [{{"text": "...", "type": "..."}}]}}"""

}

]

inputtext = tokenizer.applychattemplate(

messages, tokenize=False, addgenerationprompt=True

)

inputs = tokenizer(inputtext, returntensors="pt").to(model.device)

with torch.nograd():

outputs = model.generate(

inputs,

maxnewtokens=512,

temperature=0.1

)

response = tokenizer.decode(

outputs[0][inputs["inputids"].shape[-1]:],

skipspecialtokens=True

)

try:

return json.loads(response)

except json.JSONDecodeError:

return {"rawresponse": response}

text = "Apple Inc. reported $94.8 billion in revenue for Q3 2026 at their headquarters in Cupertino, California."

entities = extractentities(text)

print(json.dumps(entities, indent=2))

Serving Gemma with vLLM

For production deployment, vLLM offers extremely fast inference:

pip install vllm

Running vLLM Server

python -m vllm.entrypoints.openai.apiserver \

--model google/gemma-2-9b-it \

--dtype bfloat16 \

--max-model-len 4096 \

--gpu-memory-utilization 0.9 \

--port 8000

Client for vLLM Server

from openai import OpenAI

client = OpenAI(

baseurl="http://localhost:8000/v1",

apikey="not-needed"

)

response = client.chat.completions.create(

model="google/gemma-2-9b-it",

messages=[

{"role": "user", "content": "Design a microservices architecture for an e-commerce platform"}

],

temperature=0.7,

maxtokens=1024

)

print(response.choices[0].message.content)

Batch Inference with vLLM

from vllm import LLM, SamplingParams

llm = LLM(

model="google/gemma-2-9b-it",

dtype="bfloat16",

maxmodellen=4096

)

samplingparams = SamplingParams(

temperature=0.7,

topp=0.9,

maxtokens=256

)

prompts = [

"What is containerization?",

"Explain the difference between SQL and NoSQL",

"How do you optimize database queries?",

]

outputs = llm.generate(prompts, samplingparams)

for output in outputs:

print(f"Prompt: {output.prompt[:50]}...")

print(f"Response: {output.outputs[0].text}\n")

Best Practices

1. Choose the Right Model Size

  • Gemma 2B: Suitable for simple tasks, classification, or edge device deployment
  • Gemma 7B/9B: Good balance between performance and efficiency, suitable for most use cases
  • Gemma 27B: For complex tasks requiring deep reasoning

2. Optimize Your Prompts

# Less optimal

prompt = "tell me about python"

More optimal - specific and structured

prompt = """Explain the top 3 advantages of Python for data science.

For each advantage, provide an example of a relevant library.

Format: bullet points."""

3. Temperature Settings

  • 0.0 - 0.3: For tasks requiring consistency (classification, extraction, code generation)
  • 0.5 - 0.7: For creative tasks with constraints (summarization, rewriting)
  • 0.8 - 1.0: For free creative tasks (brainstorming, creative writing)

4. Use System Prompts Effectively

messages = [

{

"role": "user",

"content": """You are a data analyst assistant expert in SQL and Python.

Answer the following question with code that can be run directly.

Question: How to calculate customer churn rate from a transactions table?"""

}

]

5. Limit Output with maxnewtokens

Always set maxnewtokens according to your needs to avoid overly long outputs and save inference time.

6. Use Repetition Penalty

To avoid unnecessary text repetition:

outputs = model.generate(

inputs,

maxnewtokens=512,

temperature=0.7,

repetitionpenalty=1.15

)

7. Memory Management

import gc

import torch

After you're done with the model

del model

del tokenizer

gc.collect()

torch.cuda.emptycache()

Gemma Variants Comparison

| Feature | Gemma 1 | Gemma 2 | CodeGemma | PaliGemma |

|---------|---------|---------|-----------|-----------|

| Sizes | 2B, 7B | 2B, 9B, 27B | 2B, 7B | 3B |

| Focus | General | General (improved) | Code | Vision + Language |

| Context Length | 8K | 8K | 8K | Varies |

| Use Case | Chat, text gen | Chat, reasoning | Coding, debugging | Image understanding |

Troubleshooting

OutOfMemoryError

If you get a CUDA out of memory error:

# Use quantization

Or reduce maxseqlength

Or use a smaller model

Check memory usage

print(f"Allocated: {torch.cuda.memoryallocated() / 1e9:.2f} GB")

print(f"Reserved: {torch.cuda.memoryreserved() / 1e9:.2f} GB")

Slow Generation

# Use torch.compile to speed up inference

model = torch.compile(model)

Or use Flash Attention

model = AutoModelForCausalLM.frompretrained(

modelid,

attnimplementation="flashattention2",

torchdtype=torch.bfloat16,

devicemap="auto"

)

Conclusion

Google Gemma is an excellent choice for anyone looking to use high-quality open-source language models. With various size variants and broad ecosystem support, Gemma can be used from laptop experiments to large-scale production deployments.

Key takeaways:

  • Choose the model variant based on your needs and available resources
  • Use quantization to run large models on limited hardware
  • Leverage chat templates for optimal results with instruction-tuned models
  • Fine-tuning with LoRA is efficient and affordable
  • For production, consider vLLM or Ollama for optimal serving

With mastery of Gemma, you have a strong foundation for building powerful and efficient AI applications.

Related Articles

Complete Guide to Hugging Face AutoTrain: Fine-Tune AI Models Without the Hassle

Tutorial Lengkap Hugging Face AutoTrain: Fine-Tuning Model AI Tanpa Ribet Hugging Face AutoTrain adalah platform dan lib...

Complete Tutorial: Fine-tuning LLMs with LoRA and PEFT

Tutorial Lengkap Fine-tuning LLM dengan LoRA dan PEFT Fine-tuning Large Language Models (LLM) secara tradisional membutu...

MergeKit Tutorial: Merging LLM Models to Create Custom AI Models

Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom Model bahasa besar (LLM) open-source terus berk...

LitGPT Tutorial: A Framework for Training and Fine-Tuning Large Language Models

Tutorial LitGPT: Framework untuk Melatih dan Fine-Tuning Large Language Models LitGPT adalah framework open-source dari ...