Modal: Serverless GPU Cloud untuk Deploy Model ML

Salah satu tantangan terbesar dalam machine learning bukan membuat model, melainkan men-deploy-nya ke production. Anda perlu mengelola server, mengkonfigurasi GPU driver, mengatur auto-scaling, dan banyak lagi. Modal hadir untuk menghilangkan semua kompleksitas tersebut.

Modal adalah platform serverless cloud yang memungkinkan Anda menjalankan kode Python di cloud dengan akses GPU, tanpa perlu mengelola infrastruktur sama sekali. Cukup tulis fungsi Python biasa, tambahkan decorator, dan Modal akan mengurus sisanya.

Mengapa Modal?

Perbandingan dengan pendekatan tradisional:

• Setup server EC2/GCE dengan GPU
• Install CUDA, cuDNN, driver
• Konfigurasi Docker, Kubernetes
• Setup load balancer, auto-scaling
• Monitoring, logging, alerting
• Total waktu setup: berhari-hari hingga berminggu-minggu

• Tulis fungsi Python
• Tambahkan decorator @app.function(gpu="A100")
• Deploy dengan modal deploy
• Total waktu: hitungan menit

Modal menawarkan pricing pay-per-second, artinya Anda hanya membayar saat kode benar-benar berjalan. Tidak ada biaya idle.

Instalasi dan Autentikasi

Install Modal

pip install modal

Setup Autentikasi

modal setup

Perintah ini akan membuka browser untuk login dan mengkonfigurasi token API secara otomatis. Setelah selesai, Anda bisa langsung menggunakan Modal dari terminal.

Verifikasi instalasi:

modal --version

Basic Functions dengan @app.function

Konsep dasar Modal sangat sederhana: Anda mendefinisikan fungsi Python biasa dan menandainya dengan decorator agar berjalan di cloud.

import modal

app = modal.App("hello-modal")

@app.function()
def hello(name: str) -> str:
    return f"Hello, {name}! This is running on Modal cloud."

@app.localentrypoint()

def main():
    # Memanggil fungsi yang berjalan di cloud
    result = hello.remote("World")
    print(result)

modal run helloapp.py

Fungsi hello akan dieksekusi di cloud Modal, bukan di mesin lokal Anda. Decorator @app.localentrypoint() menandai fungsi yang berjalan secara lokal sebagai entry point.

import modal

app = modal.App("gpu-example")

Menggunakan NVIDIA T4 (budget-friendly, inferensi)
@app.function(gpu="T4")
def inferencet4():
    import torch
    print(f"GPU available: {torch.cuda.isavailable()}")

    print(f"GPU name: {torch.cuda.getdevicename(0)}")

    return "T4 inference complete"

Menggunakan NVIDIA A10G (balance performa/harga)
@app.function(gpu="A10G")
def traina10g():
    import torch
    print(f"GPU: {torch.cuda.getdevicename(0)}")
    print(f"Memory: {torch.cuda.getdeviceproperties(0).totalmem / 1e9:.1f} GB")

    return "A10G training complete"

Menggunakan NVIDIA A100 (high-performance training)
@app.function(gpu="A100")
def traina100():
    import torch
    print(f"GPU: {torch.cuda.getdevicename(0)}")
    return "A100 training complete"

Menggunakan NVIDIA H100 (cutting-edge, LLM training)
@app.function(gpu="H100")
def trainh100():

    import torch
    print(f"GPU: {torch.cuda.getdevicename(0)}")

    return "H100 training complete"

Multiple GPU
@app.function(gpu=modal.gpu.A100(count=2))
def multigputraining():

    import torch
    print(f"Number of GPUs: {torch.cuda.devicecount()}")
    return "Multi-GPU training complete"

Pilih GPU berdasarkan kebutuhan:

• T4: Inferensi, model kecil-menengah, paling hemat biaya
• A10G: Balance antara performa dan biaya, fine-tuning
• A100: Training model besar, fine-tuning LLM
• H100: Performa tertinggi, training LLM skala besar

Container Images (modal.Image)

Modal menggunakan container untuk menjalankan kode Anda. Anda bisa mengkustomisasi environment dengan modal.Image:

import modal

Image dasar dengan pip packages
image = modal.Image.debianslim(pythonversion="3.11").pipinstall(

    "torch",
    "transformers",
    "accelerate",
    "scipy"
)

app = modal.App("custom-image", image=image)

@app.function(gpu="T4")
def runmodel():
    from transformers import pipeline
    classifier = pipeline("sentiment-analysis")
    result = classifier("Modal is amazing for ML deployment!")
    return result

Image dengan Conda

image = modal.Image.fromregistry(
    "continuumio/miniconda3"
).condainstall(
    "pytorch",
    "torchvision",
    channels=["pytorch", "nvidia"]
).pipinstall(

    "transformers"
)

image = modal.Image.fromdockerfile("./Dockerfile")

Image dengan Pre-built Model

def downloadmodel():
    from transformers import AutoModelForCausalLM, AutoTokenizer
    modelname = "microsoft/DialoGPT-medium"
    AutoTokenizer.frompretrained(modelname)
    AutoModelForCausalLM.frompretrained(modelname)

image = (
    modal.Image.debianslim(pythonversion="3.11")
    .pipinstall("torch", "transformers")

    .runfunction(downloadmodel)

)

Dengan .runfunction(), model di-download saat build image, bukan saat runtime. Ini membuat cold start jauh lebih cepat.

Volumes dan Storage

Modal menyediakan beberapa opsi penyimpanan untuk data persisten:

Modal Volume (Persistent Storage)

import modal

app = modal.App("volume-example")
volume = modal.Volume.fromname("my-data-volume", createifmissing=True)


@app.function(volumes={"/data": volume})
def savedata():
    # Menulis data ke volume
    with open("/data/results.txt", "w") as f:
        f.write("Training results: accuracy=0.95")
    volume.commit()  # Pastikan data tersimpan
    return "Data saved"

@app.function(volumes={"/data": volume})
def loaddata():

    volume.reload()  # Refresh untuk melihat perubahan terbaru
    with open("/data/results.txt", "r") as f:
        content = f.read()
    return content

nfs = modal.NetworkFileSystem.fromname("my-nfs", createifmissing=True)

@app.function(networkfilesystems={"/shared": nfs})
def sharedstorage():

    # NFS cocok untuk sharing data antar fungsi yang berjalan bersamaan
    with open("/shared/sharedconfig.json", "w") as f:
        import json
        json.dump({"batchsize": 32, "lr": 0.001}, f)

import modal

app = modal.App("web-api")

@app.function()
@modal.webendpoint(method="GET")
def health():
    return {"status": "healthy", "service": "ml-api"}

@app.function()
@modal.webendpoint(method="POST")

def predict(data: dict):
    text = data.get("text", "")
    # Lakukan prediksi
    return {"prediction": "positive", "confidence": 0.95, "text": text}

import modal
from fastapi import FastAPI

webapp = FastAPI()
app = modal.App("fastapi-modal")

image = modal.Image.debianslim().pipinstall("fastapi")

@webapp.get("/")

async def root():
    return {"message": "Hello from Modal + FastAPI"}

@webapp.post("/predict")
async def predict(data: dict):
    return {"result": "prediction here"}

@app.function(image=image)
@modal.asgiapp()

def fastapiapp():
    return webapp

modal deploy webapp.py
Output: https://your-app--fastapi-app.modal.run

Scheduled Jobs (Cron)

Modal mendukung penjadwalan fungsi seperti cron job:

import modal
from datetime import datetime

app = modal.App("scheduled-jobs")

@app.function(schedule=modal.Period(hours=1))
def hourlydatarefresh():
    """Dijalankan setiap jam untuk refresh data."""
    print(f"Refreshing data at {datetime.now()}")
    # Lakukan ETL, retrain model, dll.

@app.function(schedule=modal.Cron("0 6   "))

def dailymodelevaluation():
    """Dijalankan setiap hari jam 6 pagi UTC."""
    print(f"Running daily model evaluation at {datetime.now()}")
    # Evaluasi performa model
    # Kirim alert jika ada degradasi

@app.function(schedule=modal.Cron("0 0   0"))

def weeklyretrain():

    """Dijalankan setiap minggu pada hari Minggu."""
    print(f"Starting weekly model retrain at {datetime.now()}")

# Set secret via CLI
modal secret create my-api-keys \
  OPENAIAPIKEY=sk-xxx \

  HFTOKEN=hfxxx

import modal

app = modal.App("secrets-example")

@app.function(secrets=[modal.Secret.fromname("my-api-keys")])
def callopenai():

    import os
    import openai

    # Secret tersedia sebagai environment variable
    client = openai.OpenAI(apikey=os.environ["OPENAIAPIKEY"])

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello!"}]
    )
    return response.choices[0].message.content

Secrets dari environment variable lokal
@app.function(secrets=[modal.Secret.fromdotenv()])

def useenvsecrets():

    import os
    token = os.environ.get("HFTOKEN")
    return f"Token loaded: {token[:10]}..."

import modal

app = modal.App("parallel-processing")

image = modal.Image.debianslim().pipinstall("Pillow", "torch", "torchvision")

@app.function(gpu="T4", image=image)
def processimage(imageurl: str) -> dict:
    """Proses satu gambar - berjalan di GPU."""
    # Simulasi image processing
    import time
    time.sleep(2)  # Simulasi waktu processing
    return {"url": imageurl, "status": "processed"}


@app.localentrypoint()
def main():
    # Proses 100 gambar secara paralel
    imageurls = [f"https://example.com/image{i}.jpg" for i in range(100)]

    # .map() menjalankan fungsi secara paralel di multiple containers
    results = list(processimage.map(imageurls))

    print(f"Processed {len(results)} images")
    for r in results[:5]:
        print(r)

@app.function(gpu="T4")
def trainmodel(learningrate: float, batchsize: int, epochs: int) -> dict:

    """Train model dengan hyperparameter tertentu."""
    # Simulasi training
    accuracy = 0.85 + learningrate  10 - batchsize  0.001

    return {
        "lr": learningrate,
        "batchsize": batchsize,
        "epochs": epochs,
        "accuracy": round(accuracy, 4)
    }

@app.localentrypoint()

def hyperparametersearch():
    # Grid search secara paralel
    configs = [
        (0.001, 16, 10),
        (0.001, 32, 10),
        (0.01, 16, 10),
        (0.01, 32, 10),
        (0.1, 16, 10),
        (0.1, 32, 10),
    ]

    results = list(trainmodel.starmap(configs))


    # Temukan konfigurasi terbaik
    best = max(results, key=lambda x: x["accuracy"])
    print(f"Best config: {best}")

import modal

app = modal.App("model-serving")

Definisi image dengan semua dependensi
image = modal.Image.debianslim(pythonversion="3.11").pipinstall(
    "torch",
    "transformers",
    "accelerate"
)

@app.cls(gpu="T4", image=image, containeridletimeout=300)
class SentimentAnalyzer:
    @modal.enter()
    def loadmodel(self):

        """Load model saat container start (hanya sekali)."""
        from transformers import pipeline
        self.classifier = pipeline(
            "sentiment-analysis",
            model="distilbert-base-uncased-finetuned-sst-2-english",
            device=0  # Gunakan GPU
        )
        print("Model loaded successfully!")

    @modal.method()
    def predict(self, text: str) -> dict:
        """Prediksi sentimen teks."""
        result = self.classifier(text)[0]
        return {
            "text": text,
            "label": result["label"],
            "score": round(result["score"], 4)
        }

    @modal.method()
    def predictbatch(self, texts: list) -> list:
        """Prediksi sentimen batch teks."""
        results = self.classifier(texts)
        return [
            {
                "text": text,
                "label": r["label"],
                "score": round(r["score"], 4)
            }
            for text, r in zip(texts, results)
        ]

@app.function()
@modal.webendpoint(method="POST")

def analyze(data: dict):
    analyzer = SentimentAnalyzer()
    text = data.get("text", "")

    if isinstance(text, list):
        return {"results": analyzer.predictbatch.remote(text)}
    else:
        return {"result": analyzer.predict.remote(text)}

import modal
from typing import Optional

Konfigurasi
MODELNAME = "mistralai/Mistral-7B-Instruct-v0.2"

GPUCONFIG = modal.gpu.A100(count=1, size="40GB")

Image dengan model pre-downloaded
def downloadmodel():

    from transformers import AutoModelForCausalLM, AutoTokenizer
    import torch

    print(f"Downloading {MODELNAME}...")
    tokenizer = AutoTokenizer.frompretrained(MODELNAME)
    model = AutoModelForCausalLM.frompretrained(

        MODELNAME,
        torchdtype=torch.float16,

        devicemap="auto"
    )
    print("Model downloaded successfully!")

image = (
    modal.Image.debianslim(pythonversion="3.11")
    .pipinstall(

        "torch",
        "transformers",
        "accelerate",
        "scipy",
        "sentencepiece",
        "protobuf"
    )
    .runfunction(
        downloadmodel,

        secrets=[modal.Secret.fromname("huggingface-token")],
        gpu=GPUCONFIG

    )
)

app = modal.App("hf-model-api", image=image)

@app.cls(
    gpu=GPUCONFIG,
    containeridletimeout=300,
    allowconcurrentinputs=10,
    secrets=[modal.Secret.fromname("huggingface-token")]

)
class LLMInference:
    @modal.enter()
    def load(self):
        """Load model ke GPU saat container dimulai."""
        from transformers import AutoModelForCausalLM, AutoTokenizer
        import torch

        self.tokenizer = AutoTokenizer.frompretrained(MODELNAME)

        self.model = AutoModelForCausalLM.frompretrained(
            MODELNAME,

            torchdtype=torch.float16,
            devicemap="auto"

        )

        # Set pad token
        if self.tokenizer.padtoken is None:
            self.tokenizer.padtoken = self.tokenizer.eostoken

        print("Model loaded and ready!")

    @modal.method()
    def generate(
        self,
        prompt: str,
        maxnewtokens: int = 512,
        temperature: float = 0.7,
        topp: float = 0.9

    ) -> str:
        """Generate teks dari prompt."""
        import torch

        # Format prompt untuk instruction model
        messages = [{"role": "user", "content": prompt}]
        formatted = self.tokenizer.applychattemplate(

            messages,
            tokenize=False,
            addgenerationprompt=True

        )

        inputs = self.tokenizer(formatted, returntensors="pt").to("cuda")

        with torch.nograd():

            outputs = self.model.generate(
                inputs,

                maxnewtokens=maxnewtokens,

                temperature=temperature,
                topp=topp,

                dosample=True,
                padtokenid=self.tokenizer.padtokenid
            )

        # Decode hanya token yang di-generate (bukan prompt)
        newtokens = outputs[0][inputs["inputids"].shape[1]:]
        response = self.tokenizer.decode(newtokens, skipspecialtokens=True)


        return response.strip()

    @modal.method()
    def healthcheck(self) -> dict:
        """Check apakah model sudah loaded."""
        import torch
        return {
            "status": "healthy",
            "model": MODELNAME,

            "gpu": torch.cuda.getdevicename(0),

            "gpumemoryallocated": f"{torch.cuda.memoryallocated() / 1e9:.2f} GB"
        }

Web endpoints
@app.function()
@modal.webendpoint(method="POST")

def generatetext(data: dict):
    """API endpoint untuk text generation."""
    llm = LLMInference()

    prompt = data.get("prompt", "")
    maxtokens = data.get("maxtokens", 512)
    temperature = data.get("temperature", 0.7)

    if not prompt:
        return {"error": "prompt is required"}, 400

    response = llm.generate.remote(
        prompt=prompt,
        maxnewtokens=maxtokens,

        temperature=temperature
    )

    return {
        "model": MODELNAME,
        "prompt": prompt,
        "response": response,
        "parameters": {
            "maxtokens": maxtokens,
            "temperature": temperature
        }
    }

@app.function()
@modal.webendpoint(method="GET")

def health():
    """Health check endpoint."""
    llm = LLMInference()
    return llm.healthcheck.remote()

Local testing
@app.localentrypoint()

def main():
    llm = LLMInference()

    # Test health check
    status = llm.healthcheck.remote()
    print(f"Status: {status}")

    # Test generation
    prompts = [
        "Explain machine learning in simple terms.",
        "Write a Python function to calculate fibonacci numbers.",
        "What are the benefits of serverless computing?"
    ]

    for prompt in prompts:
        print(f"\nPrompt: {prompt}")
        response = llm.generate.remote(prompt)
        print(f"Response: {response[:200]}...")

# Test secara lokal (berjalan di cloud Modal)
modal run hfmodelapi.py

Deploy sebagai service
modal deploy hfmodelapi.py

https://your-workspace--hf-model-api-generate-text.modal.run

# Health check
curl https://your-workspace--hf-model-api-health.modal.run

Generate text
curl -X POST https://your-workspace--hf-model-api-generate-text.modal.run \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Explain quantum computing", "maxtokens": 256}'

# Pre-download model di image build, bukan di runtime
image = (
    modal.Image.debianslim()
    .pipinstall("transformers", "torch")

    .runfunction(downloadmodel)  # Model sudah ada di image

)

Gunakan containeridletimeout untuk menjaga container tetap warm

@app.function(containeridletimeout=600)  # 10 menit idle sebelum shutdown

def myfunction():
    pass

# Gunakan timeout untuk mencegah biaya tak terduga
@app.function(gpu="A100", timeout=3600)  # Maksimal 1 jam
def longtraining():

    pass

Gunakan T4 untuk inferensi (lebih murah)
@app.function(gpu="T4")
def inference():
    pass

Gunakan A100/H100 hanya untuk training
@app.function(gpu="A100")
def training():
    pass

# Jalankan dengan mode debug
modal run --detach myapp.py

Lihat logs
modal app logs my-app

Shell interaktif di container
modal shell myapp.py

# Izinkan multiple request ke satu container
@app.cls(
    gpu="T4",
    allowconcurrentinputs=20,  # Hingga 20 request bersamaan

    containeridletimeout=300

)
class MyModel:
    @modal.enter()
    def setup(self):
        self.model = loadmodel()

    @modal.method()
    def predict(self, data):
        return self.model(data)

• Zero infrastructure management: Tidak perlu setup server, Docker, atau Kubernetes
• Pay-per-second billing: Bayar hanya saat kode berjalan
• Akses GPU mudah: Dari T4 hingga H100, tinggal pilih
• Auto-scaling: Dari 0 hingga ratusan container secara otomatis
• Developer experience*: API Python yang intuitif dan modern

Mulai dengan Modal untuk proyek ML Anda berikutnya dan rasakan perbedaannya. Fokus pada membangun model, bukan mengelola infrastruktur.