Tutorial Lengkap Azure ML Managed Endpoints: Deployment Model Production

Azure ML Managed Endpoints menyediakan solusi fully managed untuk deploy model machine learning secara scalable. Endpoints menangani infrastruktur, scaling, security, dan monitoring secara otomatis.

Mengapa Managed Endpoints?

Manfaat Utama:

• Fully managed: Tidak perlu mengelola infrastruktur
• Auto-scaling: Scale berdasarkan traffic
• Blue-green deployments: Rollout yang aman
• Built-in monitoring: Metrics dan logging
• Security: Authentication dan network isolation

Tipe Endpoint:

• Online endpoints: Real-time inference
• Batch endpoints: Batch processing skala besar

Prerequisites

pip install azure-ai-ml azure-identity

Azure CLI
az login
az extension add -n ml

Online Endpoints

1. Buat Online Endpoint

from azure.ai.ml import MLClient
from azure.ai.ml.entities import ManagedOnlineEndpoint
from azure.identity import DefaultAzureCredential

mlclient = MLClient(

    credential=DefaultAzureCredential(),
    subscriptionid="your-subscription-id",
    resourcegroupname="my-resource-group",
    workspacename="my-ml-workspace"

)

Buat endpoint
endpoint = ManagedOnlineEndpoint(
    name="my-online-endpoint",
    description="Online endpoint untuk real-time inference",
    authmode="key",  # atau "amltoken"

    tags={"environment": "production"}
)

mlclient.onlineendpoints.begincreateorupdate(endpoint).result()
print(f"Endpoint dibuat: {endpoint.name}")

2. Buat Deployment

from azure.ai.ml.entities import (
    ManagedOnlineDeployment,
    Model,
    Environment,
    CodeConfiguration
)

Buat deployment
bluedeployment = ManagedOnlineDeployment(

    name="blue",
    endpointname="my-online-endpoint",
    model=Model(path="./model"),
    codeconfiguration=CodeConfiguration(

        code="./scoring",
        scoringscript="score.py"
    ),
    environment=Environment(
        condafile="./environment.yml",

        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest"
    ),
    instancetype="StandardDS3v2",
    instancecount=1

)

mlclient.onlinedeployments.begincreateorupdate(bluedeployment).result()

print("Deployment dibuat")

3. Scoring Script

# scoring/score.py
import json
import joblib
import numpy as np
import os
import logging

def init():
    """Inisialisasi model saat startup."""
    global model
    modelpath = os.path.join(os.getenv("AZUREMLMODELDIR"), "model.joblib")
    model = joblib.load(modelpath)

    logging.info("Model berhasil dimuat")

def run(rawdata):
    """Jalankan inference pada data masuk."""
    try:
        data = json.loads(rawdata)

        features = np.array(data["features"])

        # Jalankan prediksi
        predictions = model.predict(features)
        probabilities = model.predictproba(features)

        return {
            "predictions": predictions.tolist(),
            "probabilities": probabilities.tolist()
        }

    except Exception as e:
        logging.error(f"Error: {str(e)}")
        return {"error": str(e)}

4. Set Traffic

# Arahkan semua traffic ke deployment blue
endpoint.traffic = {"blue": 100}
mlclient.onlineendpoints.begincreateorupdate(endpoint).result()


Dapatkan detail endpoint
endpoint = mlclient.onlineendpoints.get("my-online-endpoint")

print(f"Scoring URI: {endpoint.scoringuri}")
print(f"Traffic: {endpoint.traffic}")

5. Test Endpoint

import json

Test data
testdata = {

    "features": [[5.1, 3.5, 1.4, 0.2], [6.2, 3.4, 5.4, 2.3]]
}

Panggil endpoint
response = mlclient.onlineendpoints.invoke(

    endpointname="my-online-endpoint",
    requestfile=json.dumps(testdata)
)

print(f"Response: {response}")

6. Invoke dengan REST API

import requests

Dapatkan detail endpoint
endpoint = mlclient.onlineendpoints.get("my-online-endpoint")
scoringuri = endpoint.scoringuri

Dapatkan key
keys = mlclient.onlineendpoints.getkeys("my-online-endpoint")

apikey = keys.primarykey


Buat request
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {apikey}"
}

data = {"features": [[5.1, 3.5, 1.4, 0.2]]}

response = requests.post(scoringuri, headers=headers, json=data)

print(response.json())

Blue-Green Deployments

1. Buat Green Deployment

# Buat deployment baru dengan model terupdate
greendeployment = ManagedOnlineDeployment(
    name="green",
    endpointname="my-online-endpoint",

    model=Model(path="./modelv2"),
    codeconfiguration=CodeConfiguration(

        code="./scoring",
        scoringscript="score.py"
    ),
    environment=Environment(
        condafile="./environment.yml",

        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest"
    ),
    instancetype="StandardDS3v2",
    instancecount=1

)

mlclient.onlinedeployments.begincreateorupdate(greendeployment).result()

2. Traffic Shift Bertahap

# Mulai dengan 10% traffic ke green
endpoint.traffic = {"blue": 90, "green": 10}
mlclient.onlineendpoints.begincreateorupdate(endpoint).result()

Monitor metrics, lalu tingkatkan
endpoint.traffic = {"blue": 50, "green": 50}
mlclient.onlineendpoints.begincreateorupdate(endpoint).result()


Full rollout ke green
endpoint.traffic = {"blue": 0, "green": 100}
mlclient.onlineendpoints.begincreateorupdate(endpoint).result()

3. Rollback

# Rollback ke blue jika ada masalah
endpoint.traffic = {"blue": 100, "green": 0}
mlclient.onlineendpoints.begincreateorupdate(endpoint).result()


Hapus deployment yang gagal
mlclient.onlinedeployments.begindelete(
    endpointname="my-online-endpoint",

    deploymentname="green"
).result()

Auto-Scaling

1. Konfigurasi Auto-Scaling

from azure.ai.ml.entities import (
    ManagedOnlineDeployment,
    OnlineRequestSettings,
    ProbeSettings
)

Deployment dengan konfigurasi scaling
scaleddeployment = ManagedOnlineDeployment(

    name="scaled",
    endpointname="my-online-endpoint",
    model=Model(path="./model"),
    codeconfiguration=CodeConfiguration(

        code="./scoring",
        scoringscript="score.py"
    ),
    environment=Environment(
        condafile="./environment.yml",

        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest"
    ),
    instancetype="StandardDS3v2",
    instancecount=2,

    requestsettings=OnlineRequestSettings(
        requesttimeoutms=90000,
        maxconcurrentrequestsperinstance=10
    ),
    livenessprobe=ProbeSettings(

        initialdelay=10,
        period=10,
        failurethreshold=30

    ),
    readinessprobe=ProbeSettings(
        initialdelay=10,

        period=10,
        failurethreshold=30
    )
)

mlclient.onlinedeployments.begincreateorupdate(scaleddeployment).result()

2. Azure Monitor Auto-Scale

from azure.mgmt.monitor import MonitorManagementClient
from azure.mgmt.monitor.models import (
    AutoscaleSettingResource,
    AutoscaleProfile,
    ScaleRule,
    MetricTrigger,
    ScaleAction,
    ScaleCapacity
)

Konfigurasi auto-scale via Azure Monitor
monitorclient = MonitorManagementClient(

    credential=DefaultAzureCredential(),
    subscriptionid="your-subscription-id"
)

autoscalesetting = AutoscaleSettingResource(

    location="eastus",
    profiles=[
        AutoscaleProfile(
            name="auto-scale-profile",
            capacity=ScaleCapacity(
                minimum="1",
                maximum="10",
                default="2"
            ),
            rules=[
                ScaleRule(
                    metrictrigger=MetricTrigger(
                        metricname="RequestsPerInstance",

                        metricresourceuri="/subscriptions/.../endpoints/my-endpoint",

                        timegrain="PT1M",
                        statistic="Average",
                        timewindow="PT5M",

                        timeaggregation="Average",
                        operator="GreaterThan",
                        threshold=100
                    ),
                    scaleaction=ScaleAction(

                        direction="Increase",
                        type="ChangeCount",
                        value="1",
                        cooldown="PT5M"
                    )
                ),
                ScaleRule(
                    metrictrigger=MetricTrigger(
                        metricname="RequestsPerInstance",

                        operator="LessThan",
                        threshold=20
                    ),
                    scaleaction=ScaleAction(
                        direction="Decrease",
                        type="ChangeCount",
                        value="1",
                        cooldown="PT10M"
                    )
                )
            ]
        )
    ],
    targetresourceuri="/subscriptions/.../deployments/scaled"
)

Batch Endpoints

1. Buat Batch Endpoint

from azure.ai.ml.entities import BatchEndpoint

batchendpoint = BatchEndpoint(

    name="my-batch-endpoint",
    description="Batch endpoint untuk inference skala besar"
)

mlclient.batchendpoints.begincreateorupdate(batchendpoint).result()

print("Batch endpoint dibuat")

2. Buat Batch Deployment

from azure.ai.ml.entities import (
    BatchDeployment,
    BatchRetrySettings,
    CodeConfiguration
)

batchdeployment = BatchDeployment(
    name="batch-scorer",
    endpointname="my-batch-endpoint",

    model=Model(path="./model"),
    codeconfiguration=CodeConfiguration(
        code="./batch-scoring",
        scoringscript="batchscore.py"
    ),
    environment=Environment(
        condafile="./environment.yml",

        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest"
    ),
    compute="cpu-cluster",
    instancecount=2,
    maxconcurrencyperinstance=2,

    minibatchsize=10,

    outputaction="appendrow",

    outputfilename="predictions.csv",

    retrysettings=BatchRetrySettings(
        maxretries=3,

        timeout=300
    ),
    logginglevel="info"
)

mlclient.batchdeployments.begincreateorupdate(batchdeployment).result()

3. Batch Scoring Script

# batch-scoring/batchscore.py
import os
import pandas as pd
import joblib
import logging

def init():
    """Inisialisasi model."""
    global model
    modelpath = os.path.join(os.getenv("AZUREMLMODELDIR"), "model.joblib")
    model = joblib.load(modelpath)

    logging.info("Model dimuat")

def run(minibatch):
    """Proses mini-batch file."""
    results = []

    for filepath in minibatch:
        # Baca data
        df = pd.readcsv(filepath)

        # Prediksi
        predictions = model.predict(df.values)

        # Tambahkan prediksi ke results
        for i, pred in enumerate(predictions):
            results.append({
                "file": os.path.basename(filepath),

                "index": i,
                "prediction": pred
            })

    return pd.DataFrame(results)

4. Invoke Batch Endpoint

from azure.ai.ml import Input
from azure.ai.ml.constants import AssetTypes

Mulai batch job
job = mlclient.batchendpoints.invoke(

    endpointname="my-batch-endpoint",
    inputs=Input(
        path="azureml://datastores/workspaceblobstore/paths/batch-data/",
        type=AssetTypes.URIFOLDER

    )
)

print(f"Batch job dimulai: {job.name}")

Monitor job
mlclient.jobs.stream(job.name)

Dapatkan lokasi output
job = mlclient.jobs.get(job.name)

print(f"Output: {job.outputs}")

Monitoring

1. Dapatkan Logs

# Dapatkan deployment logs
logs = mlclient.onlinedeployments.getlogs(
    endpointname="my-online-endpoint",

    deploymentname="blue",
    lines=100
)
print(logs)

2. Metrics

from azure.monitor.query import MetricsQueryClient
from datetime import datetime, timedelta

metricsclient = MetricsQueryClient(credential=DefaultAzureCredential())


Query metrics
response = metricsclient.queryresource(

    resourceuri="/subscriptions/.../endpoints/my-online-endpoint",
    metricnames=["RequestsPerMinute", "RequestLatency", "RequestsSucceeded"],

    timespan=timedelta(hours=1)
)

for metric in response.metrics:
    print(f"{metric.name}: {metric.timeseries[0].data[-1].average}")

3. Application Insights

# Aktifkan Application Insights di deployment
deployment = ManagedOnlineDeployment(
    name="monitored",
    endpointname="my-online-endpoint",
    model=Model(path="./model"),
    # ... konfigurasi lain
    appinsightsenabled=True
)

Security

1. Network Isolation

# Buat endpoint dengan private endpoint
privateendpoint = ManagedOnlineEndpoint(

    name="private-endpoint",
    description="Private endpoint",
    authmode="key",
    publicnetworkaccess="disabled"  # Disable public access
)

mlclient.onlineendpoints.begincreateorupdate(privateendpoint).result()

2. Managed Identity

from azure.ai.ml.entities import ManagedIdentityConfiguration

Buat endpoint dengan managed identity
identityendpoint = ManagedOnlineEndpoint(

    name="identity-endpoint",
    authmode="amltoken",

    identity=ManagedIdentityConfiguration(
        type="SystemAssigned"
    )
)

mlclient.onlineendpoints.begincreateorupdate(identityendpoint).result()

Best Practices

1. Health Checks

# Konfigurasi probes dengan benar
deployment = ManagedOnlineDeployment(
    name="healthy",
    # ...
    livenessprobe=ProbeSettings(
        initialdelay=30,  # Tunggu model loading

        period=10,
        timeout=2,
        failurethreshold=3
    ),
    readinessprobe=ProbeSettings(

        initialdelay=30,
        period=10,
        timeout=2,
        successthreshold=1,

        failurethreshold=3
    )
)

2. Resource Cleanup

# Hapus deployment
mlclient.onlinedeployments.begindelete(

    endpointname="my-online-endpoint",
    deploymentname="blue"

).result()

Hapus endpoint
mlclient.onlineendpoints.begindelete("my-online-endpoint").result()

Kesimpulan

Azure ML Managed Endpoints menyediakan:

Simplicity: Infrastruktur fully managed

Scalability: Kemampuan auto-scaling

Reliability: Blue-green deployments

Security: Network isolation dan identity

Observability: Built-in monitoring

Key takeaways:

• Gunakan online endpoints untuk real-time inference
• Gunakan batch endpoints untuk processing skala besar
• Implementasikan blue-green deployments untuk rollout aman
• Konfigurasi auto-scaling untuk variasi traffic
• Monitor performa dan kesehatan secara kontinu