Azure DevOps for MLOps Tutorial: CI/CD for Machine Learning

# Tutorial Lengkap Azure DevOps untuk MLOps: CI/CD untuk Machine Learning Azure DevOps menyediakan kemampuan CI/CD komprehensif untuk proyek machine learning. Tutorial ini mencakup pembangunan pipeli...

By Ruby Abdullah · · tutorial
AzureDevOpsMLOpsCI/CDAutomationPipeline

Complete Azure DevOps for MLOps Tutorial: CI/CD for Machine Learning

Azure DevOps provides comprehensive CI/CD capabilities for machine learning projects. This tutorial covers building automated ML pipelines, model deployment, and continuous delivery using Azure DevOps.

Why Azure DevOps for MLOps?

Key Benefits:
  • End-to-end automation: From code to deployment
  • Version control: Git repos for code and data
  • Pipeline orchestration: Multi-stage ML workflows
  • Integration: Native Azure ML integration
  • Collaboration: Team-based development

Components:
  • Azure Repos: Git repositories
  • Azure Pipelines: CI/CD automation
  • Azure Artifacts: Package management
  • Azure Boards: Work tracking

Prerequisites

pip install azure-devops azure-ai-ml


Azure CLI


az login


az extension add --name azure-devops


az devops configure --defaults organization=https://dev.azure.com/myorg

Project Setup

1. Create DevOps Project

# Create project

az devops project create --name "MLOps-Project" --org https://dev.azure.com/myorg



Create repository


az repos create --name "ml-models" --project "MLOps-Project"

2. Repository Structure

ml-models/

├── src/


│   ├── train.py


│   ├── evaluate.py


│   └── score.py


├── tests/


│   └── testmodel.py


├── pipelines/


│   ├── train-pipeline.yml


│   ├── deploy-pipeline.yml


│   └── cd-pipeline.yml


├── infrastructure/


│   └── arm-templates/


├── environment.yml


├── requirements.txt


└── azure-pipelines.yml

3. Service Connection

# Create service connection to Azure

az devops service-endpoint azurerm create \


    --azure-rm-service-principal-id "your-sp-id" \


    --azure-rm-subscription-id "your-subscription-id" \


    --azure-rm-subscription-name "Your Subscription" \


    --azure-rm-tenant-id "your-tenant-id" \


    --name "azure-ml-connection"

CI Pipeline for ML

1. Basic CI Pipeline

# azure-pipelines.yml

trigger:


  branches:


    include:


  • main
    • 

  • develop
    • 

  paths:


    include:


  • src/
    • 

  • tests/
    • 



pool:


  vmImage: 'ubuntu-latest'



variables:


  pythonVersion: '3.9'



stages:


  • stage: Build
    • 

    displayName: 'Build and Test'


    jobs:


  • job: BuildJob
    • 

        steps:


  • task: UsePythonVersion@0
    • 

            inputs:


              versionSpec: '$(pythonVersion)'


            displayName: 'Use Python $(pythonVersion)'


  • script: |
    • 

              python -m pip install --upgrade pip


              pip install -r requirements.txt


              pip install pytest pytest-cov


            displayName: 'Install dependencies'


  • script: |
    • 

              python -m pytest tests/ --cov=src --cov-report=xml


            displayName: 'Run tests'


  • task: PublishTestResults@2
    • 

            inputs:


              testResultsFiles: '*/test-.xml'


              testRunTitle: 'Python Tests'


  • task: PublishCodeCoverageResults@1
    • 

            inputs:


              codeCoverageTool: Cobertura


              summaryFileLocation: '$(System.DefaultWorkingDirectory)/*/coverage.xml'

2. Linting and Code Quality

# Add to azure-pipelines.yml

  • script: |
    • 

              pip install flake8 black mypy


              flake8 src/ --max-line-length=100


              black --check src/


              mypy src/


            displayName: 'Code quality checks'

Training Pipeline

1. ML Training Pipeline

# pipelines/train-pipeline.yml

trigger:


  branches:


    include:


  • main
    • 

  paths:


    include:


  • src/
    • 

  • data/
    • 



variables:


  • group: ml-variables
    • 

  • name: resourceGroup
    • 

    value: 'ml-rg'

Related Articles