MLflow Tutorial: Open-Source Platform for End-to-End MLOps
MLflow is an open-source platform developed by Databricks for managing the entire machine learning lifecycle. From experiment tracking and model packaging to production deployment, MLflow provides integrated and easy-to-use tools. With over 18,000 stars on GitHub and widespread industry adoption, MLflow has become the de facto standard in the MLOps ecosystem.
In this tutorial, we will learn how to use MLflow comprehensively, from installation, experiment tracking, model registry, to serving models in production.
Why MLflow?
Before diving into implementation, it's important to understand the problems MLflow solves:
- Reproducibility: Without proper tracking, it's difficult to reproduce previous experiment results. MLflow automatically records all parameters, metrics, and artifacts.
- Collaboration: Data science teams often work in parallel. MLflow provides a centralized server for sharing experiments and models.
- Deployment: The transition from notebook to production is often a bottleneck. MLflow simplifies this process with a standard model format and built-in serving.
- Governance: MLflow's Model Registry enables versioning, staging, and approval workflows for production models.
Installation and Setup
Basic Installation
pip install mlflow
For additional features, install with extras:
pip install mlflow[extras]
If you're using specific frameworks, also install the corresponding dependencies:
pip install mlflow scikit-learn xgboost lightgbm tensorflow torch
Running MLflow UI
MLflow provides a web UI for viewing experiments and models. Launch it with:
mlflow ui --port 5000
Open your browser and navigate to http://localhost:5000. You'll see an empty dashboard ready to receive experiment data.
Setting Up a Tracking Server (Optional)
For teams, it's recommended to use a centralized tracking server:
mlflow server \
--backend-store-uri sqlite:///mlflow.db \
--default-artifact-root ./mlflow-artifacts \
--host 0.0.0.0 \
--port 5000
With PostgreSQL for production:
mlflow server \
--backend-store-uri postgresql://user:password@localhost:5432/mlflow \
--default-artifact-root s3://my-mlflow-bucket/artifacts \
--host 0.0.0.0 \
--port 5000
Experiment Tracking
Experiment tracking is MLflow's core feature. Every time you train a model, MLflow records parameters, metrics, and artifacts in a "run".
Basic Tracking
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import loadiris
from sklearn.modelselection import traintestsplit
from sklearn.metrics import accuracyscore, f1score, precisionscore
X, y = loadiris(returnXy=True)
Xtrain, Xtest, ytrain, ytest = traintestsplit(X, y, testsize=0.2, randomstate=42)
mlflow.setexperiment("iris-classification")
with mlflow.startrun(runname="random-forest-baseline"):
nestimators = 100
maxdepth = 5
minsamplessplit = 2
mlflow.logparam("nestimators", nestimators)
mlflow.logparam("maxdepth", maxdepth)
mlflow.logparam("minsamplessplit", minsamplessplit)
model = RandomForestClassifier(
nestimators=nestimators,
maxdepth=maxdepth,
minsamplessplit=minsamplessplit,
randomstate=42
)
model.fit(Xtrain, ytrain)
ypred = model.predict(Xtest)
accuracy = accuracyscore(ytest, ypred)
f1 = f1score(ytest, ypred, average="weighted")
precision = precisionscore(ytest, ypred, average="weighted")
mlflow.logmetric("accuracy", accuracy)
mlflow.logmetric("f1score", f1)
mlflow.logmetric("precision", precision)
mlflow.sklearn.logmodel(model, "model")
print(f"Accuracy: {accuracy:.4f}")
print(f"F1 Score: {f1:.4f}")
Autologging
MLflow supports autologging for popular frameworks, so you don't need to manually record parameters and metrics:
import mlflow
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.datasets import loadiris
from sklearn.modelselection import traintestsplit
mlflow.sklearn.autolog()
X, y = loadiris(returnXy=True)
Xtrain, Xtest, ytrain, ytest = traintestsplit(X, y, testsize=0.2, randomstate=42)
mlflow.setexperiment("iris-autolog")
with mlflow.startrun():
model = GradientBoostingClassifier(nestimators=200, maxdepth=3)
model.fit(Xtrain, ytrain)
Autologging is available for many frameworks:
mlflow.sklearn.autolog()
mlflow.tensorflow.autolog()
mlflow.pytorch.autolog()
mlflow.xgboost.autolog()
mlflow.lightgbm.autolog()
mlflow.spark.autolog()
Logging Artifacts
Besides parameters and metrics, you can save artifact files such as plots, data, or configurations:
import mlflow
import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import confusionmatrix, ConfusionMatrixDisplay
import json
with mlflow.startrun(runname="with-artifacts"):
model = RandomForestClassifier(nestimators=100)
model.fit(Xtrain, ytrain)
ypred = model.predict(Xtest)
cm = confusionmatrix(ytest, ypred)
disp = ConfusionMatrixDisplay(confusionmatrix=cm)
disp.plot()
plt.savefig("confusionmatrix.png")
plt.close()
mlflow.logartifact("confusionmatrix.png")
featureimportance = dict(zip(
[f"feature{i}" for i in range(Xtrain.shape[1])],
model.featureimportances.tolist()
))
with open("featureimportance.json", "w") as f:
json.dump(featureimportance, f, indent=2)
mlflow.logartifact("featureimportance.json")
mlflow.logdict(
{"modeltype": "RandomForest", "framework": "scikit-learn"},
"modelconfig.json"
)
Nested Runs for Hyperparameter Tuning
import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.modelselection import ParameterGrid
paramgrid = {
"nestimators": [50, 100, 200],
"maxdepth": [3, 5, 10],
"minsamplessplit": [2, 5]
}
mlflow.setexperiment("iris-hyperparameter-tuning")
with mlflow.startrun(runname="grid-search"):
bestaccuracy = 0
bestparams = None
for params in ParameterGrid(paramgrid):
with mlflow.startrun(runname=f"rf-{params}", nested=True):
mlflow.logparams(params)
model = RandomForestClassifier(params, randomstate=42)
model.fit(Xtrain, ytrain)
accuracy = model.score(Xtest, ytest)
mlflow.logmetric("accuracy", accuracy)
if accuracy > bestaccuracy:
bestaccuracy = accuracy
bestparams = params
mlflow.logparams({f"best{k}": v for k, v in bestparams.items()})
mlflow.logmetric("bestaccuracy", bestaccuracy)
print(f"Best params: {bestparams}, Accuracy: {bestaccuracy:.4f}")
Model Registry
The Model Registry is MLflow's feature for managing models throughout their lifecycle. You can register models, version them, and manage transitions between stages.
Registering a Model
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
mlflow.setexperiment("iris-registry")
with mlflow.startrun():
model = RandomForestClassifier(nestimators=100, maxdepth=5, randomstate=42)
model.fit(Xtrain, ytrain)
accuracy = model.score(Xtest, ytest)
mlflow.logmetric("accuracy", accuracy)
mlflow.sklearn.logmodel(
model,
"model",
registeredmodelname="iris-classifier"
)
Managing Model Versions
from mlflow.client import MlflowClient
client = MlflowClient()
client.setregisteredmodeltag(
"iris-classifier",
"task",
"classification"
)
client.setmodelversiontag(
"iris-classifier",
version="1",
key="validationstatus",
value="approved"
)
modelinfo = client.getregisteredmodel("iris-classifier")
print(f"Model: {modelinfo.name}")
print(f"Latest versions: {[v.version for v in modelinfo.latestversions]}")
Model Aliases
MLflow uses aliases to mark specific model versions (replacing the legacy stages system):
from mlflow.client import MlflowClient
client = MlflowClient()
client.setregisteredmodelalias("iris-classifier", "champion", 1)
championmodel = mlflow.pyfunc.loadmodel("models:/iris-classifier@champion")
client.setregisteredmodelalias("iris-classifier", "challenger", 2)
championuri = "models:/iris-classifier@champion"
challengeruri = "models:/iris-classifier@challenger"
Comparing Model Versions
from mlflow.client import MlflowClient
client = MlflowClient()
versions = client.searchmodelversions("name='iris-classifier'")
for v in versions:
run = client.getrun(v.runid)
accuracy = run.data.metrics.get("accuracy", "N/A")
print(f"Version {v.version}: accuracy={accuracy}, aliases={v.aliases}")
MLflow Models and Flavors
MLflow Models provides a standard format for saving models from various frameworks. Each framework has its own "flavor".
Custom Model with PythonModel
import mlflow
import mlflow.pyfunc
import numpy as np
class PreprocessingModel(mlflow.pyfunc.PythonModel):
def init(self, scalerparams, model):
self.scalermean = scalerparams["mean"]
self.scalerstd = scalerparams["std"]
self.model = model
def predict(self, context, modelinput, params=None):
scaledinput = (modelinput - self.scalermean) / self.scalerstd
predictions = self.model.predict(scaledinput)
return predictions
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
scaler = StandardScaler()
Xtrainscaled = scaler.fittransform(Xtrain)
rfmodel = RandomForestClassifier(nestimators=100, randomstate=42)
rfmodel.fit(Xtrainscaled, ytrain)
custommodel = PreprocessingModel(
scalerparams={"mean": scaler.mean, "std": scaler.scale},
model=rfmodel
)
with mlflow.startrun():
mlflow.pyfunc.logmodel(
artifactpath="model",
pythonmodel=custommodel,
registeredmodelname="iris-preprocessed-classifier"
)
Model Signature
Model signatures document the expected inputs and outputs:
import mlflow
from mlflow.models import infersignature
import pandas as pd
Xsample = pd.DataFrame(Xtrain[:5], columns=["sepallength", "sepalwidth", "petallength", "petalwidth"])
ysample = model.predict(Xsample)
signature = infersignature(Xsample, ysample)
with mlflow.startrun():
mlflow.sklearn.logmodel(
model,
"model",
signature=signature,
inputexample=Xsample.iloc[:2]
)
Model Serving
MLflow provides several ways to deploy models to production.
Local Serving
mlflow models serve -m "models:/iris-classifier@champion" --port 8000 --no-conda
Test with curl:
curl -X POST http://localhost:8000/invocations \
-H "Content-Type: application/json" \
-d '{"dataframesplit": {"columns": ["sepallength", "sepalwidth", "petallength", "petalwidth"], "data": [[5.1, 3.5, 1.4, 0.2]]}}'
Batch Prediction
import mlflow
import pandas as pd
model = mlflow.pyfunc.loadmodel("models:/iris-classifier@champion")
batchdata = pd.DataFrame({
"sepallength": [5.1, 6.2, 4.8],
"sepalwidth": [3.5, 2.8, 3.0],
"petallength": [1.4, 4.8, 1.4],
"petalwidth": [0.2, 1.8, 0.3]
})
predictions = model.predict(batchdata)
print(f"Predictions: {predictions}")
Docker Deployment
MLflow can generate Docker images for your models:
mlflow models build-docker -m "models:/iris-classifier/1" -n "iris-classifier-docker"
docker run -p 8000:8080 iris-classifier-docker
MLflow Projects
MLflow Projects provide a standard format for packaging ML code for reproducibility.
Project Structure
Create an MLproject file in your root directory:
name: iris-training
condaenv: conda.yaml
entrypoints:
main:
parameters:
nestimators: {type: int, default: 100}
maxdepth: {type: int, default: 5}
testsize: {type: float, default: 0.2}
command: "python train.py --n-estimators {nestimators} --max-depth {maxdepth} --test-size {testsize}"
evaluate:
parameters:
modeluri: {type: str}
command: "python evaluate.py --model-uri {modeluri}"
Training script train.py:
import argparse
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import loadiris
from sklearn.modelselection import traintestsplit
from sklearn.metrics import accuracyscore
def main():
parser = argparse.ArgumentParser()
parser.addargument("--n-estimators", type=int, default=100)
parser.addargument("--max-depth", type=int, default=5)
parser.addargument("--test-size", type=float, default=0.2)
args = parser.parseargs()
X, y = loadiris(returnXy=True)
Xtrain, Xtest, ytrain, ytest = traintestsplit(
X, y, testsize=args.testsize, randomstate=42
)
with mlflow.startrun():
model = RandomForestClassifier(
nestimators=args.nestimators,
maxdepth=args.maxdepth,
randomstate=42
)
model.fit(Xtrain, ytrain)
accuracy = accuracyscore(ytest, model.predict(Xtest))
mlflow.logmetric("accuracy", accuracy)
mlflow.sklearn.logmodel(model, "model")
print(f"Accuracy: {accuracy:.4f}")
if name == "main":
main()
Running a Project
mlflow run . -P nestimators=200 -P maxdepth=10
mlflow run https://github.com/user/ml-project -P n
estimators=200
MLflow with Deep Learning
PyTorch Integration
import mlflow
import mlflow.pytorch
import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset
class IrisNet(nn.Module):
def init(self, hiddensize=64):
super().init()
self.net = nn.Sequential(
nn.Linear(4, hiddensize),
nn.ReLU(),
nn.Dropout(0.2),
nn.Linear(hiddensize, 32),
nn.ReLU(),
nn.Linear(32, 3)
)
def forward(self, x):
return self.net(x)
mlflow.pytorch.autolog()
mlflow.setexperiment("iris-pytorch")
with mlflow.startrun():
Xtensor = torch.FloatTensor(Xtrain)
ytensor = torch.LongTensor(ytrain)
dataset = TensorDataset(Xtensor, ytensor)
dataloader = DataLoader(dataset, batchsize=16, shuffle=True)
model = IrisNet(hiddensize=64)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
for epoch in range(50):
model.train()
totalloss = 0
for batchX, batchy in dataloader:
optimizer.zerograd()
outputs = model(batchX)
loss = criterion(outputs, batchy)
loss.backward()
optimizer.step()
totalloss += loss.item()
avgloss = totalloss / len(dataloader)
mlflow.logmetric("trainloss", avgloss, step=epoch)
model.eval()
with torch.nograd():
Xtesttensor = torch.FloatTensor(Xtest)
outputs = model(Xtesttensor)
, predicted = torch.max(outputs, 1)
accuracy = (predicted.numpy() == ytest).mean()
mlflow.logmetric("testaccuracy", accuracy)
mlflow.pytorch.logmodel(model, "model")
TensorFlow/Keras Integration
import mlflow
import mlflow.tensorflow
import tensorflow as tf
mlflow.tensorflow.autolog()
mlflow.setexperiment("iris-tensorflow")
with mlflow.startrun():
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, activation="relu", inputshape=(4,)),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(32, activation="relu"),
tf.keras.layers.Dense(3, activation="softmax")
])
model.compile(
optimizer="adam",
loss="sparsecategoricalcrossentropy",
metrics=["accuracy"]
)
model.fit(
Xtrain, ytrain,
epochs=50,
batchsize=16,
validationsplit=0.2,
verbose=0
)
loss, accuracy = model.evaluate(Xtest, ytest, verbose=0)
print(f"Test Accuracy: {accuracy:.4f}")
MLflow Evaluate
MLflow Evaluate provides automated evaluation for ML models and LLMs:
import mlflow
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
mlflow.setexperiment("iris-evaluate")
with mlflow.startrun():
model = RandomForestClassifier(nestimators=100, randomstate=42)
model.fit(Xtrain, ytrain)
mlflow.sklearn.logmodel(model, "model")
modeluri = mlflow.getartifacturi("model")
evaldata = pd.DataFrame(Xtest, columns=["sepallength", "sepalwidth", "petallength", "petalwidth"])
evaldata["label"] = ytest
result = mlflow.evaluate(
model=modeluri,
data=evaldata,
targets="label",
modeltype="classifier",
evaluators=["default"]
)
print(f"Metrics: {result.metrics}")
Best Practices
1. Experiment Organization
Use consistent naming conventions:
mlflow.setexperiment("project-name/task/sub-task")
mlflow.set
experiment("fraud-detection/training/v2")
2. Tagging for Search
with mlflow.startrun():
mlflow.set
tag("team", "ml-platform")
mlflow.settag("dataversion", "2026-01-15")
mlflow.settag("environment", "staging")
mlflow.settag("commithash", "abc123")
3. Environment Management
Always log the environment for reproducibility:
import mlflow
import sys
with mlflow.startrun():
mlflow.logparam("pythonversion", sys.version)
mlflow.logartifact("requirements.txt")
4. Use Context Managers
Always use the with mlflow.startrun() context manager so runs are automatically closed even if errors occur:
with mlflow.startrun() as run:
try:
model.fit(X
train, ytrain)
mlflow.log
metric("accuracy", accuracy)
except Exception as e:
mlflow.settag("error", str(e))
raise
5. Model Signature and Input Examples
Always include signatures and input examples when logging models:
from mlflow.models import infersignature
signature = infersignature(Xtrain, model.predict(Xtrain))
mlflow.sklearn.logmodel(
model,
"model",
signature=signature,
inputexample=Xtrain[:2]
)
6. Tracking Server for Teams
For production, use a centralized tracking server with database backend and object storage:
mlflow server \
--backend-store-uri postgresql://user:pass@db:5432/mlflow \
--default-artifact-root s3://mlflow-artifacts/ \
--host 0.0.0.0
CI/CD Integration
Example GitHub Actions workflow for training and model registration:
name: ML Training Pipeline
on:
push:
paths:
- 'training/'
- 'data/'
jobs:
train:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: pip install -r requirements.txt
- name: Train and register model
env:
MLFLOWTRACKINGURI: ${{ secrets.MLFLOWTRACKINGURI }}
run: |
python train.py --n-estimators 200 --max-depth 10
Conclusion
MLflow is a comprehensive and mature MLOps platform for managing the entire machine learning lifecycle. Its key features include:
- Experiment Tracking: Automatically record all parameters, metrics, and artifacts
- Model Registry: Manage model versions with aliases and tags
- Model Serving: Deploy models via REST API, Docker, or batch prediction
- MLflow Projects: Package ML code for reproducibility
- MLflow Evaluate**: Automated evaluation for ML models and LLMs
With support for nearly all popular ML frameworks and easy integration with existing infrastructure, MLflow is a solid choice for starting or improving MLOps practices in your organization. Start with simple experiment tracking, then gradually adopt other features as your team's needs grow.