Complete Comet ML Tutorial: MLOps Platform for Experiment Tracking and Model Management

In modern machine learning, manually managing experiments becomes a significant challenge as project complexity grows. Comet ML is an MLOps platform that enables data scientists and ML engineers to track experiments, compare models, and manage the machine learning lifecycle efficiently. This tutorial covers how to use Comet ML from installation through advanced features for production ML projects.

What Is Comet ML?

Comet ML is an experiment tracking and model management platform that provides a centralized dashboard for logging parameters, metrics, artifacts, and code from every ML experiment. Unlike manual tracking with spreadsheets or log files, Comet ML automatically captures all important information and presents it through an intuitive visual interface.

Key advantages of Comet ML include:

• Automatic logging for popular frameworks like PyTorch, TensorFlow, scikit-learn, and XGBoost
• Experiment comparison with real-time metric visualization
• Model registry for versioning and deployment tracking
• Artifact management for dataset and model versioning
• Team collaboration with sharing and reporting features

Installation and Setup

Package Installation

Install Comet ML using pip:

pip install cometml

pip install cometml[pytorch]
pip install cometml[tensorflow]

pip install cometml[sklearn]

API Key Configuration

After creating an account on comet.com, obtain your API key from the Settings page. There are several ways to configure your API key:

export COMETAPIKEY="your-api-key-here"

Create a .comet.config file in your home directory:

[comet]
apikey=your-api-key-here

projectname=my-ml-project
workspace=my-workspace

import cometml

cometml.login(apikey="your-api-key-here")

import cometml

experiment = cometml.Experiment(

    projectname="test-project",
    autometriclogging=True,
    autoparamlogging=True,
)

experiment.logparameter("testparam", "hello")
experiment.logmetric("testmetric", 0.95)
experiment.end()

print("Comet ML successfully configured!")

Basic Usage: Experiment Tracking

Logging Parameters and Metrics

Here is a basic example of using Comet ML to track a simple classification experiment:

import cometml
from sklearn.datasets import loadiris
from sklearn.modelselection import traintestsplit

from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracyscore, f1score


experiment = cometml.Experiment(
    projectname="iris-classification",

    autometriclogging=False,

)

X, y = loadiris(returnXy=True)
Xtrain, Xtest, ytrain, ytest = traintestsplit(
    X, y, testsize=0.2, randomstate=42
)

params = {
    "nestimators": 100,

    "maxdepth": 5,
    "minsamplessplit": 2,
    "randomstate": 42,

}
experiment.logparameters(params)

model = RandomForestClassifier(params)

model.fit(Xtrain, ytrain)

ypred = model.predict(Xtest)

accuracy = accuracyscore(ytest, ypred)

f1 = f1score(ytest, ypred, average="weighted")

experiment.logmetric("accuracy", accuracy)

experiment.logmetric("f1score", f1)


experiment.addtag("baseline")
experiment.addtag("random-forest")


experiment.end()
print(f"Accuracy: {accuracy:.4f}, F1: {f1:.4f}")

import cometml
import numpy as np

experiment = cometml.Experiment(projectname="training-loop-demo")

numepochs = 50

for epoch in range(numepochs):
    trainloss = 1.0 / (epoch + 1) + np.random.normal(0, 0.02)

    valloss = 1.0 / (epoch + 1) + np.random.normal(0, 0.05) + 0.1
    trainacc = 1 - trainloss + np.random.normal(0, 0.01)
    valacc = 1 - valloss + np.random.normal(0, 0.02)

    experiment.logmetric("trainloss", trainloss, step=epoch)

    experiment.logmetric("valloss", valloss, step=epoch)
    experiment.logmetric("trainaccuracy", trainacc, step=epoch)

    experiment.logmetric("valaccuracy", valacc, step=epoch)

experiment.end()

import cometml
import matplotlib.pyplot as plt
from sklearn.metrics import confusionmatrix, ConfusionMatrixDisplay
from sklearn.datasets import loadiris

from sklearn.modelselection import traintestsplit
from sklearn.ensemble import RandomForestClassifier

experiment = cometml.Experiment(projectname="visualization-demo")

X, y = loadiris(returnXy=True)

Xtrain, Xtest, ytrain, ytest = traintestsplit(X, y, testsize=0.2)

model = RandomForestClassifier(nestimators=100)

model.fit(Xtrain, ytrain)

ypred = model.predict(Xtest)


cm = confusionmatrix(ytest, ypred)
disp = ConfusionMatrixDisplay(confusionmatrix=cm)

disp.plot(cmap="Blues")
plt.title("Confusion Matrix")

experiment.logfigure(figurename="confusionmatrix", figure=plt)
plt.close()

experiment.logconfusionmatrix(
    ytrue=ytest.tolist(),
    ypredicted=ypred.tolist(),
    labels=["setosa", "versicolor", "virginica"],
)

experiment.end()

import cometml
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

experiment = cometml.Experiment(
    projectname="pytorch-demo",

    autometriclogging=True,

    autoparamlogging=True,

    loggraph=True,
)

class SimpleNet(nn.Module):
    def init(self, inputdim, hiddendim, outputdim):

        super().init()
        self.fc1 = nn.Linear(inputdim, hiddendim)

        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(0.3)
        self.fc2 = nn.Linear(hiddendim, outputdim)


    def forward(self, x):
        x = self.fc1(x)
        x = self.relu(x)
        x = self.dropout(x)
        x = self.fc2(x)
        return x

params = {
    "inputdim": 20,
    "hiddendim": 64,

    "outputdim": 3,
    "learningrate": 0.001,

    "batchsize": 32,
    "epochs": 20,
}
experiment.logparameters(params)


Xtrain = torch.randn(500, params["inputdim"])

ytrain = torch.randint(0, params["outputdim"], (500,))


dataset = TensorDataset(Xtrain, ytrain)

dataloader = DataLoader(dataset, batchsize=params["batchsize"], shuffle=True)


model = SimpleNet(params["inputdim"], params["hiddendim"], params["outputdim"])
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=params["learningrate"])


for epoch in range(params["epochs"]):
    model.train()
    totalloss = 0
    correct = 0
    total = 0

    for batchX, batchy in dataloader:
        optimizer.zerograd()

        outputs = model(batchX)
        loss = criterion(outputs, batchy)

        loss.backward()
        optimizer.step()

        totalloss += loss.item()
        , predicted = torch.max(outputs, 1)

        total += batchy.size(0)
        correct += (predicted == batchy).sum().item()


    avgloss = totalloss / len(dataloader)

    accuracy = correct / total

    experiment.logmetric("epochloss", avgloss, step=epoch)
    experiment.logmetric("epochaccuracy", accuracy, step=epoch)

experiment.logmodel("simplenet", "./model.pth")
experiment.end()

import cometml
from sklearn.datasets import fetchcaliforniahousing

from sklearn.modelselection import traintestsplit, crossvalscore
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.metrics import meansquarederror, r2score

import numpy as np

experiment = cometml.Experiment(
    projectname="sklearn-regression",

    autooutputlogging="native",

)

data = fetchcaliforniahousing()

Xtrain, Xtest, ytrain, ytest = traintestsplit(

    data.data, data.target, testsize=0.2, randomstate=42

)

params = {
    "nestimators": 200,
    "maxdepth": 4,

    "learningrate": 0.1,
    "subsample": 0.8,
    "minsamplesleaf": 10,
}
experiment.logparameters(params)


model = GradientBoostingRegressor(params, randomstate=42)
model.fit(Xtrain, ytrain)

ypred = model.predict(Xtest)
mse = meansquarederror(ytest, ypred)
rmse = np.sqrt(mse)
r2 = r2score(ytest, ypred)


experiment.logmetric("mse", mse)
experiment.logmetric("rmse", rmse)

experiment.logmetric("r2score", r2)


cvscores = crossvalscore(model, data.data, data.target, cv=5, scoring="r2")
experiment.logmetric("cvmeanr2", cvscores.mean())
experiment.logmetric("cvstdr2", cvscores.std())

featureimportance = dict(zip(data.featurenames, model.featureimportances))
experiment.logparameters({"featureimportance": featureimportance})


experiment.end()

import cometml
import xgboost as xgb
from sklearn.datasets import loadbreastcancer
from sklearn.modelselection import traintestsplit

from sklearn.metrics import accuracyscore, rocaucscore

experiment = cometml.Experiment(projectname="xgboost-demo")

data = loadbreastcancer()
Xtrain, Xtest, ytrain, ytest = traintestsplit(
    data.data, data.target, testsize=0.2, randomstate=42
)

dtrain = xgb.DMatrix(Xtrain, label=ytrain)
dtest = xgb.DMatrix(Xtest, label=ytest)

params = {
    "maxdepth": 4,

    "learningrate": 0.1,
    "objective": "binary:logistic",
    "evalmetric": "auc",

    "nestimators": 100,
}
experiment.logparameters(params)


model = xgb.train(
    params,
    dtrain,
    numboostround=params["nestimators"],
    evals=[(dtrain, "train"), (dtest, "eval")],
    verboseeval=10,

)

ypredproba = model.predict(dtest)

ypred = (ypredproba > 0.5).astype(int)

accuracy = accuracyscore(ytest, ypred)

auc = rocaucscore(ytest, ypredproba)

experiment.logmetric("accuracy", accuracy)

experiment.logmetric("auc", auc)
experiment.end()

Advanced Usage

Artifact Management

Comet ML Artifacts allow you to track versions of datasets and models:

import cometml
from cometml import Artifact

experiment = cometml.Experiment(projectname="artifact-demo")

datasetartifact = Artifact(

    name="training-dataset",
    artifacttype="dataset",
    version="1.0.0",
    metadata={
        "numsamples": 10000,

        "numfeatures": 20,
        "split": "train",
        "preprocessing": "standardscaler",

    },
)

datasetartifact.add("./data/train.csv")
datasetartifact.add("./data/metadata.json")


experiment.logartifact(datasetartifact)


modelartifact = Artifact(
    name="trained-model",
    artifacttype="model",

    version="1.0.0",
    metadata={
        "framework": "pytorch",
        "accuracy": 0.95,
        "architecture": "resnet50",
    },
)
modelartifact.add("./models/model.pth")
modelartifact.add("./models/config.json")


experiment.logartifact(modelartifact)

experiment.end()

import cometml
from cometml import API


api = API()

experiment = cometml.Experiment(projectname="model-registry-demo")


experiment.logparameter("modeltype", "gradientboosting")
experiment.logmetric("accuracy", 0.94)

experiment.logmetric("f1score", 0.93)

experiment.logmodel("mymodel", "./model/")


experiment.registermodel("mymodel", registryname="production-classifier")

experiment.end()

registeredmodel = api.getmodel(
    workspace="my-workspace",
    modelname="production-classifier"

)
print(f"Model versions: {registeredmodel.findversions()}")

import cometml
from cometml import Optimizer

from sklearn.datasets import loaddigits
from sklearn.modelselection import crossvalscore

from sklearn.svm import SVC

config = {
    "algorithm": "bayes",
    "parameters": {
        "C": {
            "type": "float",
            "min": 0.01,
            "max": 100.0,
            "scalingType": "loguniform",
        },
        "gamma": {
            "type": "discrete",
            "values": ["scale", "auto"],
        },
        "kernel": {
            "type": "categorical",
            "values": ["rbf", "poly", "sigmoid"],
        },
    },
    "spec": {
        "maxCombo": 30,
        "objective": "maximize",
        "metric": "cvaccuracy",
    },
}

X, y = loaddigits(returnXy=True)


optimizer = Optimizer(config, projectname="svm-optimization")

for experiment in optimizer.getexperiments():

    C = experiment.getparameter("C")
    gamma = experiment.getparameter("gamma")

    kernel = experiment.getparameter("kernel")

    model = SVC(C=float(C), gamma=gamma, kernel=kernel)

    scores = crossvalscore(model, X, y, cv=5, scoring="accuracy")
    meanaccuracy = scores.mean()


    experiment.logmetric("cvaccuracy", meanaccuracy)
    experiment.logmetric("cvstd", scores.std())
    experiment.end()

Custom Panels and Visualizations

import cometml
import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import precisionrecallcurve, roccurve

experiment = cometml.Experiment(projectname="custom-panels-demo")

ytrue = np.random.randint(0, 2, 200)

yscores = np.random.random(200)

precision, recall,  = precisionrecallcurve(ytrue, yscores)

fpr, tpr,  = roccurve(ytrue, yscores)


fig, axes = plt.subplots(1, 2, figsize=(14, 5))

axes[0].plot(recall, precision)
axes[0].setxlabel("Recall")
axes[0].setylabel("Precision")

axes[0].settitle("Precision-Recall Curve")
axes[0].grid(True)

axes[1].plot(fpr, tpr)
axes[1].plot([0, 1], [0, 1], "k--")
axes[1].setxlabel("False Positive Rate")

axes[1].setylabel("True Positive Rate")
axes[1].settitle("ROC Curve")

axes[1].grid(True)

plt.tightlayout()
experiment.logfigure(figurename="evaluationcurves", figure=plt)

plt.close()

data = [[x, np.sin(x), np.cos(x)] for x in np.linspace(0, 10, 100)]
experiment.logtable("trigfunctions.csv", tabulardata=data, headers=["x", "sin", "cos"])

experiment.loghtml("
Model Summary
This model achieved 95% accuracy on the test set.
")


experiment.end()

import cometml

offlineexperiment = cometml.OfflineExperiment(
    projectname="offline-project",

    offlinedirectory="./cometoffline",

)

offlineexperiment.logparameter("model", "randomforest")
offlineexperiment.logmetric("accuracy", 0.92)

offlineexperiment.end()

comet upload ./cometoffline/*.zip

Programmatic Experiment Comparison

from cometml import API

api = API()

experiments = api.get("my-workspace/my-project")

results = []
for exp in experiments:
    metrics = exp.getmetricssummary()

    params = exp.getparameterssummary()


    accuracy = next(
        (m["valueCurrent"] for m in metrics if m["name"] == "accuracy"), None
    )
    results.append({
        "experimentkey": exp.key,
        "name": exp.name,
        "accuracy": accuracy,
    })

results.sort(key=lambda x: float(x["accuracy"] or 0), reverse=True)

print("Top 5 Experiments:")
for i, r in enumerate(results[:5], 1):
    print(f"  {i}. {r['name']}: accuracy={r['accuracy']}")

bestexp = api.getexperimentbykey(results[0]["experimentkey"])

print(f"\nBest experiment: {bestexp.name}")
print(f"URL: {bestexp.url}")

import cometml

experiment = cometml.Experiment(

    projectname="text-classification",
    workspace="my-team",
)

experiment.setname("bert-base-lr0001-bs32")


experiment.addtags([
    "bert",
    "text-classification",
    "production-candidate",
])

experiment.logother("datasetversion", "v2.3")
experiment.logother("datasplitseed", 42)

experiment.logother("gputype", "A100")

import cometml

experiment = cometml.Experiment(

    projectname="reproducibility-demo",
    logcode=True,

    logenvdetails=True,

    logenvgpu=True,

    logenvcpu=True,

    loggitmetadata=True,

    loggitpatch=True,

)

experiment.logdependency("torch", "2.1.0")
experiment.logdependency("transformers", "4.35.0")

import cometml

with cometml.Experiment(projectname="context-manager-demo") as experiment:
    experiment.logparameter("learningrate", 0.001)
    experiment.logmetric("accuracy", 0.95)

# .github/workflows/train.yml
name: Model Training
on:
  push:
    branches: [main]

jobs:
  train:
    runs-on: ubuntu-latest
    steps:
uses: actions/checkout@v4
name: Setup Python
        uses: actions/setup-python@v5

        with:
          python-version: "3.11"
name: Install dependencies
        run: pip install -r requirements.txt

name: Train model
        env:

          COMETAPIKEY: ${{ secrets.COMETAPIKEY }}

          COMETPROJECTNAME: "production-training"

        run: python train.py

import cometml

try:
    experiment = cometml.ExistingExperiment(

        previousexperiment="experiment-key-to-resume"
    )
    print("Resuming existing experiment")
except Exception:
    experiment = cometml.Experiment(projectname="robust-training")
    print("Starting new experiment")

try:
    for epoch in range(100):
        loss = 1.0 / (epoch + 1)
        experiment.logmetric("loss", loss, step=epoch)


        if epoch % 10 == 0:
            experiment.logmodel(f"checkpointepoch{epoch}", "./checkpoints/")
except Exception as e:
    experiment.logother("error", str(e))

    raise
finally:
    experiment.end()

• Use cometml.Experiment for every new experiment and always call experiment.end()
• Leverage auto-logging for supported frameworks
• Organize experiments with consistent projects, tags, and naming conventions
• Use Artifacts for dataset and model versioning
• Leverage the Model Registry for managing production models
• Integrate with CI/CD pipelines for automated training workflows
• Use the Optimizer for efficient hyperparameter tuning

By applying these practices, you can build ML workflows that are structured, reproducible, and ready for production scale.