FiftyOne by Voxel51: The Complete Guide to Dataset Curation and Model Evaluation for Computer Vision

# Tutorial Lengkap FiftyOne by Voxel51: Tool Open-Source untuk Dataset dan Model Computer Vision **Penulis: Ruby Abdullah** **Tanggal: 9 September 2026** ## Pendahuluan Dalam dunia computer vision,...

By Ruby Abdullah · · tutorial
FiftyOneComputer VisionDataset CurationModel EvaluationVoxel51

FiftyOne by Voxel51: The Complete Guide to Dataset Curation and Model Evaluation for Computer Vision

Introduction

Building high-quality computer vision models is not just about choosing the right architecture or training for more epochs. In practice, the quality of your dataset determines the ceiling of your model's performance. Noisy labels, duplicate images, class imbalances, and edge cases buried deep in your data can silently degrade your results. Yet most practitioners spend far more time tuning hyperparameters than inspecting their data.

FiftyOne, developed by Voxel51, is an open-source toolkit designed to solve exactly this problem. It provides a powerful set of tools for exploring, visualizing, curating, and evaluating computer vision datasets and model predictions. Whether you are working on image classification, object detection, instance segmentation, or keypoint estimation, FiftyOne gives you a systematic way to understand what is happening in your data and your models.

In this tutorial, we will walk through FiftyOne from installation to advanced usage. You will learn how to load datasets from various sources, launch the interactive App to visually explore your data, evaluate model predictions against ground truth, identify annotation errors, use embeddings for dataset curation, and integrate FiftyOne into your existing PyTorch, TensorFlow, and Hugging Face workflows. Every section includes runnable code examples that you can copy and execute in your own environment.

By the end of this guide, you will have a practical understanding of how FiftyOne fits into a modern computer vision pipeline and how it can help you build better datasets and, consequently, better models.

Installation

FiftyOne runs on Python 3.8 and above. The recommended way to install it is via pip. It is best practice to use a virtual environment to avoid dependency conflicts with other projects.

Basic Installation

# Create and activate a virtual environment

python -m venv fiftyone-env

source fiftyone-env/bin/activate # On Windows: fiftyone-env\Scripts\activate

Install FiftyOne

pip install fiftyone

Installation with Optional Dependencies

FiftyOne has several optional integrations. If you plan to work with specific frameworks, you can install the relevant extras.

# Install with all optional dependencies

pip install fiftyone[all]

Or install specific extras

pip install fiftyone torch torchvision # For PyTorch integration

pip install fiftyone tensorflow # For TensorFlow integration

pip install fiftyone transformers # For Hugging Face integration

Verifying the Installation

After installation, verify that everything works by launching a quick test.

import fiftyone as fo

Print the version

print(fo.version)

Load a small sample dataset to verify

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=10)

print(dataset)

If the installation is successful, you should see a summary of the dataset printed to your terminal. FiftyOne ships with a built-in dataset zoo that includes popular datasets like COCO, CIFAR-10, ImageNet, and more, making it easy to get started without downloading anything manually.

Launching the FiftyOne App

FiftyOne includes a browser-based application for visual data exploration. You can launch it from Python.

import fiftyone as fo

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=50)

Launch the App

session = fo.launchapp(dataset)

This opens a web interface at http://localhost:5151 where you can browse images, filter by labels, view model predictions, and interactively explore your dataset. The App is one of FiftyOne's most powerful features, and we will reference it throughout this tutorial.

Basic Usage

Loading Your Own Dataset

While the dataset zoo is convenient for experimentation, you will typically want to work with your own data. FiftyOne supports loading data from a variety of formats.

Loading Images from a Directory

The simplest case is loading a directory of images without any labels.

import fiftyone as fo

dataset = fo.Dataset.fromdir(

datasetdir="/path/to/your/images",

datasettype=fo.types.ImageDirectory,

name="my-image-dataset",

)

print(dataset)

print(dataset.first())

Loading a COCO-Format Dataset

If your dataset uses the COCO format (common for object detection and segmentation), FiftyOne can parse it directly.

import fiftyone as fo

dataset = fo.Dataset.fromdir(

datasettype=fo.types.COCODetectionDataset,

datapath="/path/to/images",

labelspath="/path/to/annotations.json",

name="my-coco-dataset",

)

print(dataset)

Loading a YOLO-Format Dataset

For datasets in YOLO format, FiftyOne handles the conversion seamlessly.

import fiftyone as fo

dataset = fo.Dataset.fromdir(

datasettype=fo.types.YOLOv5Dataset,

datasetdir="/path/to/yolo-dataset",

split="train",

name="my-yolo-dataset",

)

Loading from CSV or Custom Formats

For custom formats, you can build a dataset sample by sample.

import fiftyone as fo

import csv

dataset = fo.Dataset(name="custom-dataset")

with open("annotations.csv", "r") as f:

reader = csv.DictReader(f)

for row in reader:

sample = fo.Sample(filepath=row["imagepath"])

# Add a classification label

sample["groundtruth"] = fo.Classification(label=row["label"])

# Add custom metadata

sample["confidencescore"] = float(row["score"])

dataset.addsample(sample)

print(dataset)

Exploring Data in the App

Once your dataset is loaded, the FiftyOne App becomes your primary tool for visual exploration.

import fiftyone as fo

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=200)

session = fo.launchapp(dataset)

In the App, you can:

  • Browse images in a grid layout and click on any image to see its full details.
  • Filter by label to show only images containing a specific class (e.g., "person" or "car").
  • Sort by metadata fields such as file size, number of detections, or any custom field you add.
  • Select samples to create subsets for further analysis.

Using Views to Filter Data

Views are a core concept in FiftyOne. A view is a filtered, sorted, or transformed perspective on your dataset. Views do not copy data; they are lightweight references.

import fiftyone as fo

from fiftyone import ViewField as F

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=200)

Filter to samples that contain at least one "person" detection

view = dataset.filterlabels(

"groundtruth", F("label") == "person"

)

print(f"Samples with 'person': {len(view)}")

Filter to high-confidence predictions only

highconfview = dataset.filterlabels(

"predictions", F("confidence") > 0.9

)

Sort by number of detections (descending)

sortedview = dataset.sortby(

F("groundtruth.detections").length(), reverse=True

)

Show the filtered view in the App

session = fo.launchapp(view)

Tagging Samples

Tags allow you to mark samples for later reference, such as flagging problematic images for review.

# Tag specific samples

for sample in dataset.take(10):

sample.tags.append("needs-review")

sample.save()

Retrieve tagged samples

reviewview = dataset.matchtags("needs-review")

print(f"Samples needing review: {len(reviewview)}")

Working with Different Label Types

FiftyOne supports all common computer vision label types.

import fiftyone as fo

sample = fo.Sample(filepath="/path/to/image.jpg")

Classification

sample["classification"] = fo.Classification(

label="cat", confidence=0.95

)

Object detection (bounding box in [x, y, width, height] relative format)

sample["detections"] = fo.Detections(detections=[

fo.Detection(

label="cat",

boundingbox=[0.1, 0.2, 0.3, 0.4], # relative coordinates

confidence=0.92,

),

fo.Detection(

label="dog",

boundingbox=[0.5, 0.1, 0.4, 0.6],

confidence=0.87,

),

])

Semantic segmentation

import numpy as np

mask = np.zeros((480, 640), dtype=np.uint8)

mask[100:300, 200:500] = 1 # Region labeled as class 1

sample["segmentation"] = fo.Segmentation(mask=mask)

Keypoints

sample["keypoints"] = fo.Keypoints(keypoints=[

fo.Keypoint(

label="person",

points=[(0.5, 0.1), (0.45, 0.3), (0.55, 0.3)], # relative coords

),

])

Advanced Usage

Evaluating Model Predictions

One of FiftyOne's most valuable features is its ability to evaluate model predictions against ground truth annotations. This goes beyond simple aggregate metrics by letting you drill into per-sample and per-detection results.

Object Detection Evaluation

import fiftyone as fo

import fiftyone.zoo as foz

Load a dataset with both ground truth and predictions

dataset = foz.loadzoodataset("quickstart", maxsamples=200)

Run evaluation (COCO-style mAP)

results = dataset.evaluatedetections(

"predictions", # field containing model predictions

gtfield="groundtruth", # field containing ground truth

evalkey="eval", # key to store per-sample results

computemAP=True,

)

Print aggregate metrics

print(f"mAP: {results.mAP():.4f}")

results.printreport()

The evaluation results are stored directly on each sample and each detection, allowing you to filter and visualize specific types of errors.

from fiftyone import ViewField as F

Find all false positive detections

fpview = dataset.filterlabels("predictions", F("eval") == "fp")

print(f"Samples with false positives: {len(fpview)}")

Find all false negative (missed) detections

fnview = dataset.filterlabels("groundtruth", F("eval") == "fn")

print(f"Samples with missed detections: {len(fnview)}")

Visualize false positives in the App

session = fo.launchapp(fpview)

Classification Evaluation

import fiftyone as fo

Assuming your dataset has "groundtruth" and "predictions" classification fields

results = dataset.evaluateclassifications(

"predictions",

gtfield="groundtruth",

evalkey="eval",

)

Print a classification report

results.printreport()

Plot a confusion matrix

plot = results.plotconfusionmatrix()

plot.show()

Finding Annotation Errors

Annotation errors are one of the most common sources of model degradation, yet they are notoriously difficult to find manually. FiftyOne provides tools to systematically identify mislabeled samples.

Using Model Predictions to Flag Errors

A practical approach is to use a well-trained model's predictions to flag samples where the model confidently disagrees with the ground truth label.

import fiftyone as fo

import fiftyone.brain as fob

from fiftyone import ViewField as F

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=500)

Compute mistakenness scores

This analyzes how "mistaken" each label might be based on model predictions

fob.computemistakenness(

dataset,

"predictions", # model predictions field

labelfield="groundtruth", # ground truth field

)

Sort by mistakenness (most likely errors first)

mistakeview = dataset.sortby("mistakenness", reverse=True)

View the top 50 most likely annotation errors

topmistakes = mistakeview.take(50)

session = fo.launchapp(topmistakes)

The mistakenness score ranks samples by how likely it is that the ground truth label is incorrect. Reviewing the top-ranked samples often reveals genuine annotation errors that would be nearly impossible to find by browsing randomly.

Finding Duplicate and Near-Duplicate Images

Duplicate images in your dataset can bias training and inflate evaluation metrics.

import fiftyone as fo

import fiftyone.brain as fob

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=500)

Compute image similarity using embeddings

fob.computesimilarity(

dataset,

brainkey="imgsim",

)

Find near-duplicate images

dupview = fob.computeexactduplicates(dataset)

print(f"Found {len(dupview)} exact duplicates")

For near-duplicates, use similarity index

results = dataset.sortbysimilarity(

dataset.first().id, # reference image

brainkey="imgsim",

k=10, # top 10 most similar

)

session = fo.launchapp(results)

Dataset Curation with Embeddings

Embeddings provide a numerical representation of your images that captures semantic content. FiftyOne uses embeddings for visualization, similarity search, and dataset curation.

Computing and Visualizing Embeddings

import fiftyone as fo

import fiftyone.brain as fob

import fiftyone.zoo as foz

dataset = foz.loadzoodataset("quickstart", maxsamples=500)

Compute embeddings using a pre-trained model

fob.computevisualization(

dataset,

brainkey="imgviz",

method="umap", # Dimensionality reduction method

)

Launch the App - embeddings plot will be available

session = fo.launchapp(dataset)

In the App, click the "Embeddings" tab to see the 2D visualization

The embeddings visualization reveals the structure of your dataset. Clusters correspond to visually similar images, and outliers may indicate unusual or mislabeled samples.

Finding Representative Subsets

When you have a very large dataset but limited labeling budget, you want to select a diverse, representative subset for annotation.

import fiftyone as fo

import fiftyone.brain as fob

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=1000)

Compute representativeness scores

fob.computerepresentativeness(dataset)

Select the most representative samples

representativeview = dataset.sortby("representativeness", reverse=True)

toprepresentative = representativeview.take(100)

These 100 samples best represent the diversity of your dataset

session = fo.launchapp(toprepresentative)

Finding Unique Samples

Conversely, you might want to find the most unique or unusual samples in your dataset.

import fiftyone.brain as fob

Compute uniqueness scores

fob.computeuniqueness(dataset)

View the most unique samples

uniqueview = dataset.sortby("uniqueness", reverse=True)

session = fo.launchapp(uniqueview.take(50))

View the least unique (most redundant) samples

redundantview = dataset.sortby("uniqueness")

session.view = redundantview.take(50)

FiftyOne is designed to fit into existing ML workflows. It integrates seamlessly with PyTorch, TensorFlow, and Hugging Face.

PyTorch Integration

You can convert a FiftyOne dataset into a PyTorch DataLoader for training or use PyTorch models to generate predictions that you then analyze in FiftyOne.

import fiftyone as fo

import fiftyone.utils.torch as fout

import torch

import torchvision

from torchvision import transforms

Load dataset

dataset = fo.zoo.loadzoodataset("cifar10", split="test", maxsamples=500)

Convert to PyTorch DataLoader

transform = transforms.Compose([

transforms.Resize((224, 224)),

transforms.ToTensor(),

transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),

])

torchdataset = fout.TorchImageClassificationDataset(

dataset,

labelfield="groundtruth",

transform=transform,

)

dataloader = torch.utils.data.DataLoader(

torchdataset, batchsize=32, shuffle=False

)

Run inference with a pre-trained model

model = torchvision.models.resnet50(pretrained=True)

model.eval()

Add predictions back to FiftyOne

with torch.nograd():

for batchidx, (images, ) in enumerate(dataloader):

outputs = model(images)

probs = torch.nn.functional.softmax(outputs, dim=1)

confs, preds = torch.max(probs, 1)

# Map predictions back to samples

start = batchidx 32

for i, (pred, conf) in enumerate(zip(preds, confs)):

sample = dataset.skip(start + i).first()

sample["torchprediction"] = fo.Classification(

label=str(pred.item()),

confidence=conf.item(),

)

sample.save()

Using FiftyOne's Model Zoo with PyTorch

FiftyOne also provides a model zoo with pre-trained models ready for inference.

import fiftyone as fo

import fiftyone.zoo as foz

dataset = foz.loadzoodataset("quickstart", maxsamples=100)

Load a pre-trained detection model from the zoo

model = foz.loadzoomodel("faster-rcnn-resnet50-fpn-coco-torch")

Apply the model to your dataset

dataset.applymodel(model, labelfield="zoopredictions")

Compare zoo predictions with existing predictions

session = fo.launchapp(dataset)

TensorFlow Integration

FiftyOne works with TensorFlow datasets and models in a similar fashion.

import fiftyone as fo

import fiftyone.utils.tf as fout

Load a FiftyOne dataset

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=100)

Export to TFRecords for TensorFlow training

dataset.export(

exportdir="/path/to/tfrecords",

datasettype=fo.types.TFObjectDetectionDataset,

labelfield="groundtruth",

)

Or use TensorFlow Hub models for inference

import fiftyone.zoo as foz

model = foz.loadzoomodel("ssd-mobilenet-v2-fpnlite-coco-tf2")

dataset.applymodel(model, labelfield="tfpredictions")

session = fo.launchapp(dataset)

Hugging Face Integration

FiftyOne integrates with the Hugging Face ecosystem, letting you load datasets from the Hugging Face Hub and use Transformers models for inference.

import fiftyone as fo

import fiftyone.utils.huggingface as fouh

Load a dataset from Hugging Face Hub

dataset = fouh.loadfromhub(

"detection-datasets/coco",

name="coco-hub-dataset",

maxsamples=200,

)

print(dataset)

session = fo.launchapp(dataset)

You can also use Hugging Face Transformers models to generate predictions.

import fiftyone as fo

from transformers import pipeline

from PIL import Image

dataset = fo.zoo.loadzoodataset("quickstart", maxsamples=50)

Load a Hugging Face object detection pipeline

detector = pipeline("object-detection", model="facebook/detr-resnet-50")

for sample in dataset.itersamples(progress=True):

img = Image.open(sample.filepath)

results = detector(img)

detections = []

w, h = img.size

for r in results:

box = r["box"]

# Convert from absolute to relative coordinates

x = box["xmin"] / w

y = box["ymin"] / h

bw = (box["xmax"] - box["xmin"]) / w

bh = (box["ymax"] - box["ymin"]) / h

detections.append(fo.Detection(

label=r["label"],

boundingbox=[x, y, bw, bh],

confidence=r["score"],

))

sample["hfpredictions"] = fo.Detections(detections=detections)

sample.save()

session = fo.launchapp(dataset)

Pushing Datasets to Hugging Face Hub

You can also push your curated FiftyOne datasets back to the Hugging Face Hub for sharing and reproducibility.

import fiftyone.utils.huggingface as fouh

Push your curated dataset to the Hub

fouh.pushtohub(

dataset,

"your-username/curated-detection-dataset",

private=True,

)

Comparing Multiple Models

A common workflow is comparing predictions from multiple models to understand their relative strengths and weaknesses.

import fiftyone as fo

import fiftyone.zoo as foz

dataset = foz.loadzoodataset("quickstart", maxsamples=200)

Apply two different models

modela = foz.loadzoomodel("faster-rcnn-resnet50-fpn-coco-torch")

modelb = foz.loadzoomodel("ssd-mobilenet-v2-fpnlite-coco-tf2")

dataset.applymodel(modela, labelfield="modelapreds")

dataset.applymodel(modelb, labelfield="modelbpreds")

Evaluate both models

resultsa = dataset.evaluatedetections(

"modelapreds", gtfield="groundtruth",

evalkey="evala", computemAP=True,

)

resultsb = dataset.evaluatedetections(

"modelbpreds", gtfield="groundtruth",

evalkey="evalb", computemAP=True,

)

print(f"Model A mAP: {resultsa.mAP():.4f}")

print(f"Model B mAP: {resultsb.mAP():.4f}")

Find samples where Model A is correct but Model B fails

from fiftyone import ViewField as F

acorrectbwrong = dataset.match(

(F("modelapreds.detections").length() > 0) &

(F("evalbfp") > F("evalafp"))

)

session = fo.launchapp(acorrectbwrong)

Best Practices

1. Always Persist Your Datasets

By default, FiftyOne datasets are persistent and stored in a local MongoDB database. Give your datasets meaningful names so you can reload them later.

import fiftyone as fo

Create a named, persistent dataset

dataset = fo.Dataset(name="project-v2-train", persistent=True)

Later, reload it

dataset = fo.loaddataset("project-v2-train")

List all available datasets

print(fo.listdatasets())

2. Use Views Instead of Copying Data

Views are lightweight and do not duplicate data. Use them liberally to slice and filter your dataset for analysis.

# Good: use views for filtering

smallobjects = dataset.filterlabels(

"groundtruth",

(F("boundingbox")[2] F("boundingbox")[3]) < 0.01

)

Avoid: creating new datasets for every filter operation

This wastes storage and is harder to manage

3. Track Dataset Versions

When you modify labels or remove samples, keep track of what changed.

# Use tags to track dataset versions

for sample in dataset:

sample.tags.append("v1.0")

sample.save()

After cleaning, tag the new version

cleanedview = dataset.matchtags("v1.0").match(F("quality") == "good")

for sample in cleanedview:

sample.tags.append("v1.1-cleaned")

sample.save()

4. Automate Quality Checks

Build quality check pipelines that run automatically before training.

import fiftyone as fo

import fiftyone.brain as fob

def runqualitychecks(dataset):

"""Run a standard suite of quality checks on a dataset."""

# Check for duplicates

fob.computeexactduplicates(dataset)

# Compute uniqueness

fob.computeuniqueness(dataset)

# Flag low-uniqueness samples

redundant = dataset.match(F("uniqueness") < 0.1)

print(f"Warning: {len(redundant)} potentially redundant samples found")

# Check for missing labels

unlabeled = dataset.match(~F("groundtruth").exists())

print(f"Warning: {len(unlabeled)} samples have no labels")

# Check label distribution

counts = dataset.countvalues("groundtruth.detections.label")

print("Label distribution:")

for label, count in sorted(counts.items(), key=lambda x: -x[1]):

print(f" {label}: {count}")

return dataset

dataset = fo.loaddataset("my-project")

runqualitychecks(dataset)

5. Export Clean Datasets for Training

After curation, export your cleaned dataset in the format your training pipeline expects.

import fiftyone as fo

dataset = fo.loaddataset("my-project")

Create a clean view (exclude flagged samples)

cleanview = dataset.matchtags("approved")

Export to COCO format

cleanview.export(

exportdir="/path/to/clean-dataset",

datasettype=fo.types.COCODetectionDataset,

labelfield="groundtruth",

)

Export to YOLO format

cleanview.export(

exportdir="/path/to/clean-dataset-yolo",

datasettype=fo.types.YOLOv5Dataset,

labelfield="groundtruth",

)

6. Leverage the FiftyOne App During Model Development

Do not treat data exploration as a one-time activity. Launch the App frequently during model development to inspect failure cases, compare model iterations, and track improvements over time. The visual feedback loop is one of the most effective ways to improve model performance.

7. Use Delegated Operations for Large Datasets

When working with datasets containing hundreds of thousands or millions of samples, use FiftyOne's delegated operations to run computations asynchronously.

import fiftyone as fo

import fiftyone.brain as fob

dataset = fo.loaddataset("large-dataset")

For large datasets, delegate heavy computations

fob.computevisualization(

dataset,

brainkey="embeddings",

method="umap",

numworkers=8, # Parallelize computation

)

Conclusion

FiftyOne fills a critical gap in the computer vision workflow. While there are many tools for model training and deployment, few provide robust support for the data-centric tasks that often determine whether a project succeeds or fails. By giving you the ability to visualize, explore, evaluate, and curate your datasets systematically, FiftyOne transforms data quality from a guessing game into an engineering discipline.

In this tutorial, we covered the full spectrum of FiftyOne's capabilities. We started with installation and basic dataset loading, moved through visual exploration with the App and views, and then advanced to model evaluation, annotation error detection, and embedding-based curation. We also demonstrated how FiftyOne integrates with PyTorch, TensorFlow, and Hugging Face, ensuring that it fits into whatever ML stack you are already using.

The key takeaways are:

  • Data quality matters more than model complexity. FiftyOne gives you the tools to systematically improve your data.
  • Visual exploration is not optional. The FiftyOne App lets you see what aggregate metrics cannot tell you.
  • Evaluation should be granular. Per-sample and per-detection evaluation results help you understand exactly where your model fails.
  • Annotation errors are common and findable. The mistakenness computation can surface labeling mistakes that would otherwise go unnoticed.
  • Embeddings unlock dataset curation at scale. Similarity search, uniqueness scoring, and representativeness help you build better datasets with less labeling effort.
  • Integration is seamless. FiftyOne works with the frameworks and formats you already use, so adopting it does not require rearchitecting your pipeline.

To continue learning, visit the official FiftyOne documentation at docs.voxel51.com, explore the FiftyOne Tutorials repository on GitHub, and join the Voxel51 community on Slack. The open-source community around FiftyOne is active and welcoming, and new features and integrations are added regularly.

Start by loading one of your existing datasets into FiftyOne today. Spend thirty minutes exploring it in the App. You will almost certainly discover something about your data that you did not know before, and that discovery is the first step toward building a better model.

Related Articles

Surya OCR Tutorial: Document AI for Text Detection, OCR, and Layout Analysis

Tutorial Surya OCR: Document AI untuk Text Detection, OCR, dan Layout Analysis Surya adalah library Document AI open-sou...

Florence-2: Microsoft's Multi-Task Vision Foundation Model

Florence-2: Model Vision Multi-Task dari Microsoft Daftar Isi Pendahuluan Prasyarat Instalasi Memuat Model Florence-2

Supervision: Computer Vision Toolkit by Roboflow

Supervision: Toolkit Computer Vision dari Roboflow Dalam proyek computer vision, setelah model mendeteksi objek, Anda ma...

Albumentations Tutorial: Advanced Image Augmentation for Computer Vision

Albumentations - Augmentasi Gambar Tingkat Lanjut Daftar Isi Pendahuluan Prasyarat Instalasi dan Pengaturan [Memahami Pi...