Pixeltable: Declarative AI Data Infrastructure for Multimodal Data

If you have ever built an AI pipeline that has to process images, video, audio, or documents, you probably know the same particular kind of fatigue: your data is scattered everywhere. There is a folder with raw files, a script for resizing and converting, another folder for embedding outputs, a separate vector database, one CSV file holding model predictions, and one messy notebook trying to glue all of it together. The moment a single new piece of data arrives, you have to rerun half the pipeline by hand and pray the order is right. Pixeltable is the answer to this chaos.

In this tutorial I want to walk you through Pixeltable from scratch. We start with installation, creating a table, and inserting data, then climb up to the parts that make Pixeltable special: computed columns that update automatically, UDFs, an embedding index for similarity search, and how it handles everything from images to video frames. I am going to keep the tone relaxed so it is easy to digest, but every bit of code here is runnable.

What Pixeltable Is and Why Declarative Matters

Pixeltable is a Python library for AI data infrastructure that is declarative by design. What "declarative" means here is that you describe what you want (for example, "this column contains thumbnails of the images in that column") rather than writing out step by step when and how those thumbnails get made. Pixeltable handles the execution, the caching, and deciding when something needs to be recomputed.

The core idea is simple but powerful: all your data, whether it is text, images, video, audio, or documents, lives inside a table. This table is like a database table, with rows and columns. The difference is that a single column can natively store heavy data types like images or video, not just numbers and strings. On top of that, you can add a computed column, which is a column whose value is derived from other columns through a function. When new data arrives, the computed column fills itself in automatically. When you add a new computed column to a table that already has data, Pixeltable immediately backfills it for every existing row.

Why does this matter? Because AI work is 90 percent repetitive data transformation: resize images, extract video frames, call a model for inference, build embeddings, and store the index. If all of those steps can be expressed as columns, you no longer need to write loops, you no longer need to manage cache files by hand, and you no longer have to fear forgetting to rerun one of the stages. Pixeltable also automatically stores your data along with its versions, so every change is traceable and can be rolled back.

Compare this with the old way. Normally I have one preprocessing script, one inference script, one vector database, and one artifacts directory. Four different systems that I have to keep in sync myself. In Pixeltable, those four things become one table with a few computed columns plus one embedding index. One source of truth, and everything is connected.

Key Concepts to Understand

Before we code, here are a few terms that will keep showing up:

• Table: the main container for data, similar to a SQL table but supporting multimodal types.
• Column: either a regular column (data you insert) or a computed column (derived automatically).
• Insert: how you add new data to a table. Each insert triggers recomputation of the related computed columns.
• UDF (User-Defined Function): a plain Python function you register with Pixeltable to use in computed columns.
• Embedding index: a vector index attached to a column, used for similarity search.
• View and iterator: a way to "explode" one row into many rows, for example turning one video into many frames.

Now let us jump into the practice.

Installation

Pixeltable requires Python 3.9 or higher. I recommend creating a virtual environment first to keep things clean, then installing with pip.

# Create and activate a virtual environment (run in your terminal)
python -m venv venv
source venv/bin/activate      # Linux / macOS
venv\Scripts\activate         # Windows

Install Pixeltable
pip install pixeltable

Once it is installed, try importing it to make sure everything is fine:

import pixeltable as pxt

print(pxt.version)