Complete Jupyter AI Tutorial: Integrate LLMs Directly Into Your Notebooks
Jupyter Notebook has been the primary tool for data scientists and ML engineers for years. With Jupyter AI, you can now integrate Large Language Models (LLMs) directly into your Jupyter workflow. Jupyter AI is an official extension from Project Jupyter that adds generative AI capabilities to JupyterLab, including magic commands for cell-level interaction, a chat interface for contextual conversations, and RAG capabilities for querying your local documents.
In this tutorial, we'll cover installation, configuration, basic and advanced usage of Jupyter AI, and best practices to maximize your productivity.
Why Jupyter AI?
Before we begin, let's understand why Jupyter AI is worth using over alternatives:
- Native Integration: Runs directly in JupyterLab without switching applications
- Multi-Provider: Supports OpenAI, Anthropic, Cohere, HuggingFace, Google, Mistral, and local models via Ollama
- Magic Commands: Use
%%aiand%aimagics directly in notebook cells - Chat Interface: Integrated chat panel in the JupyterLab sidebar
- Built-in RAG:
/learncommand to index documents and/askto query local content - Notebook Generation:
/generatecommand to create complete notebooks from text descriptions - Open Source: Developed by the Project Jupyter community
Installation
Prerequisites
Ensure you have Python 3.8+ and JupyterLab 4.0+ installed.
# Create a new virtual environment (optional but recommended)
python -m venv jupyter-ai-env
source jupyter-ai-env/bin/activate # Linux/Mac
jupyter-ai-env\Scripts\activate # Windows
Install JupyterLab if not already present
pip install jupyterlab>=4.0
Install Jupyter AI
# Install the main jupyter-ai package (includes chat UI and magic commands)
pip install jupyter-ai
Or install only magic commands without the chat UI
pip install jupyter-ai-magics
Install Provider Dependencies
Jupyter AI requires additional packages depending on which LLM provider you want to use:
# For OpenAI (GPT-4, GPT-4o, etc.)
pip install langchain-openai
For Anthropic (Claude)
pip install langchain-anthropic
For Google (Gemini)
pip install langchain-google-genai
For HuggingFace Hub
pip install langchain-huggingface
For Mistral AI
pip install langchain-mistralai
For local models via Ollama
pip install langchain-ollama
For Cohere
pip install langchain-cohere
Configure API Keys
Set API keys as environment variables before launching JupyterLab:
# OpenAI
export OPENAIAPIKEY="sk-..."
Anthropic
export ANTHROPICAPIKEY="sk-ant-..."
Google
export GOOGLEAPIKEY="AIza..."
Mistral
export MISTRALAPIKEY="..."
Or save to .env file for automatic loading
echo 'OPENAIAPIKEY=sk-...' >> ~/.bashrc
source ~/.bashrc
Launch JupyterLab:
jupyter lab
Basic Usage: Magic Commands
Cell Magic %%ai
The %%ai cell magic is the fastest way to interact with LLMs directly from a notebook cell. Write %%ai on the first line of a cell, followed by the provider and model name.
%%ai openai-chat:gpt-4o
Explain the concept of gradient descent in machine learning
in simple, easy-to-understand terms.
The output will be displayed directly below the cell as formatted markdown.
Specifying Output Format
You can specify the output format using the -f flag:
%%ai openai-chat:gpt-4o -f code
Write a Python function to calculate the moving average
of a list of numbers with a configurable window size.
Available formats:
text- Plain text (default)code- Code blockmarkdownormd- Markdown formattedmath- LaTeX mathhtml- HTML renderedjson- JSON formatted
Line Magic %ai
The %ai line magic is used for quick single-line queries:
# Quick question
%ai openai-chat:gpt-4o "What's the difference between a list and a tuple in Python?"
Registering Model Aliases
To avoid typing the full provider and model name repeatedly:
# Register an alias
%ai register mymodel openai-chat:gpt-4o
Use the alias
%%ai mymodel
Create an example of pandas groupby with multiple column aggregations.
Using Python Variables in Prompts
You can interpolate Python variables into prompts using curly braces:
# Define variables
datasetinfo = "Dataset contains 10000 rows with columns: age, income, education, purchaseamount"
targetvariable = "purchaseamount"
%%ai openai-chat:gpt-4o
Based on the following dataset information: {datasetinfo}
Suggest the best feature engineering approaches
for predicting {targetvariable}.
Interpolating Previous Cell Output
Jupyter AI can reference output from previous cells using In and Out:
# Cell 1: Run code
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
df.describe()
# Cell 2: Ask AI to analyze the output
%%ai openai-chat:gpt-4o
Analyze the following descriptive statistics and provide insights:
{Out[1]}
Basic Usage: Chat Interface
Opening the Chat Panel
Click the Jupyter AI chat icon in the left sidebar of JupyterLab. The chat panel will open on the right side.
Selecting a Model
At the top of the chat panel, click the gear icon to open settings. Here you can configure:
gpt-4o, claude-3-opus)text-embedding-3-small)Contextual Conversations
The chat interface supports multi-turn conversations. The AI remembers previous conversation context:
User: I'm working on a sentiment classification project using BERT.
AI: [provides suggestions]
User: How should I handle class imbalance in my dataset?
AI: [responds in the context of BERT sentiment classification]
Sending Code Selections to Chat
You can send highlighted code to the chat:
This is particularly useful for debugging or requesting explanations of specific code.
Advanced Features
RAG with the /learn Command
The /learn feature allows you to index local documents so the AI can answer questions based on their content.
# In the chat panel, type:
/learn /path/to/your/documents/
Or learn a specific file
/learn /home/user/project/README.md
Jupyter AI will:
After learning is complete, use /ask to query:
/ask How do I configure the database connection in this project?
/ask What does the DataProcessor class do?
Generate Notebooks with /generate
The /generate command creates complete notebooks from text descriptions:
/generate Create a notebook for exploratory data analysis (EDA)
on the Titanic dataset, including distribution visualizations,
feature correlations, and missing value handling
Jupyter AI will produce a new notebook with cells containing code and markdown explanations.
Using Local Models via Ollama
For privacy or offline usage, you can use local models:
# Install and run Ollama first
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.1
ollama pull nomic-embed-text # for embeddings
Then in Jupyter AI:
%%ai ollama:llama3.1
Implement a binary search tree in Python
with insert, search, and delete methods.
For the chat interface, select ollama as the provider in settings.
Custom Model Provider
You can create custom providers for models not supported by default:
from jupyteraimagics import BaseProvider
from jupyteraimagics.providers import EnvAuthStrategy
from langchainopenai import ChatOpenAI
class CustomProvider(BaseProvider):
id = "custom-provider"
name = "My Custom Provider"
modelidkey = "modelname"
models = ["my-model-v1", "my-model-v2"]
authstrategy = EnvAuthStrategy(
name="CUSTOMAPIKEY",
keywordparam="apikey"
)
pypipackagedeps = ["langchain-openai"]
@classmethod
def getchatmodel(cls, args, kwargs):
return ChatOpenAI(
baseurl="https://my-api.example.com/v1",
kwargs
)
Save this file and register it as an entry point in your setup.py or pyproject.toml.
Prompt Templates
Create reusable prompt templates for frequently performed tasks:
# Define a template
codereviewtemplate = """
Perform a code review for the following code:
python
{code}
Focus on:
Potential bugs
Performance issues
Readability
Python best practices
Provide a score from 1-10 and specific improvement suggestions.
"""
Use with magic command
code = """
def processdata(data):
result = []
for i in range(len(data)):
if data[i] != None:
result.append(data[i] 2)
return result
"""
%%ai openai-chat:gpt-4o
{codereviewtemplate.format(code=code)}
Error Debugging with AI
Use Jupyter AI for quick error debugging:
# Cell that produces an error
import pandas as pd
df = pd.readcsv("data.csv")
result = df.groupby("category").agg({"value": "mean", "count": "sum"})
# If there's an error, send the traceback to AI
%%ai openai-chat:gpt-4o -f markdown
I got the following error while running a pandas groupby:
{Out[-1]}
My data has columns: category (string), value (float), count (int).
What's wrong and how do I fix it?
Advanced Configuration
Configuration File
Jupyter AI stores configuration in ~/.jupyter/jupyteraiconfig.json:
{
"languagemodel": {
"provider": "openai-chat",
"modelid": "gpt-4o"
},
"embeddingmodel": {
"provider": "openai",
"modelid": "text-embedding-3-small"
},
"apikeys": {},
"chunksize": 2000,
"chunkoverlap": 200
}
RAG Configuration
Customize chunking parameters for better RAG results:
# In a notebook or startup script
from jupyterai.config import AiConfig
config = AiConfig()
config.chunksize = 1500 # Chunk size (characters)
config.chunkoverlap = 200 # Overlap between chunks
config.maxresults = 5 # Number of chunks to retrieve
Using Multiple Providers Simultaneously
You can use different providers for different tasks in the same notebook:
# Use GPT-4o for complex reasoning
%%ai openai-chat:gpt-4o
Design a microservice architecture for a real-time recommendation system.
Use Claude for code generation
%%ai anthropic-chat:claude-sonnet-4-6
Implement collaborative filtering using matrix factorization in Python.
Use local model for simple tasks
%%ai ollama:llama3.1
Convert this list comprehension to a regular for loop:
result = [x2 for x in range(10) if x % 2 == 0]
Best Practices
1. Choose the Right Model for the Task
- Code generation: Use models strong at coding (GPT-4o, Claude Sonnet)
- Data analysis: General purpose models (GPT-4o, Claude)
- Light tasks/fast iteration: Local models (Llama 3.1 via Ollama) or smaller models
- Embeddings for RAG**:
text-embedding-3-small(OpenAI) ornomic-embed-text(local)
2. Provide Sufficient Context
# Less effective
%%ai openai-chat:gpt-4o
Fix this code.
More effective
%%ai openai-chat:gpt-4o
The following code should calculate the weighted average of the 'score' column
based on the 'weight' column per group in the 'category' column.
But the result is always NaN for some groups.
DataFrame: {df.head().tostring()}
Dtypes: {df.dtypes.tostring()}
3. Use RAG for Large Projects
For projects with many files and documentation:
/learn /path/to/project/src/
/learn /path/to/project/docs/
Then ask specific questions:
/ask How does data flow from the /process API endpoint to the database?
/ask Which function is responsible for data validation?
4. Build Consistent Workflows
Example data analysis workflow with Jupyter AI:
# Step 1: Load and inspect data
import pandas as pd
df = pd.readcsv("salesdata.csv")
df.info()
# Step 2: Ask AI to analyze the data structure
%%ai openai-chat:gpt-4o -f markdown
Based on the following dataset info:
{Out[1]}
Suggest the EDA and preprocessing steps that need to be performed.
# Step 3: Ask AI to generate preprocessing code
%%ai openai-chat:gpt-4o -f code
Based on the suggestions above, write Python code to:
Handle missing values in numeric columns with median
Encode categorical variables
Remove outliers using the IQR method
5. Mind Security and Privacy
- Never send sensitive data (PII, credentials) to cloud API providers
- Use local models (Ollama) for confidential data
- Review AI-generated code before executing it
- Don't store API keys directly in notebooks
6. Leverage Output Formats
# For documentation: use markdown
%%ai openai-chat:gpt-4o -f md
Document the following DataPipeline class with all its methods.
For presentations: use HTML
%%ai openai-chat:gpt-4o -f html
Create a comparison table between Random Forest, XGBoost, and LightGBM.
For papers/reports: use math
%%ai openai-chat:gpt-4o -f math
Write the loss function formula for binary cross-entropy.
Integration with ML Workflows
Experiment Tracking with AI Assistance
import mlflow
Ask AI to suggest hyperparameters
%%ai openai-chat:gpt-4o -f json
Given a dataset with 50000 samples, 20 numerical features,
and a binary classification target with a 70:30 ratio,
suggest a hyperparameter grid for Random Forest in JSON format:
{"nestimators": [...], "maxdepth": [...], "minsamplessplit": [...]}
# Parse response and run experiments
import json
params = json.loads(Out[-1])
with mlflow.startrun():
for nest in params["nestimators"]:
for depth in params["maxdepth"]:
# train and log...
pass
Data Quality Checks
# Run profiling
profile = df.describe(include='all').tostring()
%%ai openai-chat:gpt-4o -f markdown
Analyze the following data profile and identify potential data quality issues:
{profile}
Provide specific recommendations for each issue found.
Troubleshooting
Jupyter AI Not Showing in Sidebar
# Ensure the extension is properly installed
jupyter labextension list
jupyter server extension list
If not showing, try rebuilding
jupyter lab build
Error "No module named 'langchain_openai'"
# Install the required provider
pip install langchain-openai langchain-anthropic
Model Not Available
# List all available models
%ai list
Output will show all detected providers and models
Rate Limiting
If you frequently encounter rate limit errors:
# Use cheaper models for iteration
%%ai openai-chat:gpt-4o-mini
Quick question: what's the pandas syntax for merging two dataframes?
Save expensive models for complex tasks
%%ai anthropic-chat:claude-sonnet-4-6
Design and implement a complete ETL pipeline...
Conclusion
Jupyter AI transforms how we work with notebooks. By integrating LLMs directly into the Jupyter workflow, you can:
- Accelerate data exploration by asking questions directly in notebooks
- Get code generation without switching applications
- Leverage RAG to understand project codebases and documentation
- Use local models for sensitive data
- Create complete notebooks from natural language descriptions
Start by installing jupyter-ai, configure your preferred provider, and explore the magic commands and chat interface. Over time, you'll discover the most effective usage patterns for your specific workflow.
Jupyter AI is available on GitHub and full documentation can be found on the official project page.