Complete Jupyter AI Tutorial: Integrate LLMs Directly Into Your Notebooks

# Tutorial Lengkap Jupyter AI: Integrasikan LLM Langsung ke Notebook Anda Jupyter Notebook telah menjadi alat utama bagi data scientist dan ML engineer selama bertahun-tahun. Dengan hadirnya **Jupyte...

By Ruby Abdullah · · tutorial
Jupyter AIJupyterLabLLMData SciencePython

Complete Jupyter AI Tutorial: Integrate LLMs Directly Into Your Notebooks

Jupyter Notebook has been the primary tool for data scientists and ML engineers for years. With Jupyter AI, you can now integrate Large Language Models (LLMs) directly into your Jupyter workflow. Jupyter AI is an official extension from Project Jupyter that adds generative AI capabilities to JupyterLab, including magic commands for cell-level interaction, a chat interface for contextual conversations, and RAG capabilities for querying your local documents.

In this tutorial, we'll cover installation, configuration, basic and advanced usage of Jupyter AI, and best practices to maximize your productivity.

Why Jupyter AI?

Before we begin, let's understand why Jupyter AI is worth using over alternatives:

  • Native Integration: Runs directly in JupyterLab without switching applications
  • Multi-Provider: Supports OpenAI, Anthropic, Cohere, HuggingFace, Google, Mistral, and local models via Ollama
  • Magic Commands: Use %%ai and %ai magics directly in notebook cells
  • Chat Interface: Integrated chat panel in the JupyterLab sidebar
  • Built-in RAG: /learn command to index documents and /ask to query local content
  • Notebook Generation: /generate command to create complete notebooks from text descriptions
  • Open Source: Developed by the Project Jupyter community

Installation

Prerequisites

Ensure you have Python 3.8+ and JupyterLab 4.0+ installed.

# Create a new virtual environment (optional but recommended)

python -m venv jupyter-ai-env

source jupyter-ai-env/bin/activate # Linux/Mac

jupyter-ai-env\Scripts\activate # Windows

Install JupyterLab if not already present

pip install jupyterlab>=4.0

Install Jupyter AI

# Install the main jupyter-ai package (includes chat UI and magic commands)

pip install jupyter-ai

Or install only magic commands without the chat UI

pip install jupyter-ai-magics

Install Provider Dependencies

Jupyter AI requires additional packages depending on which LLM provider you want to use:

# For OpenAI (GPT-4, GPT-4o, etc.)

pip install langchain-openai

For Anthropic (Claude)

pip install langchain-anthropic

For Google (Gemini)

pip install langchain-google-genai

For HuggingFace Hub

pip install langchain-huggingface

For Mistral AI

pip install langchain-mistralai

For local models via Ollama

pip install langchain-ollama

For Cohere

pip install langchain-cohere

Configure API Keys

Set API keys as environment variables before launching JupyterLab:

# OpenAI

export OPENAIAPIKEY="sk-..."

Anthropic

export ANTHROPICAPIKEY="sk-ant-..."

Google

export GOOGLEAPIKEY="AIza..."

Mistral

export MISTRALAPIKEY="..."

Or save to .env file for automatic loading

echo 'OPENAIAPIKEY=sk-...' >> ~/.bashrc

source ~/.bashrc

Launch JupyterLab:

jupyter lab

Basic Usage: Magic Commands

Cell Magic %%ai

The %%ai cell magic is the fastest way to interact with LLMs directly from a notebook cell. Write %%ai on the first line of a cell, followed by the provider and model name.

%%ai openai-chat:gpt-4o

Explain the concept of gradient descent in machine learning

in simple, easy-to-understand terms.

The output will be displayed directly below the cell as formatted markdown.

Specifying Output Format

You can specify the output format using the -f flag:

%%ai openai-chat:gpt-4o -f code

Write a Python function to calculate the moving average

of a list of numbers with a configurable window size.

Available formats:

  • text - Plain text (default)
  • code - Code block
  • markdown or md - Markdown formatted
  • math - LaTeX math
  • html - HTML rendered
  • json - JSON formatted

Line Magic %ai

The %ai line magic is used for quick single-line queries:

# Quick question

%ai openai-chat:gpt-4o "What's the difference between a list and a tuple in Python?"

Registering Model Aliases

To avoid typing the full provider and model name repeatedly:

# Register an alias

%ai register mymodel openai-chat:gpt-4o

Use the alias

%%ai mymodel

Create an example of pandas groupby with multiple column aggregations.

Using Python Variables in Prompts

You can interpolate Python variables into prompts using curly braces:

# Define variables

datasetinfo = "Dataset contains 10000 rows with columns: age, income, education, purchaseamount"

targetvariable = "purchaseamount"

%%ai openai-chat:gpt-4o

Based on the following dataset information: {datasetinfo}

Suggest the best feature engineering approaches

for predicting {targetvariable}.

Interpolating Previous Cell Output

Jupyter AI can reference output from previous cells using In and Out:

# Cell 1: Run code

import pandas as pd

df = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})

df.describe()

# Cell 2: Ask AI to analyze the output

%%ai openai-chat:gpt-4o

Analyze the following descriptive statistics and provide insights:

{Out[1]}

Basic Usage: Chat Interface

Opening the Chat Panel

Click the Jupyter AI chat icon in the left sidebar of JupyterLab. The chat panel will open on the right side.

Selecting a Model

At the top of the chat panel, click the gear icon to open settings. Here you can configure:

  • Language Model: Choose the model for chat (e.g., gpt-4o, claude-3-opus)
  • Embedding Model: Choose the model for RAG features (e.g., text-embedding-3-small)
  • API Keys: Input API keys if not set via environment variables
  • Contextual Conversations

    The chat interface supports multi-turn conversations. The AI remembers previous conversation context:

    User: I'm working on a sentiment classification project using BERT.
    

    AI: [provides suggestions]

    User: How should I handle class imbalance in my dataset?

    AI: [responds in the context of BERT sentiment classification]

    Sending Code Selections to Chat

    You can send highlighted code to the chat:

  • Select code in your notebook
  • Right-click > "Send to Jupyter AI Chat"
  • Add your question in the chat input
  • This is particularly useful for debugging or requesting explanations of specific code.

    Advanced Features

    RAG with the /learn Command

    The /learn feature allows you to index local documents so the AI can answer questions based on their content.

    # In the chat panel, type:
    

    /learn /path/to/your/documents/

    Or learn a specific file

    /learn /home/user/project/README.md

    Jupyter AI will:

  • Read all files in the specified directory
  • Split content into chunks
  • Create embeddings using the configured embedding model
  • Store in a local vector store (FAISS)
  • After learning is complete, use /ask to query:

    /ask How do I configure the database connection in this project?
    

    /ask What does the DataProcessor class do?

    Generate Notebooks with /generate

    The /generate command creates complete notebooks from text descriptions:

    /generate Create a notebook for exploratory data analysis (EDA) 
    

    on the Titanic dataset, including distribution visualizations,

    feature correlations, and missing value handling

    Jupyter AI will produce a new notebook with cells containing code and markdown explanations.

    Using Local Models via Ollama

    For privacy or offline usage, you can use local models:

    # Install and run Ollama first
    

    curl -fsSL https://ollama.com/install.sh | sh

    ollama pull llama3.1

    ollama pull nomic-embed-text # for embeddings

    Then in Jupyter AI:

    %%ai ollama:llama3.1
    

    Implement a binary search tree in Python

    with insert, search, and delete methods.

    For the chat interface, select ollama as the provider in settings.

    Custom Model Provider

    You can create custom providers for models not supported by default:

    from jupyteraimagics import BaseProvider
    

    from jupyteraimagics.providers import EnvAuthStrategy

    from langchainopenai import ChatOpenAI

    class CustomProvider(BaseProvider):

    id = "custom-provider"

    name = "My Custom Provider"

    modelidkey = "modelname"

    models = ["my-model-v1", "my-model-v2"]

    authstrategy = EnvAuthStrategy(

    name="CUSTOMAPIKEY",

    keywordparam="apikey"

    )

    pypipackagedeps = ["langchain-openai"]

    @classmethod

    def getchatmodel(cls, args, kwargs):

    return ChatOpenAI(

    baseurl="https://my-api.example.com/v1",

    kwargs

    )

    Save this file and register it as an entry point in your setup.py or pyproject.toml.

    Prompt Templates

    Create reusable prompt templates for frequently performed tasks:

    # Define a template
    

    codereviewtemplate = """

    Perform a code review for the following code:

    python

    {code}

    
    

    Focus on:

  • Potential bugs
  • Performance issues
  • Readability
  • Python best practices
  • Provide a score from 1-10 and specific improvement suggestions.

    """

    Use with magic command

    code = """

    def processdata(data):

    result = []

    for i in range(len(data)):

    if data[i] != None:

    result.append(data[i] 2)

    return result

    """

    %%ai openai-chat:gpt-4o

    {codereviewtemplate.format(code=code)}

    Error Debugging with AI

    Use Jupyter AI for quick error debugging:

    # Cell that produces an error
    

    import pandas as pd

    df = pd.readcsv("data.csv")

    result = df.groupby("category").agg({"value": "mean", "count": "sum"})

    # If there's an error, send the traceback to AI
    

    %%ai openai-chat:gpt-4o -f markdown

    I got the following error while running a pandas groupby:

    {Out[-1]}

    My data has columns: category (string), value (float), count (int).

    What's wrong and how do I fix it?

    Advanced Configuration

    Configuration File

    Jupyter AI stores configuration in ~/.jupyter/jupyteraiconfig.json:

    {
    

    "languagemodel": {

    "provider": "openai-chat",

    "modelid": "gpt-4o"

    },

    "embeddingmodel": {

    "provider": "openai",

    "modelid": "text-embedding-3-small"

    },

    "apikeys": {},

    "chunksize": 2000,

    "chunkoverlap": 200

    }

    RAG Configuration

    Customize chunking parameters for better RAG results:

    # In a notebook or startup script
    

    from jupyterai.config import AiConfig

    config = AiConfig()

    config.chunksize = 1500 # Chunk size (characters)

    config.chunkoverlap = 200 # Overlap between chunks

    config.maxresults = 5 # Number of chunks to retrieve

    Using Multiple Providers Simultaneously

    You can use different providers for different tasks in the same notebook:

    # Use GPT-4o for complex reasoning
    

    %%ai openai-chat:gpt-4o

    Design a microservice architecture for a real-time recommendation system.

    Use Claude for code generation

    %%ai anthropic-chat:claude-sonnet-4-6

    Implement collaborative filtering using matrix factorization in Python.

    Use local model for simple tasks

    %%ai ollama:llama3.1

    Convert this list comprehension to a regular for loop:

    result = [x2 for x in range(10) if x % 2 == 0]

    Best Practices

    1. Choose the Right Model for the Task

    • Code generation: Use models strong at coding (GPT-4o, Claude Sonnet)
    • Data analysis: General purpose models (GPT-4o, Claude)
    • Light tasks/fast iteration: Local models (Llama 3.1 via Ollama) or smaller models
    • Embeddings for RAG**: text-embedding-3-small (OpenAI) or nomic-embed-text (local)

    2. Provide Sufficient Context

    # Less effective
    

    %%ai openai-chat:gpt-4o

    Fix this code.

    More effective

    %%ai openai-chat:gpt-4o

    The following code should calculate the weighted average of the 'score' column

    based on the 'weight' column per group in the 'category' column.

    But the result is always NaN for some groups.

    DataFrame: {df.head().tostring()}

    Dtypes: {df.dtypes.tostring()}

    3. Use RAG for Large Projects

    For projects with many files and documentation:

    /learn /path/to/project/src/
    

    /learn /path/to/project/docs/

    Then ask specific questions:

    /ask How does data flow from the /process API endpoint to the database?
    

    /ask Which function is responsible for data validation?

    4. Build Consistent Workflows

    Example data analysis workflow with Jupyter AI:

    # Step 1: Load and inspect data
    

    import pandas as pd

    df = pd.readcsv("salesdata.csv")

    df.info()

    # Step 2: Ask AI to analyze the data structure
    

    %%ai openai-chat:gpt-4o -f markdown

    Based on the following dataset info:

    {Out[1]}

    Suggest the EDA and preprocessing steps that need to be performed.

    # Step 3: Ask AI to generate preprocessing code
    

    %%ai openai-chat:gpt-4o -f code

    Based on the suggestions above, write Python code to:

  • Handle missing values in numeric columns with median
  • Encode categorical variables
  • Remove outliers using the IQR method
  • 5. Mind Security and Privacy

    • Never send sensitive data (PII, credentials) to cloud API providers
    • Use local models (Ollama) for confidential data
    • Review AI-generated code before executing it
    • Don't store API keys directly in notebooks

    6. Leverage Output Formats

    # For documentation: use markdown
    

    %%ai openai-chat:gpt-4o -f md

    Document the following DataPipeline class with all its methods.

    For presentations: use HTML

    %%ai openai-chat:gpt-4o -f html

    Create a comparison table between Random Forest, XGBoost, and LightGBM.

    For papers/reports: use math

    %%ai openai-chat:gpt-4o -f math

    Write the loss function formula for binary cross-entropy.

    Integration with ML Workflows

    Experiment Tracking with AI Assistance

    import mlflow
    
    

    Ask AI to suggest hyperparameters

    %%ai openai-chat:gpt-4o -f json

    Given a dataset with 50000 samples, 20 numerical features,

    and a binary classification target with a 70:30 ratio,

    suggest a hyperparameter grid for Random Forest in JSON format:

    {"nestimators": [...], "maxdepth": [...], "minsamplessplit": [...]}

    # Parse response and run experiments
    

    import json

    params = json.loads(Out[-1])

    with mlflow.startrun():

    for nest in params["nestimators"]:

    for depth in params["maxdepth"]:

    # train and log...

    pass

    Data Quality Checks

    # Run profiling
    

    profile = df.describe(include='all').tostring()

    %%ai openai-chat:gpt-4o -f markdown

    Analyze the following data profile and identify potential data quality issues:

    {profile}

    Provide specific recommendations for each issue found.

    Troubleshooting

    Jupyter AI Not Showing in Sidebar

    # Ensure the extension is properly installed
    

    jupyter labextension list

    jupyter server extension list

    If not showing, try rebuilding

    jupyter lab build

    Error "No module named 'langchain_openai'"

    # Install the required provider
    

    pip install langchain-openai langchain-anthropic

    Model Not Available

    # List all available models
    

    %ai list

    Output will show all detected providers and models

    Rate Limiting

    If you frequently encounter rate limit errors:

    # Use cheaper models for iteration
    

    %%ai openai-chat:gpt-4o-mini

    Quick question: what's the pandas syntax for merging two dataframes?

    Save expensive models for complex tasks

    %%ai anthropic-chat:claude-sonnet-4-6

    Design and implement a complete ETL pipeline...

    Conclusion

    Jupyter AI transforms how we work with notebooks. By integrating LLMs directly into the Jupyter workflow, you can:

    • Accelerate data exploration by asking questions directly in notebooks
    • Get code generation without switching applications
    • Leverage RAG to understand project codebases and documentation
    • Use local models for sensitive data
    • Create complete notebooks from natural language descriptions

    Start by installing jupyter-ai, configure your preferred provider, and explore the magic commands and chat interface. Over time, you'll discover the most effective usage patterns for your specific workflow.

    Jupyter AI is available on GitHub and full documentation can be found on the official project page.

    Related Articles

    Complete Aisuite Tutorial: Unified Multi-LLM Interface by Andrew Ng

    Tutorial Lengkap Aisuite: Unified Multi-LLM Interface dari Andrew Ng Apa itu Aisuite? Aisuite adalah library Python open...

    Complete Anthropic Claude API Tutorial: From Basics to Advanced Features

    Tutorial Lengkap Anthropic Claude API: Dari Dasar hingga Fitur Lanjutan Anthropic Claude adalah salah satu Large Languag...

    ScrapeGraphAI Tutorial: Intelligent Web Scraping with LLMs and Graph Logic

    Tutorial ScrapeGraphAI: Web Scraping Cerdas dengan LLM dan Graph Logic Pendahuluan Web scraping adalah salah satu teknik...

    MergeKit Tutorial: Merging LLM Models to Create Custom AI Models

    Tutorial MergeKit: Menggabungkan Model LLM untuk Membuat Model AI Kustom Model bahasa besar (LLM) open-source terus berk...