NeMo Guardrails Tutorial: Building Safety Rails for LLM Applications

As Large Language Model (LLM) adoption accelerates across industries, controlling AI behavior has become a critical challenge for developers and organizations. LLMs like GPT-4, Claude, or Llama can generate responses that are inappropriate, harmful, or completely off-topic. This is where NeMo Guardrails comes in.

NeMo Guardrails is an open-source toolkit from NVIDIA that enables developers to add programmable guardrails to LLM-based applications. With NeMo Guardrails, you can control conversation topics, filter harmful content, prevent jailbreak attempts, and ensure your AI operates within defined boundaries.

This tutorial covers everything you need to know about NeMo Guardrails, from installation to advanced production deployment.

Why NeMo Guardrails Matters

Before diving into the technical details, let's understand why guardrails are essential:

Installation and Setup

Prerequisites

Make sure you have Python 3.9 or newer installed on your system.

python --version

Installing NeMo Guardrails

Install NeMo Guardrails using pip:

pip install nemoguardrails

For installation with all features enabled:

pip install nemoguardrails[all]

Additional Dependencies

If you're using OpenAI as the LLM backend:

pip install openai

For local models with HuggingFace:

pip install transformers torch

API Key Configuration

Set the environment variable for your API key:

export OPENAIAPIKEY="sk-your-api-key-here"

Or create a .env file in your project root:

OPENAIAPIKEY=sk-your-api-key-here

Core Concepts: Colang and Configuration

NeMo Guardrails uses a conversation modeling language called Colang (Conversational Language). There are two versions: Colang 1.0 and Colang 2.0. This tutorial covers both.

Project Structure

A NeMo Guardrails project has the following structure:

myguardrailsapp/
├── config/
│   ├── config.yml          # Main configuration
│   ├── rails.co             # Guardrails definitions (Colang)
│   ├── prompts.yml          # Custom prompts
│   └── actions.py           # Custom actions (Python)
├── main.py                  # Application entry point
└── requirements.txt

Basic Configuration (config.yml)

The config.yml file is the central configuration:

models:
type: main
    engine: openai

    model: gpt-4o-mini

instructions:
type: general
    content: |

      You are an AI assistant for a technology company.
      You only answer questions related to technology and company products.
      You always respond politely and professionally.

sampleconversation: |

  user "Hello, how are you?"
    "Hello! I'm doing well, thank you. How can I help you with our products?"
  user "Can you tell me about your products?"
    "Of course! We provide cloud computing and AI solutions for enterprises. Would you like to know more about a specific product?"

define user ask about politics
  "What's your opinion on politics?"
  "Who is the best president?"
  "What do you think about the elections?"
  "Which political party is best?"

define user ask about religion
  "Which religion is the truest?"
  "What's your opinion on religion?"

define bot refuse political topic