GraphRAG Tutorial: Graph-Based Retrieval Augmented Generation

Introduction

Retrieval Augmented Generation (RAG) has become the standard approach for connecting Large Language Models (LLMs) with external data. However, traditional RAG has significant limitations: it only performs vector similarity search without understanding the relationships between entities in documents. This is where GraphRAG comes in.

GraphRAG, developed by Microsoft Research, combines the power of knowledge graphs with RAG to produce more comprehensive and contextual answers. Instead of simply searching for text chunks similar to a query, GraphRAG builds a knowledge graph from documents, identifies entities and their relationships, then uses the graph structure to answer questions requiring holistic understanding.

In this tutorial, we will learn how to use Microsoft's graphrag library to build a graph-based RAG system from scratch to production-ready deployment.

Why GraphRAG?

Traditional RAG works well for questions whose answers are contained in one or a few text chunks. However, traditional RAG struggles with questions requiring synthesis of information from many sources, such as:

• "What are the main themes discussed across all these documents?"
• "How does department A relate to department B in the organization?"
• "List all projects involving technology X and person Y"

GraphRAG overcomes these limitations with two query approaches:

Installation

Prerequisites

• Python 3.10 or later
• API key from OpenAI or Azure OpenAI (for LLM and embeddings)
• Minimum 8GB RAM (indexing process requires significant memory)

Installing the Library

pip install graphrag

For the latest version from the repository:

pip install git+https://github.com/microsoft/graphrag.git

Verify the installation:

import graphrag
print(graphrag.version)

Setting Up API Keys

GraphRAG requires an LLM for entity extraction and summary generation. Set up your OpenAI API key:

export GRAPHRAGAPIKEY="sk-your-openai-api-key"

Or for Azure OpenAI:

export GRAPHRAGAPIKEY="your-azure-api-key"
export GRAPHRAGAPIBASE="https://your-resource.openai.azure.com"
export GRAPHRAGAPIVERSION="2024-06-01"

Project Initialization

Creating Project Structure

mkdir graphrag-demo
cd graphrag-demo

python -m graphrag init --root .

This command generates the directory structure:

graphrag-demo/
├── settings.yaml          # Main configuration
├── .env                   # Environment variables
└── input/                 # Folder for source documents

Preparing Input Documents

Place the text documents you want to index in the input/ folder. GraphRAG supports .txt and .csv files.

mkdir -p input

As an example, we will create sample documents about a fictional technology company:

# createsampledata.py
import os

documents = {
    "companyoverview.txt": """

TechNova Inc. is a technology company founded in 2020 in San Francisco. 
The company focuses on developing artificial intelligence solutions for 
the banking and healthcare sectors. The CEO is James Chen, who previously 
worked at Google for 10 years.

The company has three main divisions: the AI Research Division led by 
Dr. Sarah Williams, the Product Development Division led by Michael Park, 
and the Business Development Division led by Elena Rodriguez.

TechNova Inc. secured Series B funding of $50 million from Sequoia Capital 
and Andreessen Horowitz in 2023. The company currently has 200 employees 
spread across San Francisco, New York, and Austin.
""",
    "projects.txt": """
TechNova Inc.'s main projects include:
MedAI - An AI-based diagnostic platform for hospitals. This project is 
led by Dr. Sarah Williams and her team in the AI Research Division. MedAI 
uses computer vision models for medical image analysis and has been adopted 
by 15 hospitals across the United States.
FinGuard - A fraud detection system for banks using machine learning. 
Developed by Michael Park along with the Product Development team. FinGuard 
can detect suspicious transactions with 99.2% accuracy and has been adopted 
by 5 national banks.
DataBridge - A data integration platform connecting various enterprise 
data sources. A collaborative project between the Product Development and 
AI Research divisions. DataBridge supports 50+ data connectors and is used 
by 30 companies.
""",
    "partnerships.txt": """
TechNova Inc. has established strategic partnerships with several institutions:
Stanford University: Research collaboration in the field of biomedical NLP. 
  Dr. Sarah Williams serves as an adjunct professor and mentors 5 PhD students.

JPMorgan Chase: Implementation of FinGuard for real-time fraud detection. 
  Elena Rodriguez led contract negotiations worth $10 million over 3 years.

Mayo Clinic: Pilot project for MedAI in radiology diagnostics. 
  The AI Research team works directly with radiology specialists.

AWS: Partnership for cloud infrastructure. TechNova Inc. achieved 
  AWS Advanced Technology Partner status and received $500K in cloud credits.

Microsoft: Joint venture for developing AI solutions for small businesses. 
  James Chen and Microsoft's VP of AI signed an MoU in January 2024.

"""
}

os.makedirs("input", existok=True)
for filename, content in documents.items():
    with open(f"input/{filename}", "w") as f:
        f.write(content)

print("Sample documents created successfully!")

python createsampledata.py

Configuration

The settings.yaml File

Edit the settings.yaml file to customize the configuration:

llm:
  apikey: ${GRAPHRAGAPIKEY}

  type: openaichat
  model: gpt-4o-mini
  maxtokens: 4096

  temperature: 0

parallelization:
  stagger: 0.3

asyncmode: threaded

embeddings:
  asyncmode: threaded

  llm:
    apikey: ${GRAPHRAGAPIKEY}
    type: openaiembedding

    model: text-embedding-3-small

chunks:
  size: 1200
  overlap: 100

entityextraction:
  maxgleanings: 1


communityreports:
  maxlength: 2000


claimextraction:
  enabled: false

storage:
  type: file
  basedir: "output"


reporting:
  type: file
  basedir: "logs"

Key Configuration Parameters

| Parameter | Purpose | Recommendation |

|-----------|---------|----------------|

| llm.model | LLM model for extraction | gpt-4o-mini for cost savings, gpt-4o for quality |

| chunks.size | Text chunk size | 1200-1500 characters optimal |

| chunks.overlap | Overlap between chunks | 100-200 characters |

| entityextraction.maxgleanings | Additional extraction iterations | 0-2, higher = more complete but more expensive |

| claimextraction.enabled | Claim/fact extraction | false to save tokens |

python -m graphrag index --root .

import pandas as pd
import os

outputdir = "output"

View extracted entities
entities = pd.readparquet(f"{outputdir}/entities.parquet")
print(f"Total entities: {len(entities)}")
print(entities[["title", "type", "description"]].head(10))

# View relationships between entities
relationships = pd.readparquet(f"{outputdir}/relationships.parquet")
print(f"\nTotal relationships: {len(relationships)}")
print(relationships[["source", "target", "description"]].head(10))

# View detected communities
communities = pd.readparquet(f"{outputdir}/communities.parquet")
print(f"\nTotal communities: {len(communities)}")
print(communities[["title", "summary"]].head())

Expected output:

Total entities: 25
                    title          type                              description
0        TECHNOVA INC.  ORGANIZATION  Technology company in San Francisco...
1          JAMES CHEN        PERSON      CEO of TechNova Inc....
2    DR. SARAH WILLIAMS    PERSON      Head of AI Research Division...
3        MICHAEL PARK      PERSON      Head of Product Development...
4      ELENA RODRIGUEZ     PERSON      Head of Business Development...

Total relationships: 30
           source              target                          description
0    JAMES CHEN    TECHNOVA INC.        CEO of the company...
1  DR. SARAH WILLIAMS  AI RESEARCH DIVISION  Leads the division...
2           MEDAI     DR. SARAH WILLIAMS     Project led by...

Performing Queries

Local Search

Local Search is suitable for specific questions about particular entities:

python -m graphrag query \
    --root . \
    --method local \
    --query "What projects are led by Dr. Sarah Williams?"

To use Local Search programmatically:

import asyncio
from graphrag.query.indexeradapters import (

    readindexerentities,

    readindexerrelationships,

    readindexertextunits,
    readindexerreports,
)
from graphrag.query.llm.oai.chatopenai import ChatOpenAI

from graphrag.query.llm.oai.embedding import OpenAIEmbedding
from graphrag.query.structuredsearch.localsearch.mixedcontext import LocalSearchMixedContext
from graphrag.query.structuredsearch.localsearch.search import LocalSearch
from graphrag.vectorstores.lancedb import LanceDBVectorStore


async def runlocalsearch(query: str):

    outputdir = "output"

    # Load data
    entities = readindexerentities(outputdir)

    relationships = readindexerrelationships(outputdir)
    textunits = readindexertextunits(outputdir)

    reports = readindexerreports(outputdir, "community")

    # Setup LLM
    llm = ChatOpenAI(
        model="gpt-4o-mini",
        apikey=os.environ["GRAPHRAGAPIKEY"],

    )

    embedding = OpenAIEmbedding(
        model="text-embedding-3-small",
        apikey=os.environ["GRAPHRAGAPIKEY"],
    )

    # Setup vector store
    descriptionembeddingstore = LanceDBVectorStore(
        collectionname="entitydescriptionembeddings"

    )
    descriptionembeddingstore.connect(dburi="output/lancedb")

    # Build context
    contextbuilder = LocalSearchMixedContext(

        entities=entities,
        entitytextembeddings=descriptionembeddingstore,

        relationships=relationships,
        textunits=textunits,

        communityreports=reports,
        embedding=embedding,
        tokenencoder=None,

    )

    # Create search engine
    searchengine = LocalSearch(
        llm=llm,
        contextbuilder=contextbuilder,
        tokenencoder=None,

        responsetype="multiple paragraphs",
    )

    result = await searchengine.asearch(query)

    print(result.response)
    return result

Run query
asyncio.run(runlocalsearch(

    "What projects are led by Dr. Sarah Williams?"
))

python -m graphrag query \
    --root . \
    --method global \
    --query "What are the main themes and focus areas of TechNova Inc.?"

from graphrag.query.structuredsearch.globalsearch.search import GlobalSearch
from graphrag.query.structuredsearch.globalsearch.communitycontext import GlobalCommunityContext

async def runglobalsearch(query: str):
    outputdir = "output"


    reports = readindexerreports(outputdir, "community")
    entities = readindexerentities(outputdir)


    llm = ChatOpenAI(
        model="gpt-4o-mini",
        apikey=os.environ["GRAPHRAGAPIKEY"],
    )

    contextbuilder = GlobalCommunityContext(

        communityreports=reports,
        entities=entities,
        tokenencoder=None,

    )

    searchengine = GlobalSearch(
        llm=llm,
        contextbuilder=contextbuilder,
        tokenencoder=None,

        responsetype="multiple paragraphs",
        maxdatatokens=12000,
    )

    result = await searchengine.asearch(query)

    print(result.response)
    return result

asyncio.run(runglobalsearch(

    "What are the main themes and focus areas of TechNova Inc.?"
))

# settings.yaml
entityextraction:
  entitytypes:

PERSON
ORGANIZATION
LOCATION
EVENT
TECHNOLOGY
PRODUCT
PROJECT
  maxgleanings: 1

Using Local Models with Ollama

GraphRAG supports local models through OpenAI-compatible APIs:

# settings.yaml
llm:
  apikey: ollama

  type: openaichat
  model: llama3.1
  apibase: http://localhost:11434/v1

  maxtokens: 4096

embeddings:
  llm:
    apikey: ollama

    type: openaiembedding
    model: nomic-embed-text
    apibase: http://localhost:11434/v1

ollama pull llama3.1
ollama pull nomic-embed-text

import networkx as nx
import matplotlib.pyplot as plt
import pandas as pd

Load data
entities = pd.readparquet("output/entities.parquet")
relationships = pd.readparquet("output/relationships.parquet")


Create graph
G = nx.Graph()

Add nodes
for , entity in entities.iterrows():
    G.addnode(entity["title"], type=entity["type"])


Add edges
for , rel in relationships.iterrows():
    G.addedge(rel["source"], rel["target"], description=rel["description"])


Colors based on entity type
colormap = {
    "PERSON": "#FF6B6B",
    "ORGANIZATION": "#4ECDC4",
    "LOCATION": "#45B7D1",
    "EVENT": "#96CEB4",
    "PROJECT": "#FFEAA7",
}

colors = [colormap.get(G.nodes[node].get("type", ""), "#DDA0DD") for node in G.nodes()]


Visualization
plt.figure(figsize=(16, 12))
pos = nx.springlayout(G, k=2, iterations=50, seed=42)
nx.draw(
    G, pos,
    nodecolor=colors,

    nodesize=2000,
    fontsize=8,

    fontweight="bold",
    withlabels=True,

    edgecolor="#cccccc",
    width=1.5,
    alpha=0.9,
)

plt.title("Knowledge Graph - TechNova Inc.", fontsize=16)
plt.tightlayout()

plt.savefig("knowledgegraph.png", dpi=150)
plt.show()
print("Graph saved to knowledgegraph.png")

# Add new documents to input folder
cp newdocument.txt input/

Run indexing with update mode
python -m graphrag update --root .

Integration with LangChain

GraphRAG can be integrated with LangChain to build more complex applications:

from langchain.retrievers import BaseRetriever
from langchain.schema import Document
from typing import List
import asyncio


class GraphRAGRetriever(BaseRetriever):
    """Custom retriever using GraphRAG for search."""

    searchengine: object = None

    searchtype: str = "local"

    class Config:
        arbitrarytypesallowed = True

    def getrelevantdocuments(self, query: str) -> List[Document]:

        result = asyncio.run(self.searchengine.asearch(query))
        return [Document(
            pagecontent=result.response,

            metadata={"contextdata": str(result.contextdata)}

        )]


Usage with LangChain
from langchain.chains import RetrievalQA
from langchainopenai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")
retriever = GraphRAGRetriever(searchengine=searchengine)

qachain = RetrievalQA.fromchaintype(

    llm=llm,
    retriever=retriever,
    chaintype="stuff",
)

answer = qachain.invoke("Describe TechNova's partnerships")

print(answer["result"])

• Use gpt-4o-mini for indexing (saves 10-20x compared to gpt-4o)
• Set claimextraction.enabled: false if not needed
• Reduce maxgleanings to 0 for well-structured documents
• Monitor token usage through logs in the logs/ folder

# Estimate indexing cost
import tiktoken

def estimatecost(inputdir: str, model: str = "gpt-4o-mini"):

    enc = tiktoken.encodingformodel(model)

    totaltokens = 0

    for filename in os.listdir(inputdir):

        if filename.endswith(".txt"):
            with open(f"{inputdir}/{filename}") as f:
                text = f.read()
                totaltokens += len(enc.encode(text))


    multiplier = 5
    estimatedtokens = totaltokens  multiplier


    costper1m = 0.15  # gpt-4o-mini input

    estimatedcost = (estimatedtokens / 1000000)  costper1m


    print(f"Total input tokens: {totaltokens:,}")
    print(f"Estimated processing tokens: {estimatedtokens:,}")

    print(f"Estimated cost: ${estimatedcost:.4f}")

estimatecost("input")

• Chunk size: Chunks that are too small lose context; too large obscures entities. 1200-1500 characters is a good starting point
• Entity types: Add custom entity types relevant to your domain
• Gleanings: Increase maxgleanings to 2-3 for complex documents with many interconnected entities
• Community detection: The Leiden algorithm automatically groups related entities; results can be inspected in communities.parquet

4. Scalability

# For large datasets, use async mode and parallelization
settings.yaml
"""
parallelization:
  stagger: 0.5
  numthreads: 4


asyncmode: threaded

snapshots:
  graphml: true  # Save graph in GraphML format for further analysis
"""

5. Quality Evaluation

# Simple script to evaluate graph quality
import pandas as pd

def evaluategraphquality(outputdir: str):

    entities = pd.readparquet(f"{outputdir}/entities.parquet")

    relationships = pd.readparquet(f"{outputdir}/relationships.parquet")

    communities = pd.readparquet(f"{outputdir}/communities.parquet")


    print("=== Graph Quality Report ===")
    print(f"Entities: {len(entities)}")
    print(f"Relationships: {len(relationships)}")
    print(f"Communities: {len(communities)}")
    print(f"Avg relationships per entity: {len(relationships)/len(entities):.1f}")

    # Entities without relationships (orphans)
    connected = set(relationships["source"].tolist() + relationships["target"].tolist())
    orphans = entities[~entities["title"].isin(connected)]
    print(f"Orphan entities: {len(orphans)}")

    if len(orphans) > 0:
        print(f"  -> {orphans['title'].tolist()}")

    # Entity type distribution
    print("\nEntity type distribution:")
    print(entities["type"].valuecounts().tostring())


evaluategraphquality("output")

openai.RateLimitError: Rate limit reached

Token limit exceeded for entity extraction

• Ensure input documents are not too short (minimum several paragraphs)
• Verify that document language matches the prompts being used
• Try increasing maxgleanings

# Use batching for large datasets
python -m graphrag index --root . --limit 100

Conclusion

GraphRAG offers a far more sophisticated approach compared to traditional RAG for cases requiring contextual understanding and relationships between entities. Here are the key takeaways:

GraphRAG is particularly well-suited for use cases such as legal document analysis, academic research, intelligence reporting, and enterprise knowledge management where entity relationships are critical for producing accurate and comprehensive answers.

For further exploration, visit the official Microsoft GraphRAG repository and documentation to see the latest features and more complex usage examples.