LightRAG: Lightweight, Fast, and Affordable Graph RAG for Your Documents
Hey everyone, today we look at a RAG approach I find genuinely interesting because it fixes a fundamental weakness of ordinary RAG: LightRAG. If you have ever built a document question answering system on a vector database and found the answers great for specific questions but useless for big picture questions, this tutorial is for you.
Here is the problem. Conventional RAG splits documents into chunks, embeds them, and retrieves the chunks most similar to the question. That works very well for questions like "what is the reimbursement limit?" because the answer lives in one chunk. But for a question like "what themes recur across all of this year's quarterly reports?" the approach fails, because the answer is scattered across dozens of chunks and none of them is semantically similar to the question.
Microsoft's GraphRAG solves this by building a knowledge graph, but it is expensive because it needs many LLM calls to build and summarize communities. LightRAG takes the same graph idea with a much cheaper approach. That is what makes it attractive for real projects with real budgets.
Introduction
Why Ordinary RAG Has Limits
I want you to really understand the limit, because many people waste time tuning chunk size when the problem is structural.
Vector RAG searches for semantic similarity. If your question has an answer written explicitly somewhere, it finds it. But two categories of question are structurally unanswerable this way.
First, questions that require connecting information across documents. For example, "which vendor appears in both project A and project C?" The vendor information for project A is in one document and project C in another. Vector search may find one of them, but it has no way to connect them.
Second, aggregate or thematic questions. "What is the most common customer complaint?" No single chunk contains this answer. It must be synthesized from the whole corpus.
How LightRAG Works
LightRAG adds a graph layer on top of vector retrieval.
When you insert documents, LightRAG uses an LLM to extract entities (people, organizations, products, concepts) and the relationships between them from each chunk. The result is stored as a knowledge graph where nodes are entities and edges are relationships. Entity and relationship descriptions are also embedded into a vector store.
When you ask a question, LightRAG retrieves at two levels simultaneously. The low level finds specific entities mentioned in the question along with their immediate graph neighbors. The high level finds thematic keywords and broadly relevant relationships. Context from both levels is merged and handed to an LLM to compose the answer.
Its main efficiency advantage over GraphRAG is that LightRAG does not build those expensive hierarchical community summaries. It relies on direct local and global graph retrieval instead. Indexing and query costs are far lower, and importantly, adding new documents is incremental with no full index rebuild.
Installation
python -m venv venv
source venv/bin/activate # Linux / Mac
venv\Scripts\activate # Windows
pip install lightrag-hku
pip install openai numpy
To use local models through Ollama:
pip install ollama
Basic Setup
import os
import asyncio
from lightrag import LightRAG, QueryParam
from lightrag.llm.openai import gpt4ominicomplete, openaiembed
from lightrag.kg.sharedstorage import initializepipelinestatus
os.environ["OPENAIAPIKEY"] = "sk-..."
WORKDIR = "./ragstorage"
os.makedirs(WORKDIR, existok=True)
async def buildrag():
rag = LightRAG(
workingdir=WORKDIR,
embeddingfunc=openaiembed,
llmmodelfunc=gpt4ominicomplete,
)
await rag.initializestorages()
await initializepipelinestatus()
return rag
Note workingdir. That is where everything lives: the knowledge graph, the entity vector store, the relationship vector store, and the document cache. That directory is your asset, back it up properly.
Inserting Documents
import asyncio
async def main():
rag = await buildrag()
# A single document
with open("companypolicy.txt", encoding="utf-8") as f:
await rag.ainsert(f.read())
# Several documents at once
docs = []
for name in ["reportq1.txt", "reportq2.txt", "reportq3.txt"]:
with open(name, encoding="utf-8") as f:
docs.append(f.read())
await rag.ainsert(docs)
print("indexing complete")
asyncio.run(main())
Insertion is the expensive part, because every chunk requires an LLM call for entity and relationship extraction. For long documents this takes minutes and a cost you should budget for.
A cost saving tip here: use a cheap model such as gpt-4o-mini for extraction. Entity extraction quality does not need the smartest model, and the cost difference is enormous on a large corpus.
The good news is that LightRAG deduplicates. Inserting the same document twice does not reprocess it. And adding new documents is incremental with no full rebuild.
The Four Query Modes
This is the most important part to understand, because picking the right mode determines answer quality.
from lightrag import QueryParam
naive mode: plain vector RAG, no graph
answer = await rag.aquery(
"What is the maximum reimbursement limit?",
param=QueryParam(mode="naive"),
)
local mode: focuses on specific entities and their neighbors
answer = await rag.aquery(
"What was John Smith's role in project Alpha?",
param=QueryParam(mode="local"),
)
global mode: focuses on themes and broad relationships
answer = await rag.aquery(
"What themes recur across all the quarterly reports?",
param=QueryParam(mode="global"),
)
hybrid mode: combines local and global
answer = await rag.aquery(
"How do vendor delays relate to margin decline?",
param=QueryParam(mode="hybrid"),
)
How to choose.
Use naive for simple factual questions whose answer certainly lives in one place. It is fastest and cheapest.
Use local when the question names specific entities such as a person, product, or project. This mode pulls that entity plus its direct relationships.
Use global for thematic questions, summaries, or patterns spread across many documents.
Use hybrid when you are unsure, or when the question mixes specific entities with broader relationship reasoning. It is the safe default despite costing a bit more.
If you build a product for end users, I recommend hybrid as the default with a naive option for clearly factual questions, because the speed difference is noticeable.
Tuning Query Parameters
param = QueryParam(
mode="hybrid",
topk=60, # entities/relations retrieved
maxtokenfortextunit=4000, # token budget for raw text chunks
maxtokenforglobalcontext=4000, # token budget for relationship context
maxtokenforlocalcontext=4000, # token budget for entity context
responsetype="Multiple Paragraphs", # answer format
)
answer = await rag.aquery("Summarize this year's main project risks", param=param)
responsetype accepts free-form natural language instructions. You can pass "Single Paragraph", "Bullet Points", "Markdown Table", or something specific like "A short answer of at most three sentences". It is the easiest way to control output format without writing your own prompt.
If you want to inspect the retrieved context without calling the LLM for a final answer, for debugging:
param = QueryParam(mode="hybrid", onlyneedcontext=True)
context = await rag.aquery("Who are our logistics vendors?", param=param)
print(context)
This is very useful when answers feel wrong. Often the problem is not the LLM but the retrieved context.
Using Local Models With Ollama
For sensitive data that must not leave your server, everything can run locally.
import os
from lightrag import LightRAG
from lightrag.llm.ollama import ollamamodelcomplete, ollamaembed
from lightrag.utils import EmbeddingFunc
async def buildlocalrag():
rag = LightRAG(
workingdir="./raglocal",
llmmodelfunc=ollamamodelcomplete,
llmmodelname="qwen2.5:14b",
llmmodelkwargs={
"host": "http://localhost:11434",
"options": {"numctx": 32768},
},
embeddingfunc=EmbeddingFunc(
embeddingdim=1024,
maxtokensize=8192,
func=lambda texts: ollamaembed(
texts, embedmodel="bge-m3", host="http://localhost:11434"
),
),
)
await rag.initializestorages()
return rag
Two important notes for local setups. First, numctx must be generous, at least 32768, because entity extraction sends long prompts. If the context window is too small, extraction gets truncated and your graph turns into garbage. This is the number one cause of bad results in local setups.
Second, pick a model smart enough for structured extraction. Models at 7B and below frequently fail to follow the extraction format. I recommend at least 14B, and qwen2.5 or llama3.1 work reasonably. For embeddings, bge-m3 is good and handles many languages.
Inspecting and Using the Graph
One benefit of the graph approach is that you can see what the system actually understood.
import networkx as nx
G = nx.readgraphml("./ragstorage/graphchunkentityrelation.graphml")
print(f"entities: {G.numberofnodes()}")
print(f"relationships: {G.numberofedges()}")
Most connected entities, usually the most central concepts
degrees = sorted(G.degree, key=lambda x: x[1], reverse=True)
for node, d in degrees[:15]:
print(f"{d:4d} {node}")
Inspect neighbors of one entity
target = "PROJECT ALPHA"
if target in G:
for neighbor in G.neighbors(target):
data = G.getedgedata(target, neighbor)
print(f"{target} -> {neighbor}: {data.get('description', '')[:80]}")
I always run this after the first indexing pass. If the most connected entities turn out to be irrelevant noise, that signals broken extraction needing prompt fixes or document cleanup.
Adding Domain Knowledge
If your documents have domain specific entity types, tell LightRAG about them.
rag = LightRAG(
workingdir=WORKDIR,
embeddingfunc=openaiembed,
llmmodelfunc=gpt4ominicomplete,
addonparams={
"entitytypes": [
"person", "organization", "product", "project",
"vendor", "regulation", "risk", "location",
],
"language": "English",
},
)
Setting language matters for non-English documents. Without it, extracted entities and their descriptions often come out in English even when the source is another language, which degrades retrieval accuracy.
Wrapping It in an API
from fastapi import FastAPI
from pydantic import BaseModel
from contextlib import asynccontextmanager
from lightrag import LightRAG, QueryParam
from lightrag.llm.openai import gpt4ominicomplete, openaiembed
from lightrag.kg.sharedstorage import initializepipelinestatus
state = {}
@asynccontextmanager
async def lifespan(app: FastAPI):
rag = LightRAG(
workingdir="./ragstorage",
embeddingfunc=openaiembed,
llmmodelfunc=gpt4ominicomplete,
)
await rag.initializestorages()
await initializepipelinestatus()
state["rag"] = rag
yield
app = FastAPI(lifespan=lifespan)
class Ask(BaseModel):
question: str
mode: str = "hybrid"
@app.post("/ask")
async def ask(req: Ask):
rag = state["rag"]
answer = await rag.aquery(
req.question,
param=QueryParam(mode=req.mode, responsetype="Multiple Paragraphs"),
)
return {"answer": answer, "mode": req.mode}
class Document(BaseModel):
text: str
@app.post("/documents")
async def adddocument(req: Document):
rag = state["rag"]
await rag.ainsert(req.text)
return {"status": "ok"}
Note the use of lifespan. Storage initialization happens once at startup rather than per request, because loading the graph and vector stores takes time.
Tips and Best Practices
Clean your documents before inserting. Repeated headers, footers, page numbers, and navigation text produce junk entities that pollute the graph.
Use a cheap model for extraction and a smart model for answering. You can configure both separately, and the savings are significant.
Always inspect the graph after your first indexing run. Five minutes looking at top entities saves hours of guessing why answers are poor.
For non-English documents, always set language in addon_params and list entity types relevant to your domain.
Do not use global mode for simple factual questions. You pay more for a vaguer answer.
Back up your working directory. Rebuilding the index for a large corpus is expensive in both time and money.
Evaluate with a question set covering both types: specific factual and broadly thematic. If you only test factual questions, you will never learn whether the graph layer is adding value.
Conclusion
LightRAG is a nice middle ground between cheap but limited vector RAG and powerful but expensive GraphRAG. The summary:
Vector RAG fails structurally on questions requiring cross-document connections or thematic synthesis.
LightRAG builds an entity and relationship graph at index time, then retrieves at two levels at query time.
Four query modes serve different needs: naive for simple facts, local for specific entities, global for themes, and hybrid as the safe default.
Document insertion is incremental, which makes it far more practical for continuously growing systems than approaches requiring full rebuilds.
Everything can run locally with Ollama, provided the context window is large enough and the model is capable of structured extraction.
The resulting graph is directly inspectable, which is a genuinely valuable debugging tool.
Take ten internal documents from work, index them with LightRAG, then ask two kinds of question: one whose answer clearly sits in a single document, and one requiring a pattern across all of them. Compare against your existing vector RAG. The gap on the second type is the real value of the graph approach. Happy building.