RAGFlow: Build a RAG Engine That Actually Understands Documents, Complete With Citations

Hey folks, it is me again. If you have ever played around with RAG (Retrieval Augmented Generation), you already know the frustration of watching your bot spit out nonsense just because the chunking was a mess. PDF documents full of tables, figures, and two-column layouts get sliced up carelessly by ordinary RAG tools, and the result is answers that are either wrong or straight up hallucinated. In this tutorial I want to introduce you to RAGFlow, an open-source RAG engine that is laser focused on deep document understanding.

RAGFlow is different from most RAG tools that just split text by characters or tokens. It has what is called deep document understanding, so it genuinely understands the structure of your documents: which part is a heading, which is a paragraph, which is a table, which is a figure, and it can even run OCR on scanned documents. On top of that it uses template-based intelligent chunking, so the way it cuts your documents adapts to the document type. And my favorite part, every answer it gives comes with grounded citations, so you can see exactly which page and which snippet an answer came from. That matters a lot for trust, especially if you want to use it for serious use cases like legal, medical, or financial work.

In this article I will walk you from zero: deploying RAGFlow with Docker Compose, creating a knowledge base, uploading documents, picking the right chunking method, configuring the chat assistant, and finally querying with the HTTP API and the Python SDK, complete with citations. Grab a coffee and let us take it slow.

Introduction: Why RAGFlow Is Different

Before we get into the technical stuff, I want you to understand why RAGFlow is worth learning. The most common problem in the RAG world is not the LLM itself, it is the quality of the retrieval. The "garbage in, garbage out" principle applies hard here. If the document chunk you feed into the LLM is already cut in the middle of a table, or mixes a footer with the actual content, the answer that comes out will be bad too.

RAGFlow attacks this problem at the root. A few things make it stand out:

First, deep document understanding. RAGFlow has a dedicated model for recognizing document layout. It can distinguish headers, paragraphs, lists, tables, and figures. For scanned documents or photos, it runs OCR first. So tables, which are usually a nightmare for other RAG tools, get extracted in RAGFlow with their structure intact.

Second, template-based chunking. RAGFlow provides several chunking templates tuned for different document types. There are templates for general documents, books, academic papers, tables, resumes, presentations, and more. So you are not forced to use a single chunking strategy for every document type. This is what makes the retrieval far more relevant.

Third, grounded citations. Every answer that comes out of RAGFlow always points back to the source chunk and the document page. So if someone doubts the bot's answer, they just check the source. This drastically reduces hallucination risk because the LLM is forced to answer based on the available context instead of making things up.

Fourth, it is all open-source and self-hosted. You can run it on your own server, your data never leaves your infrastructure, and you can use any LLM model, whether that is OpenAI, a local model via Ollama, or another provider. For those of you who care about data privacy, that is a huge plus.

Okay, enough theory. Let us jump straight into deployment.

Installation: Deploy RAGFlow With Docker Compose

The easiest and recommended way to run RAGFlow is with Docker Compose. RAGFlow is not actually a single application, it is a collection of several services: its own web server, Elasticsearch or Infinity for the vector store, MySQL for metadata, MinIO for object storage, and Redis. Docker Compose takes care of all of that at once.

System Preparation

Before we start, make sure your system meets the minimum requirements. RAGFlow is fairly heavy because it bundles Elasticsearch, so do not try running it on a potato machine, folks. Here is what you need:

• CPU with at least 4 cores
• At least 16 GB RAM (recommended, 8 GB works but it will be slow)
• At least 50 GB disk
• Docker version 24 or newer
• Docker Compose version 2.26 or newer

There is one important thing that often trips people up during deployment: Elasticsearch needs a high enough vm.maxmapcount setting. If it is too low, the Elasticsearch container will keep crashing. So let us set it first.

# Check the current vm.maxmapcount value
sysctl vm.maxmapcount

If it is below 262144, set it to 262144
sudo sysctl -w vm.maxmapcount=262144

To make it persist after reboot, add it to sysctl.conf
echo "vm.maxmapcount=262144" | sudo tee -a /etc/sysctl.conf

Clone the Repo and Run It

After that, just clone the RAGFlow repo and go into the docker folder.

# Clone the RAGFlow repository
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/docker

Look at the environment config file
cat .env | grep RAGFLOWIMAGE

Inside the .env file there is an important variable called RAGFLOWIMAGE. By default it usually uses the slim image, which is smaller and does not include the built-in embedding models. If you want the full version that includes embedding models, edit this variable.

# Edit .env, switch the image to the full version (without the -slim suffix)
Example: RAGFLOWIMAGE=infiniflow/ragflow:v0.20.0

nano .env

# Run all services with docker compose
The -d flag runs it in the background (detached)
docker compose -f docker-compose.yml up -d

Check the status of all containers
docker compose -f docker-compose.yml ps

Watch the server log until the ready banner shows up
docker logs -f ragflow-server

# RAGFlow runs on port 80 by default
Open in browser: http://localhost:80 or http://YOUR-SERVER-IP

# Save the basic configuration for all the following examples
RAGFLOWBASEURL = "http://localhost:80"  # change to match your server address

RAGFLOWAPIKEY = "ragflow-xxxxxxxxxxxxxxxxxxxxx"  # API key from the settings menu


Standard header for all HTTP API requests
HEADERS = {
    "Authorization": f"Bearer {RAGFLOWAPIKEY}",

    "Content-Type": "application/json",
}

import requests

RAGFLOWBASEURL = "http://localhost:80"

RAGFLOWAPIKEY = "ragflow-xxxxxxxxxxxxxxxxxxxxx"

HEADERS = {
    "Authorization": f"Bearer {RAGFLOWAPIKEY}",

    "Content-Type": "application/json",
}

def createknowledgebase(name, chunkmethod="naive"):
    """Create a new dataset (knowledge base) via the HTTP API."""
    url = f"{RAGFLOWBASEURL}/api/v1/datasets"
    payload = {
        "name": name,
        # chunkmethod: naive = General, qa, manual, table, paper, etc.

        "chunkmethod": chunkmethod,

        "embeddingmodel": "text-embedding-3-large@OpenAI",
    }
    resp = requests.post(url, headers=HEADERS, json=payload)
    resp.raiseforstatus()
    data = resp.json()
    datasetid = data["data"]["id"]

    print(f"Knowledge base '{name}' created, ID: {datasetid}")
    return datasetid


if name == "main":
    dsid = createknowledgebase("Product Documentation", chunkmethod="naive")

import requests

RAGFLOWBASEURL = "http://localhost:80"

RAGFLOWAPIKEY = "ragflow-xxxxxxxxxxxxxxxxxxxxx"


def uploaddocument(datasetid, filepath):
    """Upload one file to the dataset. Uses multipart, not JSON."""
    url = f"{RAGFLOWBASEURL}/api/v1/datasets/{datasetid}/documents"

    # For file upload, only send Authorization, do not set Content-Type manually
    headers = {"Authorization": f"Bearer {RAGFLOWAPIKEY}"}

    with open(filepath, "rb") as f:
        files = {"file": f}
        resp = requests.post(url, headers=headers, files=files)
    resp.raiseforstatus()
    data = resp.json()
    docid = data["data"][0]["id"]

    print(f"Document uploaded, ID: {docid}")
    return docid


def parsedocument(datasetid, docids):
    """Trigger parsing + chunking for a list of documents."""
    url = f"{RAGFLOWBASEURL}/api/v1/datasets/{datasetid}/chunks"

    headers = {
        "Authorization": f"Bearer {RAGFLOWAPIKEY}",

        "Content-Type": "application/json",
    }
    payload = {"documentids": docids}

    resp = requests.post(url, headers=headers, json=payload)
    resp.raiseforstatus()

    print("Parsing started, check status in the UI or via API.")

if name == "main":
    datasetid = "YOURDATASETID"
    docid = uploaddocument(datasetid, "./product-guide.pdf")

    parsedocument(datasetid, [docid])

Configure the Chat Assistant

The chat assistant in RAGFlow is like a bot persona connected to one or more knowledge bases. This is where you set how the bot answers. There are a few important parameters you need to understand:

• Prompt / system prompt: The base instruction for the bot. Here you can set the tone, constraints, and importantly the instruction to always answer based on context.
• Similarity threshold: The minimum similarity score for a chunk to be considered relevant. Too low and lots of irrelevant chunks get in. Too high and relevant chunks sometimes get thrown out.
• Top N: How many top chunks get sent to the LLM as context.
• Keyword similarity weight: The weight between keyword search (BM25) and vector search (semantic). RAGFlow uses hybrid search, so you can tune how much each one influences the result.
• Temperature: The creativity of the LLM's answers. For RAG, I recommend keeping it low (0.1 to 0.3) so answers stay factual.

One setting I always turn on is the "show citation" option. This is what makes RAGFlow display citations in every answer. The bot inserts markers like [ID:0] in its answer that point to the source chunk. This is RAGFlow's main selling point, so do not turn it off.

Creating a chat assistant via API is also easy:

import requests

RAGFLOWBASEURL = "http://localhost:80"
RAGFLOWAPIKEY = "ragflow-xxxxxxxxxxxxxxxxxxxxx"
HEADERS = {
    "Authorization": f"Bearer {RAGFLOWAPIKEY}",
    "Content-Type": "application/json",
}

def createchatassistant(name, datasetids):

    """Create a chat assistant connected to knowledge bases."""
    url = f"{RAGFLOWBASEURL}/api/v1/chats"

    payload = {
        "name": name,
        "datasetids": datasetids,

        "prompt": {
            "similaritythreshold": 0.2,
            "topn": 6,

            "keywordssimilarityweight": 0.3,

            # Instruction so the bot answers only from context + always cites
            "prompt": (
                "You are a documentation assistant. Answer ONLY based on the "
                "knowledge base context. If it is not in the context, say you "
                "do not know. Always include citations."
            ),
        },
        "llm": {"temperature": 0.2},
    }
    resp = requests.post(url, headers=HEADERS, json=payload)
    resp.raiseforstatus()

    chatid = resp.json()["data"]["id"]
    print(f"Chat assistant created, ID: {chatid}")

    return chatid

if name == "main":
    chatid = createchatassistant("Docs Bot", ["YOURDATASETID"])

import requests

RAGFLOWBASEURL = "http://localhost:80"

RAGFLOWAPIKEY = "ragflow-xxxxxxxxxxxxxxxxxxxxx"

HEADERS = {
    "Authorization": f"Bearer {RAGFLOWAPIKEY}",

    "Content-Type": "application/json",
}

def createsession(chatid, name="new-session"):

    url = f"{RAGFLOWBASEURL}/api/v1/chats/{chatid}/sessions"
    resp = requests.post(url, headers=HEADERS, json={"name": name})
    resp.raiseforstatus()
    return resp.json()["data"]["id"]

def ask(chatid, sessionid, question):
    """Send a question, get the answer + citation references."""
    url = f"{RAGFLOWBASEURL}/api/v1/chats/{chatid}/completions"

    payload = {
        "question": question,
        "sessionid": sessionid,

        "stream": False,  # non-streaming so it is easy to parse
    }
    resp = requests.post(url, headers=HEADERS, json=payload)
    resp.raiseforstatus()

    data = resp.json()["data"]

    print("ANSWER:")
    print(data["answer"])

    # The reference section holds the source chunks the answer is based on
    reference = data.get("reference", {})
    chunks = reference.get("chunks", [])
    print("\nCITATION SOURCES:")
    for i, c in enumerate(chunks):
        docname = c.get("documentname", "unknown")

        snippet = c.get("content", "")[:120]
        print(f"  [{i}] {docname}: {snippet}...")
    return data

if name == "main":
    chatid = "YOURCHATID"

    sessionid = createsession(chatid)
    ask(chatid, sessionid, "How do I reset the password in this product?")

There are two things to pay attention to in the response: answer which holds the answer text (with citation markers like [ID:0] inside it), and reference which holds the details of each source chunk, complete with document name, chunk content, and even page number. The combination of the two is what makes RAGFlow's answers verifiable.

Use the Official Python SDK

If you do not want to deal with manual HTTP requests, RAGFlow has an official Python SDK that is much cleaner. Just install it first.

# Install the official RAGFlow SDK
pip install ragflow-sdk

Then the code becomes far more concise and readable:

from ragflowsdk import RAGFlow

Initialize the client
rag = RAGFlow(
    apikey="ragflow-xxxxxxxxxxxxxxxxxxxxx",
    baseurl="http://localhost:80",

)

Get an existing dataset by name
datasets = rag.listdatasets(name="Product Documentation")
dataset = datasets[0]
print(f"Dataset: {dataset.name}, document count: {dataset.documentcount}")


Get the chat assistant we created
assistants = rag.listchats(name="Docs Bot")
assistant = assistants[0]

Create a session then ask
session = assistant.createsession(name="sdk-session")

answer = session.ask("What are the main features of this product?", stream=False)

print("ANSWER:")
print(answer.content)

Loop through the references to see citations
print("\nCITATIONS:")
for ref in answer.reference:
    print(f"  - {ref.get('documentname')} (p. {ref.get('page', '?')})")

The SDK also supports streaming if you want to display answers token by token like ChatGPT. Just set stream=True and iterate over the result. For production apps that need a responsive UX, streaming is really important.

Best Practices

After spending a good while working with RAGFlow, there are a few things I have learned and want to share so you do not fall into the same pits I did.

First, always verify the chunking results before production. Do not blindly trust automatic chunking. Open the visual inspection, check whether tables got extracted correctly, whether any chunk got cut off weirdly. Ten minutes of checking chunks can save you hours of debugging bad answers later.

Second, pick the chunking template based on the document type, do not just slap General on everything. I have run into cases several times where switching from General to a more specific template (say Paper for scientific documents) immediately improved answer quality significantly. If your knowledge base contains a mix of many document types, consider splitting it into several knowledge bases with their own templates.

Third, tune the similarity threshold carefully. The default value is sometimes too loose, letting irrelevant chunks in and ruining answers. If you notice the bot often gives vague, tangential answers, try nudging the threshold up gradually. Conversely, if the bot often says "I do not know" when the answer actually exists, lower the threshold.

Fourth, leverage hybrid search. RAGFlow combines semantic search with keyword search. For documents full of technical terms, product names, or specific codes, keyword search matters a lot because semantic search sometimes misses on rare terms. Raise the keyword similarity weight if your domain is full of jargon.

Fifth, always enable and display citations. This is not just about a cool feature. Citations build user trust, and more importantly, they make it easy for you to audit if there is a wrong answer. You can immediately trace which chunk a wrong answer came from, then fix it at the source.

Sixth, keep an eye on server resources. RAGFlow is heavy because of Elasticsearch. Monitor RAM and disk usage regularly. If your knowledge base is really large, consider using Infinity as a lighter alternative vector store, or add more RAM to the server. Do not let a container die in the middle of parsing an important document.

Seventh, guard your API key carefully. The RAGFlow API key has full access to all your datasets. Never hard-code it in code that gets pushed to a public repo. Use environment variables or a secret manager. This is basic but very often forgotten.

Eighth, for production, think about your document update strategy. If a source document changes, you have to re-parse it. Build an automated pipeline using the API I showed earlier so the knowledge base stays in sync with the original source. Stale documents are just as dangerous as wrong answers.

Conclusion

Alright folks, we have reached the end of the tutorial. We covered quite a lot: from why RAGFlow is different from ordinary RAG tools, deploying with Docker Compose, creating a knowledge base and uploading documents, picking the right chunking method, configuring the chat assistant, to querying with the HTTP API and Python SDK complete with citations.

The main thing I want you to take home from this article is that RAG quality is determined at the retrieval stage, not just in the LLM. RAGFlow gives you full control at that stage: deep document understanding to grasp document structure, chunking templates to cut smartly, hybrid search for accurate retrieval, and grounded citations for answers you can trust and verify. This combination makes RAGFlow a serious choice for production use cases, especially for domains where you cannot afford to be wrong, like legal, medical, or customer service.

My advice, do not stop at this tutorial. Deploy it yourself, load in your real documents, then experiment with the various chunking templates and retrieval settings. Feel for yourself how a small change in configuration can have a big effect on answer quality. Because in the end, good RAG is the result of experimentation and iteration, not a one-shot setup.

If you have questions or find interesting insights while trying RAGFlow, do not hesitate to share them with me. Happy tinkering, folks, and may your RAG bot get smarter and more trustworthy. See you in the next tutorial.