Docling: Smart Document Parsing for AI and RAG Pipelines

In the era of generative AI, the ability to automatically extract information from documents has become crucial. Whether it's annual report PDFs, PowerPoint presentations, Word documents, or HTML web pages, we need tools capable of transforming these formats into structured data ready for AI model consumption. This is where Docling comes in.

Docling is an open-source library from IBM Research designed for intelligent document parsing. It supports various input formats and produces clean Markdown or JSON output, ready for use in AI pipelines, particularly Retrieval-Augmented Generation (RAG).

In this tutorial, we will learn how to use Docling from installation, basic document conversion, to building a complete document ingestion pipeline for RAG.

Prerequisites

Before starting, make sure you have:

• Python 3.10 or later
• pip package manager
• Basic understanding of Python
• (Optional) GPU for faster OCR processing

Installation

Installing Docling is straightforward using pip. There are several installation options depending on your needs.

Basic Installation

pip install docling

Installation with OCR Support

pip install "docling[ocr]"

Full Installation (All Features)

pip install "docling[all]"

Verify Installation

import docling
print(f"Docling version: {docling.version}")

Supported Formats

Docling supports various document formats as input:

| Format | Extension | Description |

|--------|-----------|-------------|

| PDF | .pdf | Including scanned/image PDFs |

| Microsoft Word | .docx | Open XML format |

| Microsoft PowerPoint | .pptx | Open XML format |

| HTML | .html, .htm | Web pages |

| Images | .png, .jpg, .tiff | Via OCR |

| AsciiDoc | .adoc | Documentation format |

| Markdown | .md | Markdown format |

Basic Document Conversion

Let's start with simple document conversion. Docling provides an intuitive API through the DocumentConverter class.

Converting PDF to Markdown

from docling.documentconverter import DocumentConverter

Initialize converter
converter = DocumentConverter()

Convert PDF document
result = converter.convert("annualreport.pdf")

Get output in Markdown format
markdownoutput = result.document.exporttomarkdown()

print(markdownoutput)

Converting from URL

Docling also supports direct conversion from URLs:

from docling.documentconverter import DocumentConverter

converter = DocumentConverter()

Convert from URL
result = converter.convert("https://example.com/document.pdf")

markdownoutput = result.document.exporttomarkdown()
print(markdownoutput)

from docling.documentconverter import DocumentConverter

converter = DocumentConverter()

result = converter.convert("projectproposal.docx")

markdownoutput = result.document.exporttomarkdown()

Save to file
with open("projectproposal.md", "w", encoding="utf-8") as f:

    f.write(markdownoutput)

print("Conversion complete!")

Output Formats

Docling supports several output formats that you can choose based on your needs.

Export to Markdown

from docling.documentconverter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("document.pdf")

Export to Markdown
mdcontent = result.document.exporttomarkdown()

Save to file
with open("output.md", "w", encoding="utf-8") as f:
    f.write(mdcontent)

import json
from docling.documentconverter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("document.pdf")

Export to dictionary/JSON
docdict = result.document.exporttodict()


Save to JSON file
with open("output.json", "w", encoding="utf-8") as f:
    json.dump(docdict, f, indent=2, ensureascii=False)

from docling.documentconverter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("document.pdf")

Export to plain text
textcontent = result.document.exporttotext()

print(textcontent)

Table Extraction

One of Docling's standout features is its ability to accurately extract tables from documents.

from docling.documentconverter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("financialreport.pdf")

Iterate over all tables in the document
for tableix, table in enumerate(result.document.tables):

    print(f"\n=== Table {tableix + 1} ===")

    # Export table to pandas DataFrame
    df = table.exporttodataframe()
    print(df)

    # Save to CSV
    df.tocsv(f"table{tableix + 1}.csv", index=False)

    print(f"Table {tableix + 1} saved to CSV")

Table Extraction Configuration

from docling.documentconverter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipelineoptions import PdfPipelineOptions
from docling.datamodel.basemodels import InputFormat


Configure pipeline for tables
pipelineoptions = PdfPipelineOptions()
pipelineoptions.dotablestructure = True

pipelineoptions.tablestructureoptions.docellmatching = True

converter = DocumentConverter(
    formatoptions={

        InputFormat.PDF: PdfFormatOption(
            pipelineoptions=pipelineoptions

        )
    }
)

result = converter.convert("documentwithtables.pdf")


for table in result.document.tables:
    df = table.exporttodataframe()

    print(df.tostring())

Image Extraction

Docling can extract images embedded within documents.

from docling.documentconverter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipelineoptions import PdfPipelineOptions
from docling.datamodel.basemodels import InputFormat

from pathlib import Path

Enable image extraction
pipelineoptions = PdfPipelineOptions()
pipelineoptions.imagesscale = 2.0
pipelineoptions.generatepictureimages = True


converter = DocumentConverter(
    formatoptions={
        InputFormat.PDF: PdfFormatOption(
            pipelineoptions=pipelineoptions
        )
    }
)

result = converter.convert("documentwithimages.pdf")

Save extracted images
outputdir = Path("extractedimages")
outputdir.mkdir(existok=True)

for idx, picture in enumerate(result.document.pictures):
    if picture.image:
        imagepath = outputdir / f"image{idx + 1}.png"

        picture.image.pilimage.save(str(imagepath))

        print(f"Image {idx + 1} saved to {imagepath}")

OCR Support

For scanned documents or images, Docling provides OCR (Optical Character Recognition) support.

from docling.documentconverter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipelineoptions import (
    PdfPipelineOptions,
    EasyOcrOptions,
    TesseractOcrOptions,
)
from docling.datamodel.basemodels import InputFormat


Using EasyOCR
pipelineoptions = PdfPipelineOptions()
pipelineoptions.doocr = True
pipelineoptions.ocroptions = EasyOcrOptions(
    lang=["en", "id"],  # English and Indonesian
    forcefullpageocr=False,

)

converter = DocumentConverter(
    formatoptions={
        InputFormat.PDF: PdfFormatOption(
            pipelineoptions=pipelineoptions
        )
    }
)

result = converter.convert("scanneddocument.pdf")

markdownoutput = result.document.exporttomarkdown()
print(markdownoutput)

from docling.datamodel.pipelineoptions import TesseractOcrOptions

Configure Tesseract
pipelineoptions = PdfPipelineOptions()

pipelineoptions.doocr = True

pipelineoptions.ocroptions = TesseractOcrOptions(

    lang=["eng", "ind"],  # Tesseract language codes
)

converter = DocumentConverter(
    formatoptions={
        InputFormat.PDF: PdfFormatOption(
            pipelineoptions=pipelineoptions
        )
    }
)

result = converter.convert("scanneddocument.pdf")

print(result.document.exporttomarkdown())

from docling.documentconverter import DocumentConverter
from docling.chunking import HierarchicalChunker

converter = DocumentConverter()
result = converter.convert("longdocument.pdf")


Use hierarchical chunker
chunker = HierarchicalChunker()
chunks = list(chunker.chunk(result.document))

for i, chunk in enumerate(chunks):
    print(f"\n--- Chunk {i + 1} ---")
    print(f"Text: {chunk.text[:200]}...")
    print(f"Metadata: {chunk.meta}")

from docling.chunking import HybridChunker
from tokenizers import Tokenizer

Chunker with token limits
chunker = HybridChunker(
    tokenizer=Tokenizer.frompretrained("bert-base-uncased"),
    maxtokens=512,

    mergepeers=True,
)

converter = DocumentConverter()
result = converter.convert("document.pdf")

chunks = list(chunker.chunk(result.document))

for i, chunk in enumerate(chunks):
    print(f"Chunk {i + 1}: {len(chunk.text)} characters")
    print(f"Heading: {chunk.meta.headings}")
    print(f"Text preview: {chunk.text[:150]}...")
    print()

Batch Processing

For processing multiple documents at once, Docling provides batch processing capabilities.

from docling.documentconverter import DocumentConverter
from pathlib import Path

converter = DocumentConverter()

List of documents to process
inputfiles = [
    "document1.pdf",
    "document2.docx",
    "document3.pptx",
    "https://example.com/document4.html",
]

Batch convert
results = converter.convertall(inputfiles)

for result in results:
    source = result.input.file
    print(f"\nProcessing: {source}")

    if result.status.value == "success":
        md = result.document.exporttomarkdown()
        outputfile = Path("output") / f"{Path(str(source)).stem}.md"

        outputfile.parent.mkdir(existok=True)


        with open(outputfile, "w", encoding="utf-8") as f:
            f.write(md)

        print(f"  -> Successfully saved to {outputfile}")

    else:
        print(f"  -> Failed: {result.status}")

from docling.documentconverter import DocumentConverter
from pathlib import Path
import time

converter = DocumentConverter()

inputdir = Path("inputdocuments")
pdffiles = list(inputdir.glob(".pdf"))


print(f"Found {len(pdffiles)} PDF files")


starttime = time.time()
results = converter.convertall(

    [str(f) for f in pdffiles],
    raisesonerror=False,
)

successcount = 0

errorcount = 0

for result in results:
    if result.status.value == "success":
        successcount += 1

        md = result.document.exporttomarkdown()

        outputpath = Path("output") / f"{Path(str(result.input.file)).stem}.md"
        outputpath.parent.mkdir(existok=True)
        with open(outputpath, "w", encoding="utf-8") as f:

            f.write(md)
    else:
        errorcount += 1
        print(f"Error processing {result.input.file}: {result.status}")

elapsed = time.time() - starttime

print(f"\nCompleted in {elapsed:.1f} seconds")
print(f"Success: {successcount}, Failed: {errorcount}")

from docling.documentconverter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipelineoptions import PdfPipelineOptions

from docling.datamodel.basemodels import InputFormat

Custom pipeline configuration
pipelineoptions = PdfPipelineOptions()

pipelineoptions.doocr = True

pipelineoptions.dotablestructure = True
pipelineoptions.tablestructureoptions.docellmatching = True

pipelineoptions.generatepictureimages = True
pipelineoptions.imagesscale = 2.0

converter = DocumentConverter(
    allowedformats=[

        InputFormat.PDF,
        InputFormat.DOCX,
        InputFormat.PPTX,
        InputFormat.HTML,
        InputFormat.IMAGE,
    ],
    formatoptions={
        InputFormat.PDF: PdfFormatOption(
            pipelineoptions=pipelineoptions
        )
    }
)

result = converter.convert("complexdocument.pdf")

print(result.document.exporttomarkdown())

pip install llama-index-readers-docling llama-index

from llamaindex.readers.docling import DoclingReader
from llamaindex.core import VectorStoreIndex


Use DoclingReader
reader = DoclingReader()

documents = reader.loaddata(
    filepath=["report.pdf", "presentation.pptx"]

)

Create index for RAG
index = VectorStoreIndex.fromdocuments(documents)

Query engine
queryengine = index.asqueryengine()


response = queryengine.query(
    "What are the main conclusions from the report?"
)
print(response)

pip install langchain-docling langchain-openai langchain

from langchaindocling import DoclingLoader
from langchain.textsplitter import RecursiveCharacterTextSplitter
from langchainopenai import OpenAIEmbeddings, ChatOpenAI

from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA

Load documents with Docling
loader = DoclingLoader(
    filepath="financialreport.pdf",

    exporttype="markdown",
)
documents = loader.load()

Split documents
textsplitter = RecursiveCharacterTextSplitter(

    chunksize=1000,
    chunkoverlap=200,

)
splits = textsplitter.splitdocuments(documents)


Create vector store
embeddings = OpenAIEmbeddings()
vectorstore = FAISS.fromdocuments(splits, embeddings)

Create RAG chain
llm = ChatOpenAI(model="gpt-4o", temperature=0)
qachain = RetrievalQA.fromchaintype(

    llm=llm,
    chaintype="stuff",
    retriever=vectorstore.asretriever(searchkwargs={"k": 5}),
)

Query
result = qachain.invoke(

    {"query": "What is the company's total revenue this year?"}
)
print(result["result"])

"""
Document Ingestion Pipeline for RAG using Docling
"""

import json
import hashlib
from pathlib import Path
from datetime import datetime
from dataclasses import dataclass, field

from docling.documentconverter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipelineoptions import PdfPipelineOptions, EasyOcrOptions

from docling.datamodel.basemodels import InputFormat
from docling.chunking import HybridChunker
from tokenizers import Tokenizer


@dataclass
class ProcessedChunk:
    """Represents a processed chunk."""
    chunkid: str

    text: str
    sourcefile: str
    headings: list = field(defaultfactory=list)

    pagenumbers: list = field(defaultfactory=list)

    chunkindex: int = 0
    totalchunks: int = 0

    processedat: str = ""


class DocumentIngestionPipeline:
    """Pipeline for document ingestion into a RAG system."""

    def init(
        self,
        outputdir: str = "ragoutput",
        maxtokens: int = 512,

        enableocr: bool = True,
        ocrlanguages: list = None,

    ):
        self.outputdir = Path(outputdir)

        self.outputdir.mkdir(parents=True, existok=True)


        # Pipeline configuration
        pipelineoptions = PdfPipelineOptions()
        pipelineoptions.dotablestructure = True

        pipelineoptions.tablestructureoptions.docellmatching = True

        if enableocr:

            pipelineoptions.doocr = True

            pipelineoptions.ocroptions = EasyOcrOptions(

                lang=ocrlanguages or ["en", "id"],
                forcefullpageocr=False,

            )

        self.converter = DocumentConverter(
            allowedformats=[
                InputFormat.PDF,
                InputFormat.DOCX,
                InputFormat.PPTX,
                InputFormat.HTML,
                InputFormat.IMAGE,
            ],
            formatoptions={

                InputFormat.PDF: PdfFormatOption(
                    pipelineoptions=pipelineoptions

                )
            }
        )

        # Chunker configuration
        self.chunker = HybridChunker(
            tokenizer=Tokenizer.frompretrained("bert-base-uncased"),
            maxtokens=maxtokens,
            mergepeers=True,

        )

    def generatechunkid(self, text: str, source: str, index: int) -> str:
        """Generate a unique ID for each chunk."""
        content = f"{source}:{index}:{text[:100]}"
        return hashlib.md5(content.encode()).hexdigest()

    def processdocument(self, filepath: str) -> list[ProcessedChunk]:
        """Process a single document and produce chunks."""
        print(f"Processing: {filepath}")


        # Convert document
        result = self.converter.convert(filepath)

        if result.status.value != "success":
            print(f"  Failed to convert: {result.status}")
            return []

        # Chunking
        chunks = list(self.chunker.chunk(result.document))
        processedchunks = []


        for i, chunk in enumerate(chunks):
            chunkid = self.generatechunkid(

                chunk.text, filepath, i
            )

            processed = ProcessedChunk(
                chunkid=chunkid,
                text=chunk.text,
                sourcefile=str(filepath),
                headings=chunk.meta.headings if chunk.meta else [],
                chunkindex=i,

                totalchunks=len(chunks),
                processedat=datetime.now().isoformat(),

            )
            processedchunks.append(processed)

        print(f"  Generated {len(processedchunks)} chunks")

        return processedchunks

    def processdirectory(self, inputdir: str) -> list[ProcessedChunk]:
        """Process all documents in a directory."""
        inputpath = Path(inputdir)
        supportedextensions = {

            ".pdf", ".docx", ".pptx", ".html", ".htm",
            ".png", ".jpg", ".jpeg", ".tiff",
        }

        files = [
            f for f in inputpath.rglob("")
            if f.suffix.lower() in supportedextensions

        ]

        print(f"Found {len(files)} documents to process")

        allchunks = []
        for filepath in files:

            chunks = self.processdocument(str(filepath))

            allchunks.extend(chunks)

        return allchunks


    def savechunks(self, chunks: list[ProcessedChunk]) -> str:
        """Save chunks to a JSON file."""
        outputfile = self.outputdir / "chunks.json"

        chunksdata = []

        for chunk in chunks:
            chunksdata.append({
                "id": chunk.chunkid,

                "text": chunk.text,
                "metadata": {
                    "source": chunk.sourcefile,
                    "headings": chunk.headings,
                    "chunkindex": chunk.chunkindex,
                    "totalchunks": chunk.totalchunks,
                    "processedat": chunk.processedat,
                },
            })

        with open(outputfile, "w", encoding="utf-8") as f:

            json.dump(chunksdata, f, indent=2, ensureascii=False)


        print(f"\n{len(chunks)} chunks saved to {outputfile}")
        return str(outputfile)


    def exportforvectordb(self, chunks: list[ProcessedChunk]) -> dict:

        """Export chunks in a format ready for vector databases."""
        return {
            "documents": [c.text for c in chunks],
            "ids": [c.chunkid for c in chunks],
            "metadatas": [
                {
                    "source": c.sourcefile,

                    "headings": ", ".join(c.headings) if c.headings else "",
                    "chunkindex": c.chunkindex,

                }
                for c in chunks
            ],
        }


Pipeline usage
if name == "main":
    # Initialize pipeline
    pipeline = DocumentIngestionPipeline(
        outputdir="ragoutput",

        maxtokens=512,
        enableocr=True,

        ocrlanguages=["en", "id"],
    )

    # Process a single document
    chunks = pipeline.processdocument("annualreport2025.pdf")


    # Or process an entire directory
    # chunks = pipeline.processdirectory("companydocuments/")


    # Save results
    pipeline.savechunks(chunks)

    # Export for vector database
    vectordbdata = pipeline.exportforvectordb(chunks)


    # Example: insert into ChromaDB
    # import chromadb
    # client = chromadb.Client()
    # collection = client.createcollection("companydocuments")

    # collection.add(vectordbdata)


    print("\nIngestion pipeline complete!")
    print(f"Total chunks: {len(chunks)}")

Conclusion

Docling provides a comprehensive solution for document parsing in AI pipelines. With support for various input formats, OCR capabilities, table and image extraction, and integration with popular RAG frameworks like LlamaIndex and LangChain, Docling becomes an invaluable tool for building AI applications that require document understanding.

The intelligent chunking features provided by Docling also simplify data preparation for RAG pipelines, ensuring that document information can be accessed and utilized optimally by AI models.

Start experimenting with Docling on your documents and see how this library can enhance the quality of your AI pipelines!