FastEmbed: Fast and Lightweight Embeddings Without Torch, by Qdrant

Hey everyone, Ruby Abdullah here again. This time I want to take you along to explore a library that I think is seriously underrated in the world of AI and RAG, and it is called FastEmbed. This is built by the Qdrant team, who are already famous in the vector database space. If you have ever built a retrieval system, semantic search, or RAG (Retrieval Augmented Generation), then you definitely need embeddings. Normally, to generate embeddings we reach for sentence-transformers, which pulls in torch, CUDA, and a whole pile of heavy dependencies. The install size can reach gigabytes just for setup.

FastEmbed exists to solve exactly that problem. It is lightweight, fast, and most importantly it does not need torch at all. Under the hood it uses ONNX Runtime with quantized models, so inference is blazing fast even on CPU only. For those of you who want to deploy embeddings on a small server, in a serverless function, or in a resource-constrained environment, this is the perfect fit. In this tutorial I will cover everything from scratch, starting with install, generating dense embeddings, choosing models, batch embedding, all the way to the more advanced stuff like sparse embeddings with SPLADE, late interaction with ColBERT, image embeddings, and how to wire FastEmbed up to Qdrant to build a real search engine. Let us get started.

Introduction: Why FastEmbed?

Before we dive into the code, I want you to understand why FastEmbed matters and when you should pick it over the alternatives.

First, about size and speed. When you install sentence-transformers, it drags in PyTorch which is huge, especially the CUDA version. For cases where you only need inference (not training), pulling in torch is honestly overkill. FastEmbed only needs onnxruntime and a few light dependencies. The result is a much smaller install footprint and faster startup time. This is crucial for serverless like AWS Lambda or Google Cloud Functions where cold start is the main enemy.

Second, about performance. The models FastEmbed uses are quantized, meaning the model weights are converted to lower precision (for example int8) without sacrificing accuracy significantly. The effect is that CPU inference becomes very fast. For batch processing of large document volumes, this saves both time and cost.

Third, about integration. FastEmbed is built by the Qdrant team, so its integration with Qdrant is seamless. In fact the Qdrant client can now use FastEmbed behind the scenes to generate embeddings automatically. But FastEmbed can also be used standalone without Qdrant, so it is flexible.

Fourth, about the variety of embedding types. FastEmbed does not just give you plain dense embeddings. It also supports sparse embeddings (like SPLADE) which are great for keyword-based matching, late interaction models (ColBERT) which have high accuracy for re-ranking, and even image embeddings for multimodal cases. So a single library can cover a lot of needs.

So if you are building an application that needs embeddings in production with limited resources and you want something fast, FastEmbed is a wise choice. Now let us go straight to practice.

Installation

Installing FastEmbed is super easy, just one pip command. I recommend creating a virtual environment first so things stay tidy and do not mess with your other projects.

# Create a virtual environment first
python -m venv venv
source venv/bin/activate   # on Windows: venv\Scripts\activate

Install FastEmbed
pip install fastembed

If you have a GPU and want even faster inference, there is a GPU variant:

pip install fastembed-gpu

But honestly, for most cases, the regular CPU version is more than enough and that is actually one of FastEmbed's main selling points. For usage with Qdrant, you can install the Qdrant client that already bundles FastEmbed:

pip install "qdrant-client[fastembed]"

After it is installed, let us check whether it imports correctly. Try running this small script:

from fastembed import TextEmbedding

Check that the import succeeds
print("FastEmbed imported successfully!")

Initialize the default model
model = TextEmbedding()
print("Default model is ready to use.")

The first time you initialize a model, FastEmbed will automatically download its ONNX model from the repository and store it in a local cache (usually in a cache folder under your home directory). So the first run might be a bit slow because of the download, but subsequent runs will be fast since the model is already local. This is behavior you need to remember, especially when deploying in a read-only or ephemeral environment where you may need to pre-download the model first or set a persistent cache directory.

Basic Usage: Generating Dense Text Embeddings

Okay now let us get into the core usage, which is generating dense embeddings from text. A dense embedding is a representation of text in the form of a fixed-dimension vector of float numbers, for example 384 or 768 dimensions. This vector captures the semantic meaning of the text, so texts with similar meaning will have vectors that sit close together in vector space.

Using it is super simple. Note that the embed() method returns a generator, not a list directly. This is a deliberate design to save memory when processing large amounts of data. So we need to wrap it with list() or iterate over it with a loop.

from fastembed import TextEmbedding

Initialize the model. If empty, it uses the default model
which is BAAI/bge-small-en-v1.5 (dimension 384)
model = TextEmbedding()

The list of documents we want to embed
documents = [
    "FastEmbed is a lightweight embedding library from Qdrant.",
    "Qdrant is a fast and open source vector database.",
    "I like drinking coffee in the morning while coding.",
]

embed() returns a generator, so we wrap it with list()
embeddings = list(model.embed(documents))

Each embedding is a numpy array
print(f"Number of embeddings: {len(embeddings)}")
print(f"Dimension of each embedding: {embeddings[0].shape}")
print(f"First 5 numbers of first vector: {embeddings[0][:5]}")

If you run this, you will see 3 embeddings, each with shape (384,), containing float numbers. These numbers are what we later use to compute similarity between texts.

To compute similarity, we usually use cosine similarity. Cosine similarity measures the angle between two vectors, with values ranging from -1 to 1, where closer to 1 means more similar. Here is an example of how to compute it with numpy:

import numpy as np
from fastembed import TextEmbedding

model = TextEmbedding()

documents = [
    "The cat likes to play in the garden.",
    "The dog runs chasing the ball.",
    "Tech stocks rose sharply today.",
]

embeddings = list(model.embed(documents))

def cosinesimilarity(a, b):

    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

Compare the first sentence with the others
for i in range(1, len(documents)):
    sim = cosinesimilarity(embeddings[0], embeddings[i])
    print(f"Similarity '{documents[0]}' vs '{documents[i]}': {sim:.4f}")

You will see that the sentences about cats and dogs (both about animals) have higher similarity than the sentence about stocks. That is how semantic similarity works.

One thing you need to know, there is a difference between embedding documents and embedding queries. FastEmbed provides dedicated methods queryembed() for search queries and passageembed() for documents. Some models are trained with special instructions for query vs passage, so using the right method can improve result quality.

from fastembed import TextEmbedding

model = TextEmbedding()

For documents we want to store in the database
passages = [
    "Python is a popular programming language for data science.",
    "JavaScript is widely used for frontend web development.",
]
passageembeddings = list(model.passageembed(passages))

For a search query from the user
query = "programming language for data analysis"
queryembedding = list(model.queryembed(query))[0]

print(f"Passage embedding dimension: {passageembeddings[0].shape}")

print(f"Query embedding dimension: {queryembedding.shape}")

Choosing and Listing Models

One of FastEmbed's strengths is its support for many models. Each model has different characteristics, some are small and fast, some are larger but more accurate, some are specialized for multilingual use. To see the list of available models, you can call listsupportedmodels().

from fastembed import TextEmbedding

List all supported dense models
supported = TextEmbedding.listsupportedmodels()

for modelinfo in supported:

    print(f"Model: {modelinfo['model']}")
    print(f"  Dimension: {modelinfo['dim']}")

    print(f"  Description: {modelinfo.get('description', '-')}")
    print(f"  Size (GB): {modelinfo.get('sizeinGB', '-')}")

    print()

From that list you can pick the model that fits your needs. Some commonly used models:

For general English, BAAI/bge-small-en-v1.5 (default, 384 dimensions) is a good choice because it is small and fast. If you need more accuracy, there is BAAI/bge-base-en-v1.5 (768 dimensions). For multilingual cases including Indonesian, intfloat/multilingual-e5-large is excellent although it is larger. There is also the legendary and lightweight sentence-transformers/all-MiniLM-L6-v2.

To pick a model, just pass the model name during initialization:

from fastembed import TextEmbedding

Use a multilingual model so it handles Indonesian well
model = TextEmbedding(modelname="intfloat/multilingual-e5-large")

texts = [
    "Aku sedang belajar machine learning.",
    "I am learning machine learning.",
]
embeddings = list(model.embed(texts))
print(f"Dimension: {embeddings[0].shape}")

My tip, if you are unsure which model to use, just start with the default or a small one. Measure its quality on your use case. If it is not satisfying, then upgrade to a larger model. Do not jump straight to a big model because it will consume more resources and be slower.

Batch Embedding for Large Data

If you have thousands or even millions of documents to embed, you cannot just throw them all in at once without thinking about efficiency. FastEmbed is designed to handle this well. The embed() method accepts batchsize and parallel parameters to control how data is processed.

from fastembed import TextEmbedding

model = TextEmbedding()

Assume this is a large dataset
largedocuments = [f"This is document number {i} containing information." 
                   for i in range(10000)]

batchsize controls how many documents are processed per batch

parallel controls the number of worker processes. 0 means use all CPU cores
embeddingsgenerator = model.embed(
    largedocuments,

    batchsize=256,
    parallel=0,  # 0 = use all CPU cores, None = no multiprocessing
)

Since this is a generator, we process one at a time to save memory
count = 0
for embedding in embeddingsgenerator:

    count += 1
    # Here you can directly save to a database, etc
    if count % 1000 == 0:
        print(f"Processed {count} documents so far...")

print(f"Total documents processed: {count}")

The key point here is that because embed() returns a generator, you can process documents one by one without having to hold all embeddings in memory at once. This is super important when the dataset is large. Imagine you have 1 million documents with 768-dimension float32 embeddings, that could eat up several gigabytes if held all at once. With the generator pattern, you can stream directly to the database while embedding runs.

About the parallel parameter, this is worth paying attention to. If set to a positive number, FastEmbed will use multiprocessing with that many workers. If set to 0, it uses all available CPU cores. If None, it does not use multiprocessing at all (suitable for small data where multiprocessing overhead actually slows things down). For large datasets, using parallel=0 usually gives a significant speedup. But for small real-time requests, better to use parallel=None so there is no process spawn overhead.

Advanced Usage

Now we get to the more exciting part. FastEmbed is not just about dense embeddings. There are several advanced features that make it seriously powerful for modern retrieval systems.

Sparse Embeddings with SPLADE

Dense embeddings are great for capturing semantic meaning, but they can be weak for exact keyword matching. For example if a user searches for a specific product code or a rare technical term, dense embeddings can miss. This is where sparse embeddings come in. Sparse embeddings like SPLADE produce vectors with large dimensions but mostly zero values, where only relevant tokens carry weights. This combines the strengths of keyword matching with semantic understanding.

from fastembed import SparseTextEmbedding

List available sparse models
for m in SparseTextEmbedding.listsupportedmodels():

    print(m["model"])

Initialize the SPLADE model
sparsemodel = SparseTextEmbedding(modelname="prithivida/SpladePPenv1")

documents = [
    "FastEmbed supports sparse embeddings using SPLADE.",
    "A vector database stores embeddings for fast search.",
]

sparseembeddings = list(sparsemodel.embed(documents))

Sparse embeddings have indices and values, not a dense array
for i, emb in enumerate(sparseembeddings):

    print(f"Document {i}:")
    print(f"  Number of non-zero tokens: {len(emb.indices)}")
    print(f"  Indices (first 5): {emb.indices[:5]}")
    print(f"  Values (first 5): {emb.values[:5]}")

Notice that the sparse embedding output is different from dense. It has indices attributes (which token positions are active) and values (the weight of each token). This is an efficient format because we do not need to store thousands of zeros. In practice, sparse embeddings are often combined with dense embeddings in an approach called hybrid search, and Qdrant supports this natively.

Late Interaction with ColBERT

Late interaction models like ColBERT take a different approach. Instead of compressing an entire document into a single vector, ColBERT produces one vector per token. During search, it computes the interaction between each query token and each document token. The result is more accurate because per-token information is not lost, but the consequence is it needs more storage and computation. ColBERT is usually used for re-ranking, which is the second stage after we get candidates from fast dense search.

from fastembed import LateInteractionTextEmbedding

List available late interaction models
for m in LateInteractionTextEmbedding.listsupportedmodels():

    print(m["model"])

Initialize the ColBERT model
colbertmodel = LateInteractionTextEmbedding(
    modelname="colbert-ir/colbertv2.0"

)

documents = [
    "FastEmbed provides ColBERT models for late interaction.",
    "Late interaction gives high retrieval accuracy.",
]

colbertembeddings = list(colbertmodel.embed(documents))


Each document becomes a matrix: one vector per token
for i, emb in enumerate(colbertembeddings):
    print(f"Document {i} shape: {emb.shape}")  # (numtokens, dimension)

You will see that the ColBERT output is a two-dimensional matrix, not a single vector. Its shape is (numtokens, dimension). That is why it needs more storage. But for cases where retrieval accuracy is critical, ColBERT is worth it.

Image Embeddings

FastEmbed can also generate embeddings from images, not just text. This is very useful for multimodal cases like image search, where you want to find images based on visual similarity, or even search images using text queries if the model is CLIP-based.

from fastembed import ImageEmbedding

List available image embedding models
for m in ImageEmbedding.listsupportedmodels():
    print(m["model"])

Initialize the image embedding model
imagemodel = ImageEmbedding(modelname="Qdrant/clip-ViT-B-32-vision")

Paths to local image files
images = [
    "path/to/image1.jpg",
    "path/to/image2.png",
]

imageembeddings = list(imagemodel.embed(images))

for i, emb in enumerate(imageembeddings):

    print(f"Image {i} embedding dimension: {emb.shape}")

What is cool, if you use a CLIP-based model, the image embeddings and text embeddings live in the same vector space. That means you can search images using text descriptions. This is the foundation for building a feature like "find images that match this description". You just embed the text query using the CLIP text model, then compare it against the embeddings of your stored images.

Using FastEmbed with Qdrant

Now this is the part I have been waiting for, how to connect FastEmbed to Qdrant to build a real search engine. Because FastEmbed and Qdrant are from the same team, the integration is seamless. The Qdrant client can even generate embeddings automatically using FastEmbed behind the scenes.

First, the easiest way using the automatic feature:

from qdrantclient import QdrantClient

Use in-memory for testing, or replace with a Qdrant server URL
client = QdrantClient(":memory:")

Our document data
docs = [
    "FastEmbed makes embeddings fast and lightweight.",
    "Qdrant is a high-performance vector database.",
    "RAG combines retrieval with generative AI.",
    "Python is a favorite language for machine learning.",
]

add() automatically uses FastEmbed to generate embeddings
client.add(
    collectionname="mydocuments",
    documents=docs,
)

The query is also automatically embedded
results = client.query(
    collectionname="mydocuments",
    querytext="fast vector database",

    limit=2,
)

for point in results:
    print(f"Score: {point.score:.4f} | Document: {point.document}")

Super easy right? You do not need to manually generate embeddings, the Qdrant client handles it all using FastEmbed. But if you want more control, you can generate embeddings manually using FastEmbed and insert them into Qdrant yourself. Here is an example:

from qdrantclient import QdrantClient, models
from fastembed import TextEmbedding

Setup
model = TextEmbedding(modelname="BAAI/bge-small-en-v1.5")

client = QdrantClient(":memory:")

Create a collection with vector configuration
client.createcollection(
    collectionname="articles",

    vectorsconfig=models.VectorParams(
        size=384,  # matches the bge-small model dimension
        distance=models.Distance.COSINE,
    ),
)

Our data
documents = [
    "How to cook delicious and easy fried rice.",
    "Machine learning tutorial for beginners.",
    "Recipe for making trendy milk coffee at home.",
    "Deep learning guide with PyTorch.",
]

Generate embeddings using FastEmbed
embeddings = list(model.embed(documents))

Insert into Qdrant as points
client.upsert(
    collectionname="articles",

    points=[
        models.PointStruct(
            id=idx,
            vector=embedding.tolist(),
            payload={"text": doc},
        )
        for idx, (doc, embedding) in enumerate(zip(documents, embeddings))
    ],
)

Now perform a search
query = "learning AI and neural networks"
queryvector = list(model.queryembed(query))[0]


results = client.querypoints(
    collectionname="articles",

    query=queryvector.tolist(),
    limit=2,
).points

print(f"Query: {query}\n")
for point in results:
    print(f"Score: {point.score:.4f} | {point.payload['text']}")

With this manual approach you have full control over the model used, the payload stored, and the collection configuration. For production, I usually prefer this manual way because it is more explicit and easier to debug.

Best Practices

After all this lengthy discussion, I want to give you some best practices from my experience using FastEmbed so you do not stumble into the same problems I did.

First, reuse the model instance. When you initialize TextEmbedding(), that loads the model into memory which takes time and resources. Do not create a new instance every time you want to embed. Create it once at the start of your application, then reuse that instance for all embeddings. If you use a web framework like FastAPI, load the model at startup and store it as a global variable or dependency.

from fastembed import TextEmbedding

CORRECT: load once, reuse many times
model = TextEmbedding()

def embedtext(textlist):
    return list(model.embed(textlist))


WRONG: do not create a new instance every call
def embedtextwrong(textlist):
    newmodel = TextEmbedding()  # this is very wasteful!

    return list(newmodel.embed(textlist))

Second, manage the cache directory wisely. FastEmbed downloads models to a local cache. In an ephemeral production environment like a container, you want to make sure the model is not re-downloaded on every deploy. You can set the cache directory using the cachedir parameter or pre-download the model during your Docker image build.

from fastembed import TextEmbedding

Set a persistent cache directory
model = TextEmbedding(
    modelname="BAAI/bge-small-en-v1.5",

    cachedir="/data/fastembedcache",

)

Third, choose an appropriate batchsize. A batch size that is too small makes per-batch overhead high, but if it is too large it can eat a lot of memory. For a regular CPU, a batch_size between 128 and 256 is usually the sweet spot. Experiment a bit to find what fits best on your hardware.

Fourth, consistently use the same model between indexing and querying. This is a classic mistake. If you embed documents with model A, then embed queries with model B, the results will be a mess because their vector spaces are different. Always use the same model for indexing and search.

Fifth, leverage generators for large data. Do not immediately list() all embeddings if the data is millions. Iterate the generator and stream directly to the database. This saves memory drastically.

Sixth, consider quantization on the Qdrant side too. If you store millions of vectors, Qdrant has a scalar quantization feature that can shrink the memory footprint without sacrificing much accuracy. Combine lightweight FastEmbed with Qdrant quantization and you get a super efficient system.

Seventh, for multilingual, pick the right model. If your application handles Indonesian, do not use an English-only model like bge-small-en. Use a multilingual model like multilingual-e5. The embedding quality for Indonesian will be much better.

Conclusion

Okay everyone, we have traveled quite far on FastEmbed. Let me summarize what we have learned. FastEmbed is an embedding library from the Qdrant team that is lightweight and fast because it uses ONNX Runtime with quantized models, without needing heavy torch. This makes it an ideal choice for deployments with limited resources, serverless, or anywhere speed and install size matter.

We learned how to install with just a single pip command, generate dense embeddings using TextEmbedding, compute cosine similarity, choose and list available models, and batch embedding for large data using the memory-efficient generator pattern. We also explored advanced features like sparse SPLADE embeddings which are great for keyword matching, late interaction ColBERT for high retrieval accuracy, image embeddings for multimodal cases, and of course seamless integration with Qdrant to build a real search engine.

What I like most about FastEmbed is its philosophy focused on efficiency without sacrificing functionality. For those of you building RAG systems or semantic search in production, especially with resource constraints, I highly recommend giving FastEmbed a try. The combination of FastEmbed plus Qdrant is a solid and economical stack for applications of various scales.

My advice, start simple first. Try generating dense embeddings, measure their quality on your own data, then experiment with sparse and hybrid search if you need accuracy improvements. Do not immediately reach for the most complex feature if the need is not there yet. The principle is the same as coding in general, start simple, measure, then optimize.

That is all for this tutorial from me. I hope it is useful and gets you more excited to explore the world of embeddings and retrieval. If you have questions or want to discuss further, do not hesitate to reach out. See you in the next tutorial, everyone. Happy coding and keep the spirit of learning alive!