faster-whisper: 4x Faster Audio Transcription at Half the Memory

Hey everyone, if you have ever used OpenAI's Whisper for transcription you know two things: the quality is excellent, and it is slow. In this tutorial I want to introduce faster-whisper, a reimplementation that produces identical results while running roughly four times faster with far lower memory usage.

This is not a new model. The weights are the same. What changed is the engine running them. faster-whisper uses CTranslate2, an inference engine built specifically for Transformer models and optimized aggressively, with mature INT8 and float16 quantization support. So you get Whisper quality at a much more reasonable compute cost.

We will cover why the original Whisper is slow, installation, your first transcription, model size and compute type choices, language detection and word timestamps, VAD for skipping silence, batched and long file transcription, subtitle generation, and finally deployment as an API. Let us go.

Introduction

Why the Original Whisper Is Slow

The official Whisper implementation is written in PyTorch in a research style, not a production style. Every decoding step runs dynamic PyTorch operations with non-trivial Python overhead. There is no aggressive kernel fusion, the attention cache is simple, and quantization is not natively supported.

faster-whisper takes the very same weights and runs them on CTranslate2, which applies operation fusion, efficient memory layouts, kernels tuned for both CPU and GPU, and mature INT8 quantization. In practice, on the same GPU you typically see a three to five times speedup at roughly half the VRAM, and on CPU the gain is often even more noticeable.

The important part: because the weights are identical, your transcription quality does not change. This is not a quality for speed trade, it is simply a more efficient implementation.

Installation

python -m venv venv
source venv/bin/activate   # Linux / Mac
venv\Scripts\activate    # Windows

pip install faster-whisper

You also need ffmpeg installed system wide so various audio and video formats can be read.

# Ubuntu / Debian
sudo apt install ffmpeg

macOS
brew install ffmpeg

To use an NVIDIA GPU, cuBLAS and cuDNN must be available. If you already have a CUDA build of PyTorch, those libraries usually come along and faster-whisper finds them. If you hit a libcudnn error, the simplest fix is:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12

First Transcription

from fasterwhisper import WhisperModel

Loads the model, downloading it on first use
model = WhisperModel("large-v3", device="cuda", computetype="float16")

segments, info = model.transcribe("meeting.mp3", beamsize=5)


print(f"detected language: {info.language} (probability {info.languageprobability:.2f})")
print(f"audio duration: {info.duration:.1f} seconds")

for seg in segments:
    print(f"[{seg.start:.2f}s -> {seg.end:.2f}s] {seg.text}")

There is one crucial detail here: segments is a generator, not a list. Transcription only actually runs as you iterate it. This is good design because you can start showing results before the whole file finishes, but it confuses newcomers who wonder why model.transcribe() returned instantly.

If you need everything at once:

segments = list(segments)
fulltext = " ".join(s.text for s in segments)

print(fulltext)

Choosing a Model Size

Whisper comes in several sizes, and this choice matters most.

tiny and base are very fast and light, fine for simple keyword spotting or clean English. For most other languages their accuracy disappoints. small is a reasonable entry point for many applications, especially with clean audio. medium gives decent quality for non-English languages at a middling cost. large-v3 is the most accurate, especially for non-English speech, heavy accents, and noisy audio. For serious work in a language other than English I almost always use this. distil-large-v3 is a distilled variant that runs about twice as fast as large-v3 with very close accuracy for English. Note that the distilled models are English optimized, so for other languages stay with large-v3.

Compute Type: Your Biggest Speed Knob

This is the most frequently ignored parameter despite having a large impact.

# Modern GPU (Ampere and newer)
model = WhisperModel("large-v3", device="cuda", computetype="float16")


GPU with limited VRAM
model = WhisperModel("large-v3", device="cuda", computetype="int8float16")


CPU, best choice in most cases
model = WhisperModel("large-v3", device="cpu", computetype="int8", cputhreads=8)


Let the library pick per device
model = WhisperModel("large-v3", device="auto", computetype="auto")

Practical guidance: on GPU, float16 is a good default. If VRAM is tight, int8float16 saves significant memory with a quality difference you usually cannot hear. On CPU, int8 is almost always the best choice, and the gap versus float32 can reach three times.

For CPU, set cputhreads to your physical core count, not your logical thread count. Going beyond physical cores rarely helps and sometimes hurts.

Word Level Timestamps

For precise subtitles or for highlighting words during playback, you need word level timestamps.

segments, info = model.transcribe(
    "podcast.mp3",
    wordtimestamps=True,

    beamsize=5,
)

for seg in segments:
    for word in seg.words:
        print(f"{word.start:.2f}-{word.end:.2f}  {word.word}  (p={word.probability:.2f})")

Pay attention to word.probability. It is very useful for quality control. If you build an automated transcription pipeline, you can flag low probability segments for human review instead of reviewing everything.

uncertain = [s for s in segments if s.avglogprob < -0.5 or s.nospeechprob > 0.6]
print(f"{len(uncertain)} segments need manual review")

VAD: Skipping Silence

Voice Activity Detection identifies which portions of the audio actually contain human speech. I recommend this feature for almost every use case.

segments, info = model.transcribe(
    "longinterview.mp3",
    vadfilter=True,

    vadparameters=dict(
        minsilencedurationms=500,

        speechpadms=200,

    ),
)

There are two big benefits. First, speed. If your meeting recording is two hours but only 45 minutes contain speech, you save more than half the processing time. Second, and more importantly, VAD prevents hallucination. Whisper has a bad habit of inventing text when fed silence or non-speech noise, often producing lines like "Thanks for watching" that come from its training data. VAD removes those regions before the model sees them.

Handling Hallucinations

Beyond VAD, several parameters help.

segments, info = model.transcribe(
    "noisyaudio.mp3",
    vadfilter=True,

    conditiononprevioustext=False,   # stop errors from propagating
    temperature=0.0,                    # deterministic
    compressionratiothreshold=2.4,    # reject overly repetitive output
    nospeechthreshold=0.6,
    logprobthreshold=-1.0,
)

conditiononprevioustext=False matters for difficult audio. By default Whisper uses previous text as context, which helps fluency but is dangerous when one segment goes wrong, because the error can propagate and push the model into a repetition loop.

Setting Language and Initial Prompt

If you know the language, say so. It saves detection time and prevents misdetection at the start of a noisy file.

segments, info = model.transcribe(
    "meeting.mp3",
    language="en",
    initialprompt="This is an engineering team meeting about deployment, Kubernetes, and microservices.",
)

initialprompt is very useful for domain vocabulary. If your meeting is full of product names, company acronyms, or technical terms, mention them in the initial prompt. Whisper becomes much more likely to spell them correctly. It is a simple trick with a large effect in technical and medical domains.

Batched Transcription

For processing large files, modern faster-whisper ships a batched pipeline that is much faster on GPU.

from fasterwhisper import WhisperModel, BatchedInferencePipeline

model = WhisperModel("large-v3", device="cuda", computetype="float16")

batched = BatchedInferencePipeline(model=model)

segments, info = batched.transcribe("longrecording.mp3", batchsize=16)


for seg in segments:
    print(f"[{seg.start:.2f} -> {seg.end:.2f}] {seg.text}")

For long files on GPU this can give another multiple-times speedup over the standard mode. Tune batchsize to your VRAM, starting at 8 and increasing while watching memory usage.

To process many files on CPU, the most efficient pattern is one process per file with limited threads, not multithreading inside a single process.

from concurrent.futures import ProcessPoolExecutor
from fasterwhisper import WhisperModel

import glob

def transcribe(path):
    model = WhisperModel("small", device="cpu", computetype="int8", cputhreads=2)

    segments, info = model.transcribe(path, vadfilter=True)
    return path, " ".join(s.text for s in segments)

if name == "main":
    files = glob.glob("audio/.mp3")

    with ProcessPoolExecutor(maxworkers=4) as ex:

        for path, text in ex.map(transcribe, files):
            with open(path + ".txt", "w", encoding="utf-8") as f:
                f.write(text)
            print("done:", path)

Generating Subtitle Files

This is a very common need, so here is a complete helper.

def formatsrttime(seconds: float) -> str:
    h = int(seconds // 3600)
    m = int((seconds % 3600) // 60)
    s = int(seconds % 60)
    ms = int((seconds - int(seconds))  1000)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"

def writesrt(segments, outputpath):

    with open(outputpath, "w", encoding="utf-8") as f:
        for i, seg in enumerate(segments, start=1):
            f.write(f"{i}\n")
            f.write(f"{formatsrttime(seg.start)} --> {formatsrttime(seg.end)}\n")
            f.write(f"{seg.text.strip()}\n\n")

segments, info = model.transcribe("video.mp4", vadfilter=True)

writesrt(segments, "video.srt")
print("subtitles saved")

Note that faster-whisper reads video files directly because it uses ffmpeg under the hood. You do not need to extract the audio first.

Wrapping It in an API

For team use you usually want an HTTP service.

from fastapi import FastAPI, UploadFile, File
from fasterwhisper import WhisperModel

import tempfile, os

app = FastAPI()

Load the model ONCE at startup, never per request
model = WhisperModel("large-v3", device="cuda", computetype="float16")

@app.post("/transcribe")
async def transcribe(file: UploadFile = File(...), language: str = "en"):
    with tempfile.NamedTemporaryFile(delete=False, suffix=".audio") as tmp:
        tmp.write(await file.read())
        path = tmp.name

    try:
        segments, info = model.transcribe(
            path, language=language, vadfilter=True, wordtimestamps=False
        )
        result = [
            {"start": s.start, "end": s.end, "text": s.text.strip()}
            for s in segments
        ]
        return {
            "language": info.language,
            "duration": info.duration,
            "segments": result,
            "text": " ".join(s["text"] for s in result),
        }
    finally:
        os.unlink(path)

Run it with uvicorn app:app --host 0.0.0.0 --port 8000.

Two production notes. First, load the model once at module level rather than inside the handler, since loading large-v3 costs time and memory. Second, if traffic is high, put a queue in front (Celery or a Redis queue) so requests do not pile onto the GPU and cause out of memory errors.

Tips and Best Practices

Always enable vadfilter=True unless you have a strong reason not to. The speed and anti-hallucination benefits are nearly always worth it.

Specify language explicitly when you know it. Auto detection only inspects the first 30 seconds, and if that part is music or silence the whole file can be misclassified.

Use initialprompt for domain vocabulary. It is the cheapest way to improve accuracy on product names and acronyms.

Store avglogprob and nospeechprob per segment in your database. When someone complains about quality later, you will have data to investigate with.

For phone or low quality field recordings, consider noise reduction preprocessing before Whisper. Sometimes fixing the audio beats changing the model.

If you need to know who spoke, combine it with a diarization library such as pyannote.audio and map the diarization output onto the segment timestamps. Whisper itself does not do diarization.

Conclusion

faster-whisper is one of the easiest upgrades available if you still run the original Whisper. The summary:

The weights are identical to OpenAI Whisper, so transcription quality does not change, only execution efficiency through CTranslate2.

computetype is your biggest speed knob: float16 on GPU, int8 on CPU, int8float16 for GPUs with tight VRAM. transcribe returns a generator, so transcription only happens as you iterate.

The VAD filter saves time and prevents hallucinations, and should probably be your default.

Word timestamps, initialprompt, and per-segment confidence scores enable real quality control and better products.

For production, load the model once, use batching on GPU or multiprocessing on CPU, and put a queue in front of your service.

Take an old meeting recording, transcribe it with large-v3 plus VAD, and compare the wall clock against the original Whisper. The gap is usually large enough that you switch and never look back. Happy transcribing.