OpenAI Whisper - Speech-to-Text Complete Tutorial
Table of Contents
Introduction
OpenAI Whisper is a general-purpose speech recognition model trained on 680,000 hours of multilingual and multitask supervised data collected from the web. It is capable of multilingual speech recognition, speech translation, and language identification. This tutorial walks you through everything from basic transcription to production-grade deployment with FastAPI.
Prerequisites
- Python 3.9 or higher
- A CUDA-compatible GPU (recommended for larger models)
- Basic familiarity with Python and machine learning concepts
Install the core dependencies:
pip install openai-whisper
pip install faster-whisper
pip install fastapi uvicorn python-multipart
pip install torch torchaudio
pip install datasets transformers
Understanding Whisper Model Sizes
Whisper comes in several model sizes, each offering a tradeoff between speed and accuracy:
| Model | Parameters | English-only | Multilingual | Required VRAM | Relative Speed |
|--------|-----------|--------------|--------------|---------------|----------------|
| tiny | 39 M | tiny.en | tiny | ~1 GB | ~32x |
| base | 74 M | base.en | base | ~1 GB | ~16x |
| small | 244 M | small.en | small | ~2 GB | ~6x |
| medium | 769 M | medium.en | medium | ~5 GB | ~2x |
| large | 1550 M | N/A | large-v3 | ~10 GB | 1x |
Choosing the right model depends on your use case:
import whisper
For quick prototyping or low-resource environments
model = whisper.loadmodel("tiny")
For production with good accuracy
model = whisper.loadmodel("medium")
For best accuracy (requires GPU with 10GB+ VRAM)
model = whisper.loadmodel("large-v3")
The .en models are optimized for English-only tasks and tend to perform better than their multilingual counterparts for English content.
Basic Audio Transcription
The simplest use case is transcribing an audio file to text:
import whisper
model = whisper.loadmodel("medium")
Basic transcription
result = model.transcribe("audiosample.mp3")
print(result["text"])
Access detailed segment information
for segment in result["segments"]:
print(f"[{segment['start']:.2f}s -> {segment['end']:.2f}s] {segment['text']}")
You can customize transcription behavior with various parameters:
result = model.transcribe(
"audiosample.mp3",
language="en", # Force language (skip detection)
temperature=0.0, # Lower = more deterministic
wordtimestamps=True, # Get word-level timestamps
fp16=True, # Use half-precision (GPU only)
beamsize=5, # Beam search for better accuracy
bestof=5, # Number of candidates when sampling
nospeechthreshold=0.6, # Threshold to filter silent segments
conditiononprevioustext=True, # Use context from previous segments
initialprompt="Technical meeting about machine learning models."
)
With word-level timestamps
for segment in result["segments"]:
for wordinfo in segment.get("words", []):
print(f" {wordinfo['word']} [{wordinfo['start']:.2f}s - {wordinfo['end']:.2f}s]")