Whisper (OpenAI)

OpenAI's Whisper is an automatic speech recognition (ASR) system trained on 680,000 hours of multilingual and multitask supervised data collected from the web. Its large and diverse training set gives it exceptional robustness to accents, background noise, and technical language. Whisper supports transcription in multiple languages and translation from those languages into English, all within a single end-to-end encoder-decoder Transformer architecture. Input audio is split into 30-second chunks, converted into log-Mel spectrograms, and passed through an encoder. A decoder predicts text captions intermixed with special tokens for language identification, timestamps, and translation tasks. Whisper is open-source, with models and inference code publicly available, and achieves 50% fewer errors in zero-shot evaluations across diverse datasets compared to prior models, though it does not surpass models specialized for benchmarks like LibriSpeech.