Neura Market
antirez/voxtral.c logo

antirez/voxtral.c

Free

Pure C inference of Mistral Voxtral Realtime 4B speech to text model

FreeFree tier
Type
Open Source
Visit Website GitHub (38 stars)

About antirez/voxtral.c

A pure C implementation of the inference pipeline for Mistral AI's Voxtral Realtime 4B speech-to-text model. Zero external dependencies beyond the C standard library. Supports MPS (Apple Silicon) and BLAS backends, with a chunked encoder that uses overlapping windows to bound memory usage regardless of input length. Audio can be piped from stdin, captured live from the microphone (macOS), or transcribed via ffmpeg from any format. Includes a streaming C API (vox_stream_t) for incremental transcription. Also provides a self-contained Python reference implementation (python_simple_implementation.py). Motivation: Mistral released the model open weights but limited inference to a partnership with vLLM; this project offers a standalone, understandable implementation.

Key Features

Zero external dependencies (only C standard library)
Supports MPS (Apple Silicon) and BLAS backends
Chunked encoder with overlapping windows for bounded memory usage
Streaming C API (vox_stream_t) for incremental transcription
Audio input from stdin, live microphone (macOS), or ffmpeg pipe
Includes a simple Python reference implementation
Downloadable model via download_model.sh (8.9GB)
Real-time low-latency streaming with ffmpeg

Pros & Cons

Pros
  • Lightweight with zero external dependencies
  • Fast MPS inference on Apple Silicon
  • Streaming support with low latency
  • Includes both C and Python implementations for clarity
  • Open source with permissive license (MIT)
  • Can handle arbitrary length audio via chunked encoder
Cons
  • BLAS backend is slow due to continuous bf16-to-fp32 conversion
  • Live microphone only supported on macOS
  • Project still in early stage: tested on few samples, not production-ready
  • Requires downloading ~8.9GB model weights
  • No Windows or Linux live mic support yet

Best For

Transcribing audio files (WAV, MP3 via ffmpeg)Real-time microphone transcription on macOSPiping any audio format through ffmpeg for speech-to-textLong audio transcription with circular KV cache bufferEducational reference for understanding Voxtral inference pipeline

FAQ

What dependencies are required to build and run voxtral.c?
The implementation has zero external dependencies beyond the C standard library. For building, you need make and optionally OpenBLAS for the BLAS backend.
How do I transcribe an audio file?
Build with `make mps` (Apple Silicon) or `make blas`, download the model with `./download_model.sh`, then run `./voxtral -d voxtral-model -i audio.wav`. Tokens stream to stdout.
Can I use live microphone input?
Yes, on macOS with `./voxtral -d voxtral-model --from-mic`. Press Ctrl+C to stop.
Is this project production-ready?
No: the project was mostly tested against few samples and likely requires more work to be production quality. It serves as a reference implementation.
What audio formats are supported?
Directly supports raw 16kHz 16-bit mono audio (s16le). Any other format can be piped via ffmpeg using the --stdin option.