speech
Mistral’s Voxtral 3 packages voice agents, with latency and openness caveats News
Mistral launched Voxtral 3 and a voice-agent SDK that connects listening, reasoning, tool calls and speech, while public weight access remains unverified.
Tencent open-sources AuK, a speech model you direct with written instructions News
Tencent released AuK under an MIT licence, a 1.5-billion-parameter speech model that handles voice cloning, audio editing, enhancement and source separation through a single plain-language instruction interface, shipping as a 6.8 GB download with a distilled four-step variant alongside it.
A new lab ships 18 small models that run entirely on your device News
Desert Ant Labs launched with 18 on-device models covering speech, vision and text, shipped as SDKs for Swift, Kotlin and JavaScript, with its headline transcription model a 489 MB download that the company says handles 10 minutes of audio in about 2 seconds on an iPhone.
Google's new transcription model edits what you said News
Gemini 3.5 Transcribe removes filler words, silently resolves speakers' self-corrections, and can make function calls out of the transcription layer -- which makes it excellent for voice agents and unusable as a verbatim record.
A model small enough to run on a laptop reads heard speech from brain scans News
Researchers rebuilt a brain-to-speech decoder around the physical geometry of the scanner helmet, shrinking the decoder about twentyfold while matching prior accuracy, and then used it to identify which properties of speech the brain signal actually carries.
NVIDIA's Open Full-Duplex Voice Model Wants an 80GB GPU News
NVIDIA released an 11-billion-parameter speech model that listens and speaks at the same time and calls tools mid-conversation, and its own documentation requires a GPU with at least 80 GB of memory and lists more than a dozen failure modes.
Full-Duplex Speech Models: Listening and Talking at the Same Time Lesson
A full-duplex speech model processes incoming audio while it is generating outgoing audio, which removes the turn detector that decides when you have stopped speaking and makes interruption, backchannels, and overlap possible.
Neural text-to-speech: how a model turns writing into a voice Lesson
Neural text-to-speech converts written text into audio in three stages - working out the sounds, deciding how long each one lasts, and generating the actual waveform - and the last stage, the vocoder, is where most of the model's size and difficulty hides.
audio.cpp Tool
The September 30 v0.9.0 README reports a shared local audio runtime covering more than 100 models and 170 variants for recognition, synthesis, conversion, and separation.
Whistle Tool
Earlier-release roundup: October 2 CPU speech model with a 16.9 MB download, up to 30-second clips, and seven European languages. Useful for compact offline integrations, with documented quality trade-offs.
SteerTTS Tool
An English speech-synthesis package with numeric controls for pitch, rate and other delivery features; MIT code and noncommercial weights, with documented quality limits.
Pocket TTS Tool
Kyutai’s working speech-synthesis package adds tiny English checkpoints; code is MIT, weights are CC BY 4.0 with gated consent terms.
Phonon ptts Tool
Gradium’s stable speech runtime targets local devices; the 31M English model is Apache 2.0 while the 40M family has noncommercial weight terms.
NVIDIA NemotronLabs VoiceChat 11B Tool
An open-weight end-to-end full-duplex voice model that listens and speaks simultaneously and calls tools mid-conversation, shipped with both offline inference code and a containerised WebSocket streaming deployment. Needs an NVIDIA GPU with at least 80 GB of memory, and uses a single fixed voice.
Hugging Face speech-to-speech Tool
Local voice-activity detection to speech recognition to language model to text-to-speech pipeline, threaded through queues and exposed as an OpenAI Realtime-compatible server so existing clients can point at it unchanged.
Gemini 3.5 Transcribe Tool
Google's new transcription model, shipped as two endpoints: a bidirectional streaming version for live voice agents and a batch version with speaker attribution and word-level timestamps. Handles 85+ languages with mid-stream language switching, cleans filler and self-corrections automatically, and can make function calls to other Gemini models. Try it in AI Studio.
Desert Ant Core SDK Tool
SDKs for Swift, Kotlin and JavaScript that run 18 small models fully offline on iOS, macOS, Android and the web via Core ML, LiteRT and WebAssembly. Covers transcription, PII redaction, language detection, moderation, summarisation and video clipping with no API key and nothing leaving the device.
BetterWispr Tool
A macOS dictation app offering on-device speech recognition and optional external endpoints; configured external providers receive the audio.
AuK-Flash Tool
The distilled sibling of AuK, generating in four steps instead of the full sampling schedule for markedly faster inference at the same 6.8 GB download size. Same MIT licence and the same instruction interface.
AuK Demo Space Tool
A browser demo of AuK on Hugging Face Spaces for judging the output quality directly without downloading 6.8 GB of weights first, which for a generative audio model is the only assessment that counts.
AuK Tool
Tencent's 1.5B open-weights speech model, MIT-licensed, that clones voices, edits recordings, cleans up audio and separates speakers through a single plain-language instruction interface. About 6.8 GB to download, with a four-step distilled variant for faster generation and a hosted demo to try first.