- VibeVoice-ASR, a 7B-parameter speech recognition model that processes up to 60 minutes of continuous audio in a single inference pass, was integrated into Hugging Face Transformers on March 6, 2026.
- Microsoft removed the VibeVoice-TTS code from its public GitHub repository in September 2025 after discovering uses it described as “inconsistent with the stated intent.”
- The suite’s architecture uses continuous speech tokenizers operating at an ultra-low 7.5 Hz frame rate, paired with a next-token diffusion framework built on a Qwen2.5 base model.
- Microsoft has designated all VibeVoice models for research use only and advises against commercial deployment without further testing.
What Happened
Microsoft released VibeVoice-ASR on January 21, 2026—a unified automatic speech recognition model designed to process up to 60 minutes of continuous audio in a single inference pass, generating structured transcriptions that combine speaker identity, timestamps, and spoken content. On March 6, 2026, the model was added to the Hugging Face Transformers library, allowing developers to load it through the standard Python interface without custom inference code. The GitHub repository does not attribute individual researchers by name.
Why It Matters
Conventional ASR pipelines handle long recordings by segmenting audio into short chunks—typically 30 seconds or fewer—a process that can fragment speaker tracking and break sentence-level coherence across extended recordings. VibeVoice-ASR accepts a continuous 64K-token context window, removing that segmentation constraint for recordings up to one hour. The VibeVoice product line carries a complicated public history: VibeVoice-TTS, released August 25, 2025 and accepted as an Oral presentation at ICLR 2026, was removed from Microsoft’s public GitHub repository by September 5, 2025. Microsoft stated at the time that the company had “discovered instances where the tool was used in ways inconsistent with the stated intent,” citing responsible AI use as a guiding principle.
Technical Details
VibeVoice’s architecture centers on two continuous speech tokenizers—Acoustic and Semantic—both operating at a frame rate of 7.5 Hz. Microsoft states this design preserves audio fidelity while reducing the computational cost of long-sequence processing. The system uses a next-token diffusion framework: a large language model processes textual context and dialogue structure, while a separate diffusion head generates high-fidelity acoustic output. VibeVoice-ASR’s 7B-parameter model jointly performs speech recognition, speaker diarization, and timestamping in a single forward pass, and supports user-specified hotwords—custom vocabulary items such as proper nouns or technical terms—to improve domain-specific accuracy. The lightweight streaming variant, VibeVoice-Realtime-0.5B, achieves a first-audible latency of approximately 300 milliseconds and is documented to support robust generation of up to 10 minutes of continuous speech. All models inherit characteristics of their base model, Qwen2.5 1.5B, including any biases or errors present in that foundation.
Who’s Affected
Developers building transcription pipelines, meeting-summarization tools, podcast automation systems, and accessibility software can now use VibeVoice-ASR directly through Hugging Face Transformers. The model’s stated support for more than 50 languages extends its potential applicability beyond English-only workflows. Researchers who intended to build on or replicate the TTS component face a narrower path: the synthesis code was removed from the public GitHub repository, though Microsoft has separately released finetuning code and the model weights remain accessible via Hugging Face.
What’s Next
Microsoft’s repository documentation explicitly limits VibeVoice to research and development use and recommends against commercial deployment without further evaluation. The company has added vLLM inference support for faster ASR throughput, and its changelog notes that additional speaker types will be added to the Realtime model over time. A technical report for VibeVoice-ASR is available through the project page for researchers seeking architectural detail beyond what the README provides.
