NVIDIA-NeMo /Speech
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
★ 18,754⑂ 3,693👁 230⑃ 12,688● 3,089▤ 87
machine-translationspeaker-recognitionasrttsgenerative-aideeplearningneural-networksspeaker-diariazation
TA的项目
+3
日增长
18,754
stars
3,693
forks
230
watchers
12,688
prs
3,089
issues
87
releases
README
NVIDIA NeMo Speech
Checkout our HuggingFace🤗 collection for the latest open
weight checkpoints and demos!
Updates
NeMo Speech 3.0 is now available as release v3.0.0
and in the 26.07.00 NeMo Speech NGC container.
The final NeMo release before the repository split was
v2.7.3, available in the
26.04 NeMo NGC container.
- 2026-10: Nemotron 3.5 Transcribe
is now available: an open-weight, offline speech LLM for 40 languages and recordings up to one hour.
Control transcription with instructions and vocabulary context, request speaker-attributed transcripts with
word timestamps, translate speech, and ask questions about audio. Start with the
NeMo Speech quickstart
or try the demo.
- 2026-07: MagpieTTS v2607 has been released with support
for 3 new languages (Ar, Ko, Pt) + 9 existing languages (En, Es, De, Fr, Vi, It, Zh, Hi, Ja). Try out
the demo!
- 2026-06: Nemotron-3.5-ASR-Streaming-0.6B has been released with 40 languages supported, controllable latency 80ms-1s, and 240-2400 1xH100 concurrent streams. Built on cache-aware Fastconformer architecture.
- 2026-04: Parakeet-unified-en-0.6b has been released with high-quality offline and streaming (with a minimum latency of 160ms) inference in one model for English language with punctuation and capitalization support.
- 2026-03: Nemotron 3 VoiceChat is now released in Early Access. Built on the Nemotron Nano v2 LLM backbone with Nemotron speech and TTS decoder, VoiceChat delivers full-duplex, natural, interruptible conversations with low latency. Try out the demo and apply for early access.
- 2026-03: Nemotron-Speech-Streaming v2603 has been
updated. It has been trained on a larger and more diverse corpus, resulting in lower WER across all latency modes.
Try out the demo and check out
the NIM.
- 2026-03: MagpieTTS v2602 has been released with support
for 9 languages (En, Es, De, Fr, Vi, It, Zh, Hi, Ja).
- 2026-01: Nemotron-Speech-Streaming was released: One checkpoint that enables users to pick their optimal point
on the latency-accuracy Pareto curve!
- 2026-01: MagpieTTS v2512 was released.
- 2026: This repo has pivoted to focus on audio, speech, and multimodal LLMs. For the final pre-split NeMo release with
support for additional modalities, see v2.7.3.
- 2025-08: Parakeet V3 and
Canary V2 have been released with speech recognition and translation
support for 25 European languages.
- 2025-06: Canary-Qwen-2.5B has been released with record-setting
5.63% WER on English Open ASR Leaderboard.
Introduction
NVIDIA NeMo Speech is built for researchers and PyTorch developers working on Speech models including Automatic Speech
Recognition (ASR), Text to Speech (TTS), and Speech LLMs. It is designed to help you efficiently create, customize, and
deploy new AI models by leveraging existing code and pre-trained model checkpoints.
For technical documentation, please see the
NeMo Speech Developer Documentation.
Requirements
NeMo Speech woTop issues
#13825Poor WER when trying to fine-tune Parakeet v2 TDT to other dataset than English54 comments · 2025-06-04#15037Intermittent CUDA Out of Memory Errors during Parakeet-TDT-0.6B Fine-tuning27 comments · 2025-11-06#14140TDT Head Stagnation in parakeet-tdt_ctc-110m Fine-tuning on Persian Data26 comments · 2025-07-06#14613SOS! Nemo:25.07 Container Image Cmpatibility mode is UNAVAILABLE ERROR!21 comments · 2025-08-29#15820nvidia/nemotron-3.5-asr-streaming-0.6b: transcribe() fails with "ValueError: Unknown prompt key: 'None'"10 comments · 2026-06-22#14427compute_rnnt_timestamps fails with empty char offsets10 comments · 2025-08-07#15615Crazy Logging I want to shut it down9 comments · 2026-04-16#15514Question: Canary-v2 hallucination in streaming mode8 comments · 2026-03-18
Discussions
#16460Report: single-speaker Parakeet TDT 0.6B v2 fine-tune for tech dictation (Fritter)0 comments · 2026-10-27#16404Norwegian support for lightweight Parakeet models0 comments · 2026-10-19#16163Is GPU-PB phrase boosting usable on Apple Silicon (MPS / CPU-only), or is it effectively NVIDIA-only?1 comments · 2026-08-29#16116Built double-tap-Ctrl dictation for Linux on nemotron-speech-streaming-en-0.6b0 comments · 2026-08-20#15900Nemotron 3.5 ASR Streaming 0.6B in native C++/GGML. RTF = 0.0066 on RTX 50900 comments · 2026-07-09#15731What If NeMo Had Actual Physics?0 comments · 2026-05-26#15667Nemo Toolkit is not in nvcr.io/nvidia/nemo:26.04.00?1 comments · 2026-05-06#15513About the NeMo Framework restructuring1 comments · 2026-03-18
Latest release
NVIDIA NeMo Speech 3.0 v3.0.0 2026-08-07NVIDIA NeMo Speech 3.0 Release Notes NeMo Speech 3.0 is the first major release after the repo split and rename to NVIDIA-NeMo/Speech. The repo now focuses on ASR, TTS, audio processing, speaker tasks, and SpeechLM. Non-speech Framework, LLM, VLM, diffusion, export/deploy, evaluator components can now be found under separate repos under NVIDIA-NeMo organization. While this release brings new major features, the central focus is addressing technical debt: removed 800k deprecated LOC, migration to uv package manager for cleaner installs, stronger model test coverage, reduced the number of