Loading…
Product Visit
Visit tomorrow's products today
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
audio.cpp is a high-performance C++ audio inference framework built on top of ggml, designed to make modern local audio models practical, portable, and fast.
Tired of juggling a dozen Conda environments, hundreds of Python packages, and dependency conflicts just to try a few audio models? audio.cpp gives those paths a shared native runtime instead.
Important
CUDA performance headline: multiple TTS paths already run 1.8x-5.0x faster than their Python reference paths while cutting end-to-end latency by 45%-80%. VibeVoice 1.5B: generates a 93.9-minute podcast in 18.2 minutes with 10 diffusion steps and without quantization, running about 5.15x faster than real time.
It is built for real end-to-end execution rather than one-off model demos: the same runtime powers TTS, voice cloning, voice conversion, ASR, diarization, VAD, source separation, alignment, codec-style models, and higher-level workflows through a common framework surface.
Highlights:
ggml, with CLI and server entry points instead of Python-only deployment paths.The goal of the framework is to provide highly optimized, reusable building blocks for audio-related models, so new model integrations can be brought up faster, shared components can be improved once and benefit many families, and real end-to-end inference paths can stay efficient, maintainable, and portable.
Tip
Contribution focus: the most helpful contributions right now are improvements to the UI, API server, and pipeline/workflow subsystems. These areas make the existing model surface easier to use, serve, compose, and validate. See CONTRIBUTING.md for more details.
New model PRs: before starting a new model port, please check the supported model table because several families are already implemented or under testing. If you do add a model, follow the validation style in PR #19: include exact build/run commands, model paths or package ids, generated outputs, parity or path-test results, and relevant performance or memory notes.
Important
2026-07-08: Four new ASR families are now released in the framework: Higgs Audio STT, Hviske ASR, Nemotron ASR, and VibeVoice ASR. Initial model-specific streaming support also lands for VoxCPM2 TTS, Nemotron ASR, and Higgs Audio STT, with server SSE configuration and request examples for streaming speech generation and transcription.
Important
2026-07-03: Conv1DTransp module CUDA optimization: VibeVoice reaches 5.15x realtime on 93.9-minute long-form generation. Overall, VibeVoice inference time was reduced by 73.17%, PocketTTS by 35.32%, Chatterbox by 33.56%, Qwen3-TTS by 30.60%, HeartMuLa by 17.03%, and VoxCPM2 by 14.7% compared with the previous release.
Important
2026-07-02: Music generation and source separation expanded in the released framework surface: ACE-Step 1.5 Turbo/Base, HeartMuLa, Stable Audio 3 Small Music/SFX and Medium, Mel-Band RoFormer, and HTDemucs are now available through the normal audio.cpp CLI/framework paths.
--load-option vibevoice.lora.Current model status in the framework:
released: The model is fully wired into the broader framework surface and ready for normal use.testing: The model is implemented and working in this repo, and is still being validated, polished, or promoted into the broader released surface.optimization: The model is end-to-end working, but still needs more optimization work before it should be treated like a released or testing-level path.| Family | Task | Supported language(s) | Supported variant(s) in this repo | Release status |
|---|---|---|---|---|
| ace_step | music generation, music editing | 50+ langs | ACE-Step 1.5 Turbo and Base with acestep-5Hz-lm-1.7B | released |
| chatterbox | TTS, voice cloning | ar, da, de, el, en, es, fi, fr, hi, it, ko, ms, nl, no, pl, pt, sv, sw, tr | Chatterbox with 0.5B backbone | released |
| citrinet_asr | ASR | en | Citrinet-256 | released |
| heartmula | music generation | zh, en, ja, ko, es | HeartMuLa-oss-3B with HeartCodec-oss | released |
| higgs_audio_stt | ASR | en | Higgs Audio v3 STT | released |
| htdemucs | source separation | lang agnostic | HTDemucs, HTDemucs_ft | released |
| hviske_asr | ASR | da | Hviske v5.3 | released |
| marblenet_vad | VAD | lang agnostic | MarbleNet VAD | released |
| mel_band_roformer | vocal separation | lang agnostic | Mel-Band RoFormer MLX vocal separation variants | released |
| miocodec | audio codec, voice conversion backend | lang agnostic | MioCodec v2, 25 Hz, 44.1 kHz | released |
| miotts | TTS, voice cloning | en, ja | MioTTS-1.7B | released |
| omnivoice | TTS, voice cloning, voice design | 646+ langs | OmniVoice, Qwen3-0.6B based | released |
| pocket_tts | TTS, voice cloning | en, de, it, pt, es | PocketTTS-100M | released |
| nemotron_asr | ASR | 100+ ASR prompt codes incl. auto | Nemotron 3.5 ASR Streaming 0.6B | released |
| qwen3_asr | ASR | zh, en, yue, ar, de, fr, es, pt, id, it, ko, ru, th, vi, ja, tr, hi, ms, nl, sv, da, fi, pl, cs, fil, fa, el, ro, hu, mk | Qwen3-ASR-0.6B | released |
| qwen3_forced_aligner | forced alignment | zh, yue, en, de, es, fr, it, pt, ru, ko, ja | Qwen3-ForcedAligner-0.6B | released |
| qwen3_tts | TTS, voice cloning, voice design | zh, en, fr, de, it, ja, ko, pt, ru, es | Qwen3-TTS-12Hz-0.6B-Base, Qwen3-TTS-12Hz-1.7B-Base, Qwen3-TTS-12Hz-1.7B-CustomVoice, Qwen3-TTS-12Hz-1.7B-VoiceDesign | released |
| seed_vc | voice conversion | lang agnostic | SeedVC XLS-R + HiFT, SeedVC Whisper-small + BigVGAN | released |
| silero_vad | VAD | lang agnostic | Silero VAD | released |
| sortformer_diar | diarization | en | Sortformer-4spk-v1 | released |
| stable_audio | music generation, sound generation, audio editing | en | Stable Audio 3 Small Music, Stable Audio 3 Small SFX, Stable Audio 3 Medium | released |
| vevo2 | TTS, singing generation, voice conversion, singing conversion, editing | en, zh | Vevo2 with Qwen2.5-0.5B AR model | released |
| vibevoice | TTS, multi-speaker dialogue TTS | en, zh | VibeVoice-1.5B, VibeVoice-7B | released |
| vibevoice_asr | ASR | auto | VibeVoice ASR | released |
| voxcpm2 | TTS, voice cloning, voice design | ar, da, de, el, en, es, fi, fr, he, hi, id, it, ja, km, ko, lo, ms, my, nl, no, pl, pt, ru, sv, sw, th, tl, tr, vi, zh | VoxCPM2-2B, 48 kHz | released |
| higgs_tts | TTS, voice cloning, expressive speech | 100+ languages | Higgs Audio v3 TTS 4B | testing |
| index_tts2 | TTS, voice cloning, expressive speech | zh, en | IndexTTS-2 | testing |
| irodori_tts | TTS, voice cloning, voice design | ja | Irodori-TTS-500M-v3, Irodori-TTS-600M-v3-VoiceDesign | testing |
| kokoro_tts | TTS | en-us, en-gb | Kokoro-82M | testing |
| moss_tts | TTS, voice cloning | zh, yue, en, ar, cs, da, nl, fi, fr, de, el, he, hi, hu, it, ja, ko, mk, ms, fa, pl, pt, ro, ru, es, sw, sv, tl, th, tr, vi | MOSS-TTS-Local | testing |
| supertonic | TTS | en | Supertonic 3 | testing |
PocketTTS language selection is a model-load option. When the model path points at the PocketTTS root, the loader uses english unless you pass --load-option language=<name>. Kyutai's normal non-English PocketTTS releases are smaller distilled language models intended for the fast PocketTTS path. The _24l variants are larger 24-layer, undistilled preview models that can sound better but are slower. Kyutai currently publishes French only as french_24l, not as a normal distilled french language directory, so French is not listed as a normal PocketTTS language here.
Docker CPU and CUDA images are available for both CLI and server use. See Docker.md for build commands and working Docker examples.
On Linux, use a normal CMake build directory such as build/.
For single-config generators, the default build type is RelWithDebInfo.
That default configure is a CPU build unless you enable an accelerator backend explicitly.
Use GCC 13 or newer for Linux builds.
Native ggml CPU optimization is enabled by default for local performance. If your compiler or assembler rejects a generated CPU instruction such as vpdpbusd, reconfigure with -DENGINE_ENABLE_NATIVE_CPU=OFF to build portable CPU kernels.
Common Linux configure examples:
CPU-only:
cmake -S . -B build
CUDA:
cmake -S . -B build -DENGINE_ENABLE_CUDA=ON
Vulkan:
cmake -S . -B build -DENGINE_ENABLE_VULKAN=ON
Portable CPU-kernel fallback:
cmake -S . -B build -DENGINE_ENABLE_NATIVE_CPU=OFF
Build the CLI and server from the configured tree:
cmake --build build --parallel --target audiocpp_cli --target audiocpp_server
If you use an environment manager or custom toolchain, activate it before running the commands above.
The optional Linux helper script wraps the same CMake flow and uses aligned build directory names:
build/linux-cuda-releasebuild/linux-vulkan-releasebuild/linux-cpu-releaseExamples:
scripts/build_linux.sh --backend cuda --target audiocpp_cli --target audiocpp_server scripts/build_linux.sh --backend vulkan --target audiocpp_cli --target audiocpp_server scripts/build_linux.sh --backend cpu --target audiocpp_cli --target audiocpp_server scripts/build_linux.sh --backend cuda --native-cpu OFF --target audiocpp_cli --target audiocpp_server
Use --build-dir <dir> only when you intentionally want a custom output directory.
The recommended native Windows build is command-line only:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\... for CUDA buildsUse MSVC cl.exe as the compiler. For CUDA builds, cl.exe is also used as the CUDA host compiler. Native Windows nvcc does not support clang-cl as its host compiler, and the Visual Studio IDE is not required.
From PowerShell:
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\build_windows.ps1
CPU-only:
.\scripts\build_windows.ps1 -Preset windows-cpu-release -Target audiocpp_cli
From cmd.exe:
scripts\build_windows.cmd
If GNU Make is available on Windows:
