Voxnera — Real-Time Voice Translation for Any Call

ROLE

End-to-end product engineering: audio path (WASAPI, VAD, AEC, routing), AI routes, Qt/QML interface, local storage, installer and Microsoft Store package

PROBLEM

An interpreter is needed exactly while the conversation is happening — and everything that translates well translates afterwards. The classic recognise, translate, speak chain is built so that the recogniser emits no text at all until it decides a phrase has ended; only then can translation start, and synthesis after that. The wait is the sum of three waits, and it cannot be configured away, because it is a property of the order of the steps. Meanwhile the translated voice has to arrive at Discord or Zoom as microphone input, and Windows offers no user-mode way to create a microphone at all.

SOLUTION

Voxnera keeps both routes and takes the short one by default: a single speech-to-speech model hears, translates and speaks together, so it answers while the other person is still talking. Around it sits everything the model does not do — a voice gate that measures the room rather than assuming it, a speaker check so only the owner’s voice is sent, a speech queue that stops the translation interrupting itself, and an output written into a virtual cable that the calling application listens to as an ordinary microphone. Both directions run at once, each with its own languages and voice, and both are written into a local transcript. The long three-provider route stays available for when an independent transcript matters more than the wait.

OUTCOME

A conversation runs in two languages at once instead of turning into a walkie-talkie. The latency is visible in the window rather than promised — 408 ms against 2689 ms, measured the same way on the same machine. The transcript stays as text on that machine, the keys are encrypted by Windows, and audio is not recorded until somebody turns it on. The business gets a product that is a platform rather than a wrapper around one vendor — every provider is an adapter behind an interface, and swapping one is a setting.

Both sides, at the same time

Two independent directions, each with its own languages, its own voice and its own switch — not one channel taking turns.

• Either direction can run untranslated while the other is translated
• The microphone keeps reaching them even when nothing is being translated

One model instead of three

A speech-to-speech model hears, translates and speaks together, so it begins while the other person is still talking: 408 ms against 2689 ms for the recognise-translate-speak chain.

• Both routes stay available — the long one gives an independent transcript
• Measured the same way, on the same machine, on the same translation

Only your voice is sent

Three short readings are averaged into 256 numbers — not audio, and not something that can be played back. Anyone else is cut about half a second in.

• Your own voice is never delayed — audio is sent from the first moment
• A model that fails to load counts as the user, never as a stranger
• Push-to-talk is checked before the gate and before the model

The translation reaches Discord and Zoom

Windows has no user-mode way to create a microphone, so the route runs through a virtual cable — and the application names the exact device to select rather than asking the user to guess.

• The installer carries the cable, so the setup is one run rather than two
• The same interface behind both routes, so the driver replaces a device id — not an architecture

A transcript, not a recording

Both sides of every call are kept as text on that machine, searchable and exportable. Audio stays off until it is deliberately turned on.

• Full-text search across every call, through SQLite FTS5
• Export to TXT, Markdown, JSON, SRT and VTT
• API keys encrypted for this Windows account and never written in readable form

The window gets out of the way

Translating is something done while doing something else, and the window covers the very thing being translated. During a session, closing or minimising hands over to a small panel in the corner rather than ending the call.

• Subtitles draw over everything and let the mouse through
• Pause stops feeding the providers without dropping their sockets, so resuming is instant
• Global keys work while the window is behind whatever is being translated

Core

• C++23, CMake + MSVC
• Qt 6.8.3 LTS / QML, view models in C++
• Layered architecture — dependencies point inward
• No JavaScript in the real-time audio path

Audio

• WASAPI capture and playback
• Per-process loopback (Windows 10 19041+)
• Lock-free ring buffers, resampling
• Voice gate, echo cancellation, mel-band spectrum

AI routes

• OpenAI Realtime — speech to speech
• OpenAI STT / translation / speech synthesis
• Deepgram streaming STT over WebSocket
• Mock providers, so the pipeline runs without credits

Voice

• 256-d speaker embedding via ONNX Runtime
• Enrolment averaged over three takes
• Pitch estimation for automatic voice matching
• Push-to-talk over a low-level keyboard hook

Data & security

• SQLite + FTS5, versioned migrations
• Windows DPAPI for credentials
• Secrets redacted from the logs
• Export to TXT, MD, JSON, SRT and VTT

Delivery

• Inno Setup installer, virtual cable in the box
• MSIX package for the Microsoft Store
• WDK virtual microphone driver (built, signed)
• 468 tests on GoogleTest, audio path included

End-to-end latency

408ms from the end of a phrase

Against the provider chain

6.6× faster, measured the same way

Tests

468pass, audio path included