Ito: Streaming Neural Speech on a $5 Chip
A 4.9 MB text-to-speech model for the ESP32-S3. It streams, runs without a cloud and is verified in emulation. Board timings to come.
Real production measurements, not marketing rounding: where the time goes in a turn, and how each of our models compares, with the methodology to reproduce it.
In real-world voice AI, TTS latency alone doesn't tell the story. We measure time to first audio: from the moment the user stops speaking to the first sound of the reply. The agent streams token by token, so the caller never waits for a whole sentence — only for the first short chunk.
Measured on production on 10 October 2026: 30 calls, one at a time, from a machine in the same AWS region as our servers (Ireland), so your own internet connection is not in the figure. Each call says one Spanish sentence after the agent's greeting; the stages come from our servers' own log of each of those turns. Hover a segment.
Ours The language-model provider, and delivery to the caller
The 596 ms is the median of the 30 calls’ own totals. Each stage is shown at its own median, and those add to 591 ms: the median of a set of totals is not the sum of the medians of its parts.
| Stage (median of 30 calls) | Latency | Runs on |
|---|---|---|
| Waiting to be sure the caller has finished | 180 ms | Our servers |
| Speech recognition (Conv 1.0) | 0 ms on the critical path — it completes inside that wait | Our GPU servers |
| Language model: the reply, started while the caller was still finishing | 225 ms | Provider |
| Speech synthesis, to first audio (Versa 2.0) | 64 ms — between 63 and 67 ms on every call | Our GPU servers |
| Delivery to the caller | 122 ms — the gap between our servers' clock and the caller's, the same on every call | — |
| Caller-observed time to first audio | 596 ms median — fastest 557 ms, 90th percentile 925 ms | — |
The language model starts on the caller's words before they have finished; on 26 of the 30 calls its reply was ready by the end of the wait, and on the other 4 the caller waited 925–1,137 ms. Published guidance treats under 800 ms as conversationally natural; an independent survey places production voice-AI platforms at a 1,400–1,700 ms median. Streaming text to speech on its own reaches the first audio in 65 ms (median of 40 requests on an open connection; about 200 ms for the first request on a new one). In September 2026, when everything but the language model still ran on CPU-only servers (AWS c8g.2xlarge, Graviton4), callers waited 385–442 ms in the best case and about 660 ms typically, and Versa's first audio took a median of 139 ms: on CPUs alone, already in real time.
About 5.2¢ a minute for a voice agent at a typical prompt: 4.2¢ for the platform, with speech recognition, the voice, turn-taking and orchestration inside it, plus the language model by the token. Prepaid and billed by the second; there are no plans.
| Line | Price | Billed |
|---|---|---|
| Voice agent, platform | $0.042 / min | Per second |
| Language model, input / output | $0.44 / $0.94 per million tokens | Per token |
| A typical minute, all in | 5.2¢ | Web or app |
| Phone line, call received in Spain | 3.5¢ / min | Per started minute, on top of the agent |
| Text-to-speech API | $4.10 per million characters | Per character |
| Speech-to-text API | $0.0014 / min | Per second |
Phone numbers $5 a month each; every line and country on the pricing page. Dedicated and private deployments are quoted separately.
Psst is our proprietary strictly-causal noise suppression model, running as low-latency middleware ahead of speech recognition. We score it on downstream word-error rate — the only metric that reaches the caller — not on perceptual audio quality, which turned out to point the wrong way entirely.
| System | Avg. noisy WER | PESQ | SI-SDR | RTF (1 core) |
|---|---|---|---|---|
| Psst (ours, in production) | 21.74% | 2.096 | 9.89dB | 0.053 (int8) |
| No enhancement | 25.32% | 1.970 | 8.44dB | — |
| Our previous suppressor | 28.91% | 2.338 | 14.68dB | 0.102 |
| DeepFilterNet3 (published SOTA) | 33.79% | 2.794 | 17.69dB | 0.058 |
300 LibriSpeech test-clean utterances mixed with held-out noise and interfering speakers; no evaluation audio appears anywhere in training. Psst leads on downstream word error by 36% over DeepFilterNet3, the strongest published real-time CPU denoiser, at comparable cost and inside a 32ms strictly-causal budget DeepFilterNet3 doesn't meet. Against competing speech — other people talking nearby, the most common real-world complaint — Psst recovers 14–28% of word errors. It runs 18 concurrent call legs per CPU core at 2.7MB, so noise suppression adds no meaningful capacity cost.
The word-error and perceptual rankings invert completely. DeepFilterNet3 leads PESQ by 0.70 and SI-SDR by 8dB and is the worst system here for transcription — worse than no denoising at all. Psst is the only system tested that improves on doing nothing. Optimizing for the industry-standard perceptual metric would have degraded the product.
Turno, our turn-taking and barge-in model, is 18,651 parameters. On a neutral real-world test set it beats Silero VAD on Average Precision (0.931 vs 0.876) and holds that precision flat from clean audio down to 0dB signal-to-noise, at roughly half Silero's per-inference cost (0.083ms per frame). TEN-VAD is ahead on voice activity alone (0.985); Turno also predicts the end of a turn and tells a real interruption from a backchannel, which voice-activity models do not. Retraining on real echo data cut the real-device echo false-confirm rate from 93.0% to 11.4%.
Conv 1.0 serves production today: NVIDIA’s open Parakeet TDT 0.6B model, served on our own GPUs with no cloud fallback — about 45 ms to transcribe a four-second sentence, and around 36 utterances per second per card under sustained load.
Conv 2.0, our own streaming recogniser and our next release: 450 million parameters, a quarter smaller than Parakeet 0.6B. Scored with the Artificial Analysis streaming benchmark protocol (AA-WER Streaming) and its published normalisation, on our own harness; the conversational set is our public stand-in for Artificial Analysis's held-out agent and customer set.
| Benchmark | Conv 2.0 |
|---|---|
| Weighted conversational WER (AA-WER-style) | 5.59% |
| Conversational set (50% of the score) | 4.31% |
| VoxPopuli (25%) | 4.06% |
| Earnings22 long-form calls (25%, finals cut at 500 ms pauses) | 9.70% |
| Time to final, median / p95 | 14.1 ms / 18.9 ms, server-side on one NVIDIA L4 GPU, network not included |
State-of-the-art voice AI today means giant models on giant GPU fleets. That model breaks down the moment voice AI needs to live outside a data center.
A 4.9 MB text-to-speech model for the ESP32-S3. It streams, runs without a cloud and is verified in emulation. Board timings to come.
English and Spanish speech recognition on a $5 ESP32-S3 at 3.7% word error. Accurate, but about twice slower than real time. Open source.
The plans announced on 23 September 2026 at about 2¢ a minute, replaced on 9 October by a prepaid balance at about 5¢ a minute, model included.
Robots, wearables and connected devices can't carry a GPU cloud bill for every unit sold. Why voice AI is following compute back to the edge.
CPU-native voice AI runs recognition, understanding and speech synthesis on ordinary processors instead of GPUs. What it means and why it matters.
Want to run these benchmarks on your own data? Contact us for a technical consultation or access to our evaluation scripts.