The design lens: voice as an act of intent
Every voice implies a relationship. A brisk, clipped agent tells the listener this will be transactional. A warm, unhurried one promises attention it may not be able to keep. Choosing a platform is therefore partly a choice about which register you can sustain — because the register the technology makes cheap is the one you will drift toward under deadline.
There is a genuine tension between the two dominant architectures, and it is aesthetic as much as engineering. Speech-to-speech models are quick and conversationally alive, but they blur the line between thinking and speaking; you cannot easily inspect what the agent decided before it said it. Cascade pipelines are a beat slower, yet every stage is legible: a transcript, a decision, a rendering. Legibility is worth paying latency for when the conversation carries consequence.
The most overlooked criterion is restraint. The best voice experiences say less than they could. Platforms that make it trivial to generate endless fluent speech quietly encourage a kind of verbal inflation. Whatever you choose, design the silences deliberately — they are the part listeners remember.
How we compare them
- End-to-end latency, not voice quality. The single number that decides whether a caller relaxes or talks over the agent is the gap between the end of their sentence and the first syllable of the reply. Under roughly 800ms a conversation feels human; past about 1.2s people start repeating themselves. Beautiful audio delivered late still sounds broken.
- Turn-taking and interruption. Real speech is full of overlaps, 'mm-hm's and false starts. Platforms differ enormously in whether they treat a half-second pause as your turn ending. Test with a noisy room and an indecisive speaker, not a clean script.
- All-in cost per minute. Hosted platforms quote a platform fee; model-agnostic ones quote a fee plus three vendor bills. Always model a realistic minute — including silence, hold time and the failed calls you still pay for — before comparing headline prices.
- Escape hatches. Ask what happens when the model is wrong: can a human take the call over, can you replay audio, can you export transcripts, and can you move to another vendor without rewriting your agent logic?
- Compliance and recording. Consent capture, retention windows, redaction of card and health data, and regional processing are not features you bolt on later. If your sector regulates calls, this criterion outranks latency.
The platforms at a glance
| Platform | Type | Architecture | Pricing shape | Best for |
|---|---|---|---|---|
| OpenAI Realtime API | Model API (build-your-own agent) | Speech-to-speech | Metered per audio minute in and out; no platform fee, but you pay for the orchestration you build. | Teams with engineers who want the lowest possible latency and full control of the stack. |
| Vapi | Voice agent platform | Either | Per-minute platform fee on top of the model, ASR and TTS vendors you select. | Engineering teams that want platform plumbing but refuse to be locked to one model vendor. |
| Retell AI | Voice agent platform | Cascade (ASR → LLM → TTS) | Bundled per-minute pricing that already includes model and voice costs. | Business teams shipping call-handling agents this quarter without a platform team. |
| ElevenLabs Agents | Voice-first platform | Cascade (ASR → LLM → TTS) | Character/minute credits, with voice quality effectively the premium you pay for. | Brands where how the agent sounds matters more than how the agent is wired. |
| Bland AI | Telephony-first platform | Cascade (ASR → LLM → TTS) | Flat per-minute, with enterprise tiers for dedicated infrastructure. | Contact-centre style workloads measured in tens of thousands of calls. |
| LiveKit Agents | Open-source framework + cloud | Either | Open source to self-host; usage-based if you take the managed cloud. | Product teams embedding voice inside their own application rather than on a phone line. |
| Pipecat | Open-source orchestration framework | Cascade (ASR → LLM → TTS) | Free framework; you pay only the underlying vendors. | Teams that want to measure the market honestly before committing to a vendor. |
Platform by platform
OpenAI Realtime API
Model API (build-your-own agent) · Speech-to-speech
Latency. Lowest class — a single model hears and speaks, so there is no transcribe-then-generate round trip.
Pricing. Metered per audio minute in and out; no platform fee, but you pay for the orchestration you build.
Strengths
- Genuinely interruptible conversation with natural barge-in and backchannel sounds.
- Tool calling in the same turn as speech, so the agent can look something up mid-sentence.
- No vendor layer between you and the model — nothing to migrate off later.
Trade-offs
- You own telephony, call recording, retries, analytics and compliance yourself.
- You cannot swap the language model for a cheaper one — the voice and the reasoning are the same model.
- Transcripts are a by-product rather than the source of truth, which complicates QA workflows.
Vapi
Voice agent platform · Either
Latency. Very good on a tuned cascade; competitive with speech-to-speech for short turns.
Pricing. Per-minute platform fee on top of the model, ASR and TTS vendors you select.
Strengths
- Model-agnostic: pick your own ASR, LLM and voice, then change any one of them without a rewrite.
- Strong developer surface — webhooks, server-side tool calls, call artifacts.
- Telephony, warm transfer and voicemail detection handled for you.
Trade-offs
- Costs stack: platform fee plus three vendor bills is easy to under-forecast.
- The flexibility is real configuration work; there is no good five-minute setup.
Retell AI
Voice agent platform · Cascade (ASR → LLM → TTS)
Latency. Consistently low, with turn-taking tuned for phone conversations rather than demos.
Pricing. Bundled per-minute pricing that already includes model and voice costs.
Strengths
- The most predictable bill of the hosted platforms — one number per minute.
- Turn detection and interruption handling work well out of the box on noisy phone lines.
- Practical guardrails: post-call analysis, structured extraction, compliance options.
Trade-offs
- Less freedom to hand-tune the pipeline than a model-agnostic platform.
- Voice library is good but not the widest available.
ElevenLabs Agents
Voice-first platform · Cascade (ASR → LLM → TTS)
Latency. Good; the voice layer is the fastest part of the chain.
Pricing. Character/minute credits, with voice quality effectively the premium you pay for.
Strengths
- The best-sounding synthetic voices in production use, including emotion and multilingual delivery.
- Voice cloning and consistent brand voice across an entire catalogue of content.
- Same voice engine powers narration, dubbing and live agents.
Trade-offs
- Agent orchestration is younger than the voice engine it sits on.
- Cost climbs quickly once you move from demos to sustained call volume.
Bland AI
Telephony-first platform · Cascade (ASR → LLM → TTS)
Latency. Tuned aggressively for outbound phone calls at volume.
Pricing. Flat per-minute, with enterprise tiers for dedicated infrastructure.
Strengths
- Runs its own stack end to end, so call quality is unusually consistent at scale.
- Purpose-built for high-volume outbound and inbound phone operations.
- Simple pricing that survives contact with a finance team.
Trade-offs
- Least flexible if you want a non-phone surface such as in-app or web voice.
- Fewer escape hatches when you need bespoke model behaviour.
LiveKit Agents
Open-source framework + cloud · Either
Latency. Excellent, with adaptive turn detection running close to the media layer.
Pricing. Open source to self-host; usage-based if you take the managed cloud.
Strengths
- You own the code path; the framework is inspectable and forkable.
- Real-time media is the core competency, so multi-party and video-plus-voice are natural.
- No per-agent platform tax if you run it yourself.
Trade-offs
- Self-hosting is a real operational commitment: scaling, media servers, observability.
- You assemble the agent behaviour; there is no non-technical builder.
Pipecat
Open-source orchestration framework · Cascade (ASR → LLM → TTS)
Latency. Depends entirely on the vendors you plug in; the framework itself adds very little.
Pricing. Free framework; you pay only the underlying vendors.
Strengths
- The clearest mental model of a voice pipeline: frames in, frames out.
- Swap any component — including to a speech-to-speech model — without changing your app.
- Ideal for research, evaluation and cost benchmarking across vendors.
Trade-offs
- No hosting, no dashboard, no compliance story out of the box.
- Every production concern is yours to solve.
What to pick, by situation
- A considered, brand-defining voice → ElevenLabs Agents. Delivery, emotion and multilingual consistency are the whole point here, and the same voice can carry every artefact you publish.
- Conversation that must feel alive → OpenAI Realtime API. Interruption and reaction inside a single model produce the closest thing yet to unforced dialogue.
- Conversations with consequence → Retell AI. A legible cascade with transcripts, structured extraction and review workflows — the right choice when someone may later need to know exactly what was said.
- An experience embedded in your own product → LiveKit Agents. You keep the code path and the media layer, which matters when the interaction is the product rather than a channel.
- Exploration before commitment → Pipecat. A clear pipeline abstraction that lets you audition combinations honestly, at the cost of building everything else.
The verdict
- For presence and craft: ElevenLabs, with OpenAI Realtime where immediacy outranks polish.
- For accountability: a cascade platform such as Retell, where the transcript is the record.
- For ownership: LiveKit Agents, self-hosted, when the experience must remain entirely yours.
- For clear-eyed evaluation: Pipecat, long enough to learn what your own conversations actually cost.
The question is never which platform is most advanced. It is which one lets you sustain the register you promised — and which one you would still be comfortable with if the listener knew exactly how it worked.