Engineering Transparency

    Voice AI Latency:
    How the Pipeline Actually Works

    No single number describes voice AI latency, and marketing sites exploit that ambiguity. Here is what the pipeline looks like stage by stage, which two measurements people conflate, and how to measure both on your own calls.

    The voice turn pipeline

    Every turn of a conversation travels through the same five stages. Each one streams into the next, nothing waits for a complete buffer before the next stage starts.

    01

    Telephony Audio

    The call arrives over your telephony provider at 8 or 16kHz audio, depending on the trunk and codec.

    02

    Streaming STT

    Speech is transcribed as it is spoken, Sarvam saaras:v3 for Indian languages or Deepgram nova-3 for global ones. Endpointing decides when you've finished talking.

    03

    LLM Streaming

    The model starts generating the reply token-by-token as soon as the transcript finalizes, instead of waiting to compose the whole answer.

    04

    Streaming TTS

    Synthesis begins on the first LLM tokens, Sarvam bulbul:v2, ElevenLabs, or Cartesia, streaming audio chunks back as they're generated.

    05

    WebRTC / SIP Playback

    Audio chunks are streamed back down the WebRTC or SIP leg to the caller, without waiting for the full sentence to be synthesized.

    Telephony failover runs across 8 providers: Exotel, Vobiz, Twilio, Plivo, Telnyx, Vonage, Cloudonix, Asterisk ARI. See the full provider matrix on AI Agent and Languages.

    Two numbers people conflate

    "Latency" in voice AI marketing usually means one of two very different measurements.

    Pipeline overhead / time-to-first-audio

    The gap between the LLM finishing its decision and the first audio frame reaching the caller. It's dominated by streaming chunking and TTS first-byte time, and it's the smaller of the two numbers because it ignores everything that happened before the model started answering.

    End-to-end turn latency

    What the caller actually feels: the moment they stop speaking to the moment the agent starts responding. Realistically ~500–800ms depending on STT finalization, LLM time-to-first-token, TTS time-to-first-byte, telephony round-trip, and deployment region.

    About "<200ms" claims

    You may see "<200ms" claims on voice AI marketing sites. That figure describes pipeline overhead at best, not end-to-end turn latency. If a vendor won't say which measurement they're quoting, or over which network and region it was measured, the number isn't comparable.

    What drives your number

    Turn latency isn't a property of a platform, it's a property of the exact stack and region you deploy. Six factors dominate it.

    STT provider & endpointing

    How aggressively the transcriber finalizes a turn. More conservative endpointing is more accurate on long pauses, but adds delay before the LLM ever sees text.

    LLM first-token time

    Time to first token varies by model, prompt size, tool-call depth, and provider load. Streaming output means the TTS doesn't wait for the full reply.

    TTS time-to-first-byte

    How quickly the synthesizer emits its first audio chunk. Streaming TTS pipelines start speaking mid-generation; chunked ones wait for sentence boundaries.

    Telephony codec & round-trip

    8kHz PSTN audio, codec conversion, and network distance between the caller's carrier and the pipeline region all add transport milliseconds.

    Deployment region

    A pipeline hosted closer to the caller's telephony POP cuts round-trip time on every stage of the turn, this is why region choice matters for India traffic.

    Language & model choice

    Language-aware stacks (Sarvam for Indian languages, Deepgram/ElevenLabs for global) differ in how they endpoint, tokenize, and synthesize, and that changes the number.

    Measure it yourself

    Your logs are the benchmark

    The only latency figure worth trusting is the one measured on your calls, your region, and your provider stack. Crawlii's workspace dashboard logs per-tool latency and execution timings for every turn, so you can see exactly where each millisecond went instead of trusting a marketing table.

    Run a live agent in the Playground, place a real call through one of the 8 telephony providers, and read the breakdown in your workspace, or walk through it with our team on a demo call.

    Methodology, not magic numbers

    • Measure end-to-end: caller silence ends → first agent audio reaches the caller.
    • Fix the variables: same STT/LLM/TTS stack, same telephony provider, same region.
    • Average across many turns — endpointing and provider load make single turns noisy.
    • Compare like with like: a "<200ms" figure that excludes STT and telephony isn't a turn latency.

    Hear it, then measure it

    Put a live agent on a real call and read the latency breakdown in your own dashboard.