Voice agent latency is decided before pickup

Updated 14 September 2026 with production numbers, figures, and what we learned running it.

Voice agent latency on Talkif is low for a simple reason: by the time the call connects, almost nothing is left to do. The speech models were loaded when the process started. The connections to the transcriber and the voice were opened while the phone was still ringing. The first thing the agent says was generated by the language model, in full, before anyone picked up. When the media stream finally arrives, the bot attaches it, and plays a greeting that already exists. How a call reaches a warm bot in the first place is in how a call gets a bot.

Measured from the moment the media stream connects to the first audio the bot sends, over the last 90 days of production calls that lasted at least a minute (167 calls, to 12 September 2026), the median is 380 ms, and 76 % of calls heard a voice within one second. Inbound calls are faster than outbound — 252 ms against 406 ms at the median — because on an inbound call the bot has been warm since before the phone was answered.

The distribution is not a smooth curve. 121 of the 167 calls started speaking within 480 ms; most of the rest sit in a second cluster between 1.7 and 2.2 seconds — the time to first audio, or TTFA as the pipeline metrics call it, is really two numbers. Those are the calls where the shortcut described below could not be used, and the bot had to do the slow thing live. The rest of this post is about the shortcut, and then about why it sometimes cannot be taken.

The bar is set by people, not by software

In ordinary conversation, the gap between one person finishing and the next starting is about a fifth of a second. Reviewing the turn-taking literature, Holler, Kendrick, Casillas and Levinson put it at "gaps between speaking turns averaging around just 200 ms (Stivers et al., 2009)" — and note in the same paragraph that "the time it takes to produce even simple one-word-utterances" is "min. 600 ms" (Holler et al., 2015). Humans answer faster than they can physically form a word. The only way that works is that they prepare the answer while the other person is still talking.

The people who build these pipelines have converged on a number for the whole round trip. Kwindla Hultman Kramer, co-founder of Daily, writing in June 2025: "A good rule of thumb is that you should be aiming for 800ms median voice-to-voice latency (eventually)" — with a budget that gives the network, turn detection with transcription, the language model and speech synthesis each a slice of it (Advice on building voice AI in June 2025). The greeting is the one turn that has no caller speech to respond to, so nothing in that budget has to be spent on it — if the work was done before the call.

That is the whole design brief. A bot that starts working when the caller connects — load a model, open a socket, ask the LLM — cannot answer in 200 ms, or in two seconds. A bot that does that work while the phone is ringing can. Everything below is about moving work to the left of the moment the call connects.

The cold start is the smaller of two warm-ups

The first warm-up happens once, when a bot process starts, before it announces itself as available. It loads the voice-activity detector (Silero VAD) and the turn-detection model into memory and pre-imports the modules the pipeline needs. Measured over 302 process starts between 14 August and 12 September 2026, that takes a median of 1.4 s (p90 2.2 s); the VAD load alone is about 0.1 s and the turn model 0.08 s. The process does not accept a call until this is finished, so a caller never pays for it.

This is the cold start everyone talks about, and removing it matters — but it is the small half. Nothing in it is specific to the call, because at process start the bot does not know which flow it will run, which transcriber and voice the flow uses, or what the agent is supposed to say. All of that arrives with the assignment.

The decisive warm-up happens in the shadow of the ring

When the backend assigns a call to a bot, it sends the flow to run. From that moment until the caller's audio arrives there is a gap — for an outbound call it is the whole of the carrier dialling and the phone ringing; for an inbound call it is the carrier setting up the media stream — and the bot uses every millisecond of it.

It loads the flow from a per-call cache entry, builds the pipeline, and then, in parallel: opens the WebSocket to the speech-to-text provider, opens the WebSocket to the text-to-speech provider, and runs the agent's first turn through the language model. Not a placeholder — the real first turn, with the flow's prompt, its tools and its context, exactly as the live pipeline would run it. Over 306 assignments in the same window, this parallel step took a median of 2.1 s and a p90 of 3.5 s. That is two to three seconds that would otherwise sit between "hello?" and the first word, spent instead while a phone rings.

Serverless platforms sell the same idea. AWS describes Lambda's provisioned concurrency as "pre-initialized execution environments" that "are ready to respond immediately", and is candid about the cost: "Lambda bills you for initialization even if the environment instance never processes a request" (AWS Lambda docs). The same trade holds here: a pre-warmed bot spends a language-model call on a greeting even if nobody picks up.

The greeting is injected as if the model had just said it

The pre-generated first turn is not read from a cache into the audio stream. When the caller connects, its text is pushed through the pipeline as the same sequence of frames a live model response produces, carrying the same timing and token metrics the live turn would have had. Two things follow. The conversation context records the greeting as something the assistant said, so on the next turn the model remembers what it opened with and does not repeat itself. And the billing and the dashboard see a real turn, with a real time-to-first-token, not a free one.

If the pre-generation fails, or the model answers the first turn with a tool call instead of words, the stored response is empty and the live pipeline generates the greeting the ordinary way: a full language-model turn, then speech synthesis, after the caller is already listening. That is the second cluster in the distribution above — the two-second calls are the ones that had to take the slow path. The fast path is an optimisation over a working slow path, never a dependency, and the histogram is an honest record of how often each one runs.

What remains on the critical path when the media arrives: attach the socket, start the pipeline, and synthesise the greeting over a text-to-speech connection that is already open. On the browser side the greeting generation was moved out of the warm-up altogether in May 2026, to run concurrently with the browser building its connection offer — it took the warm-up on that path from about 3.5 s to about 0.4 s.

What it cost to learn

Fast greetings created a failure mode that did not exist when greetings were slow. With the bot ready to speak the instant the stream attached, line noise from the caller's side — a click, a breath, the carrier's own audio settling — could register as speech and interrupt the greeting before any of it had reached the caller. Worse, the half-said greeting was committed to the conversation context as if it had been delivered, and the model carried on from a sentence the caller never heard. The fix, in August 2026, is to hold the caller's input until the first audio has actually been sent, and to backfill the full greeting into the context if it was cut short.

Pre-connecting had its own lesson. One provider's connection was being opened before the pipeline had decided the output audio format, so the connection request went out with an empty format field, and the provider answered every voice with HTTP 403. Warming up earlier means some of the inputs you depended on are not ready yet; each pre-connect had to be checked for what it silently assumed.

And the pool is only fast while it is warm. Scaling bots from zero — a new pod plus, sometimes, a new machine — takes 60 to 90 seconds, so the pool never sits at zero and the autoscaler stays one bot ahead of the current load. An outbound call that finds no free bot waits invisibly, because nobody is on the line yet. An inbound call that finds none hears a busy tone rather than dead air.

What it does not do

Speech-to-speech models, which take audio in and produce audio out in one model, cannot be pre-connected: their receive loop needs the running pipeline, which does not exist until the call starts. Flows built on them skip this optimisation.

A bot serves one call and then re-warms. Everything call-specific is thrown away after hang-up; only the process-level models persist to the next assignment.

And the bot still has to wait for the carrier. Pre-warming removes the bot's own latency from the first word; it does nothing about how long a phone takes to ring.