What is latency? Definition, causes and how to cut it
What latency means, where the delay comes from, how to measure it, and why it decides whether a voice AI agent sounds human.
Latency is the delay between an action and its response, measured in milliseconds. On a network, it's the time a piece of data takes to travel from one point to another; in a voice AI agent, it's the pause between the moment you stop talking and the moment the agent answers.
The word covers a lot of ground: network latency, audio latency, AI model latency. This page explains the idea without jargon, shows where the milliseconds come from, separates latency from bandwidth, throughput and jitter, then walks through the tools voice AI teams use to keep a conversation fast. We compared six of them on what they publish about speed and price.
Some links to tools may be affiliate links. The definition and our take on each tool stay the same either way.
How latency works: distance, hops and processing time
Latency is a time, not a speed. Picture a server in New York sending a packet to a server in London: it leaves at 0.000 seconds and arrives at 0.145. The latency on that path is 145 milliseconds. When people say a connection is "laggy", they are talking about that number.
Two ways of counting get mixed up all the time. One-way latency is the trip in one direction. Round-trip time (RTT) is the trip there and back, roughly double. Most tools, ping included, show you the round trip.
Where do the milliseconds come from? Four sources add up:
- Distance. One of the main causes. A site hosted in Columbus, Ohio answers users in Cincinnati, about 100 miles away, in 5 to 10 ms; users in Los Angeles, about 2,200 miles away, wait closer to 40 to 50 ms.
- The medium. Light in a fiber optic cable is slower than in a vacuum: about 4.9 microseconds per kilometer instead of 3.33. Copper, Wi-Fi and mobile networks add their own delays.
- Hops. Data crosses several networks and exchange points. Each router reads the packet, sometimes splits it, and forwards it, adding a little time at every hop.
- Processing and queues. A busy server, a congested link or a slow disk makes packets wait. In AI systems, the model's thinking time is part of this bucket, and often the biggest part.
The word changes meaning with the layer you look at. Network or internet latency is the trip between two machines. Fiber optic latency is the time light takes through a cable. Audio latency is the gap between a sound and its playback. Interrupt latency is how long a computer takes to act on a signal from its hardware. Operational latency is the sum of steps run one after the other in a workflow, which is exactly how a voice agent behaves.
A detail most explanations skip: the first request to a site is slower than the next ones. Before any content flows, the browser looks up the address (DNS), opens a connection (the TCP handshake) and sets up encryption (TLS). Once the connection exists, later requests skip those steps.
Latency also stacks. A web page needs dozens of files; a voice agent runs several models in a row. A delay that looks tiny on one request becomes a visible wait once it repeats ten times.
A worked example: the latency budget of a voice AI agent
Voice is where latency hurts most. In a chat window, a two-second wait is fine. On the phone, a two-second silence after you finish a sentence makes the caller think the line dropped, and they start talking again. Listeners notice audio delays from around 30 ms, and research on human turn-taking across 10 languages measured a mean gap of about 208 ms between a question and its reply. That's the bar an agent is compared with, often without the caller knowing it.
Here's what happens, stage by stage, when a caller asks an AI agent "Can I move my appointment to Friday?":
- Turn detection. The system must decide the caller has finished. Wait too long and you add dead air; cut in too early and the agent interrupts. This setting alone can add hundreds of milliseconds.
- Speech-to-text. A streaming transcription model turns the audio into words as they arrive.
- The language model. It reads the transcript, maybe calls a calendar tool, and writes an answer, the same loop as any AI agent. It often weighs the most, and it grows with the prompt's size and with every tool call: one builder with a very long prompt reported 3 to 7 second pauses while the agent "thinks".
- Text-to-speech. A voice model starts speaking the answer. The fastest voice models publish median delays around 100 ms.
- Telephony and network. The audio crosses the phone network or the internet in both directions, through your carrier or a virtual phone system.
Add it up and you see why platforms advertise their total: Retell, for example, puts its voice agents at about 600 ms. That figure is a target to compare against, not a guarantee for your setup: your model, your tools and your callers' location move it.
Four pitfalls we found while comparing the stacks, which the general explainers don't cover:
- Voice model figures leave out the network. ElevenLabs' ~100 ms for Flash v2.5 is a median that excludes application and network latency. Count it as one line of the budget, not the whole bill.
- Speed has a price tag. On Retell, the "Fast Tier" of a model can double its per-minute cost: GPT 5.6 Terra goes from $0.064 to $0.128 a minute. Test whether callers notice the difference before you pay for it.
- A patience setting can add seconds on purpose. Retell's Response Wait time adds a deliberate pause before every answer, up to 5.5 seconds at its most patient. If the agent feels slow while each stage looks normal, lower that setting before you change any model.
- Hosting far from callers costs you twice per turn. At 4.9 microseconds per kilometer, a New York to London fiber path of about 5,600 km adds around 27 ms each way, so roughly 55 ms of round trip at every exchange, before any routing detours. Put the agent in the same region as the phone numbers.
How to measure latency and how to reduce it
Measuring. The quickest test is the ping command: it sends a small packet (an ICMP echo request) to a server and times the round trip. It's a rough number, not an exact one, because packets can take different paths each way and meet different conditions on each leg.
To see where the time goes, traceroute lists every hop on the path with its own delay, and MTR combines ping and traceroute to track both the per-hop delay and the total trip.
For websites, open your browser's developer tools, Network tab. Each request is broken down into blocked time, DNS lookup, connection, TLS handshake, sending, waiting and download. The "waiting" part is the server thinking; the rest is the network. You can also throttle the connection to see your site the way a mobile user does. Approximate minimum latencies of the built-in presets:
- Wi-Fi: 2 ms
- DSL: 5 ms
- Regular 4G/LTE: 20 ms
- Good 3G: 40 ms
- Regular 3G: 100 ms
- Regular 2G: 300 ms
- GPRS: 500 ms
Our tip: test on Regular 3G at least once. If your page or your voice demo still feels fine with 100 ms added to every request, it will feel fine for nearly everyone.
For a voice agent, measure end to end: record a test call and time the silence between the end of your sentence and the first syllable of the answer. Do it ten times, with and without a tool call, and keep the slowest result as well as the average. Callers remember the slow turns: one team running a production voice agent for restaurants and salons reported a median of about 800 ms but a 95th percentile (p95) of 2.4 seconds, and the p95 is what made the agent feel broken. Retell's own guidance is to keep the estimated total under 1.5 seconds.
Time one more thing: barge-in, how fast the agent stops talking when the caller interrupts. Another team found their agent answered faster than a competitor's (280 ms against 450 ms at p99) yet users rated it slower, because it took 380 ms to go quiet when interrupted, against 60 ms for the competitor.
Reducing. The fixes follow the causes:
- Shorten the distance. A CDN (content delivery network) stores copies of your content close to users. Edge computing runs the processing close to the source of the data. If you're rethinking your CDN provider, here is how to switch from Cloudflare.
- Make fewer, smaller requests. Fewer files, compressed images, reused connections.
- Fix the local side. A cable instead of Wi-Fi, fewer devices on the line, a restarted router.
- For AI, pick the stage to speed up. A smaller model for simple calls, a streaming voice, a tighter turn detection, and the agent hosted in the callers' region.
When latency matters, and when it doesn't
Low latency is critical for…
- Voice AI agents and phone bots, where a pause reads as a dropped call
- Video calls and live streaming, where delay breaks the back-and-forth
- Online gaming, where a click must land on screen in real time
- High-frequency trading and remote operations, where milliseconds have a cost
- Autonomous driving and industrial sensors, where the system must react at once
Latency matters less for…
- Email, file sync and backups, where a few seconds go unnoticed
- Batch jobs and reports that run overnight
- Downloading large files, where bandwidth counts more than delay
- Pre-recorded transcription, processed after the call ends
- Streaming video with a buffer that absorbs short hiccups
Latency vs bandwidth, throughput and jitter: what changes
These four words describe different things, and a fast connection on one can be slow on another.
| Latency | Bandwidth, throughput, jitter | |
|---|---|---|
| What it measures | The delay for data to arrive | Bandwidth: the maximum data the link can carry at once Throughput: the data that actually gets through Jitter: how much the delay varies |
| Unit | Milliseconds (ms) | Bandwidth and throughput: megabits or gigabits per second Jitter: milliseconds |
| Analogy | How long the drive takes | Bandwidth: how many lanes the road has Throughput: how many cars really pass Jitter: whether the drive takes 20 minutes or 40 |
| What raises it | Distance, hops, processing, queues | Bandwidth: a better plan or cable Throughput: drops with latency and packet loss Jitter: congestion and unstable links |
| Who feels it most | Voice, gaming, live video | Bandwidth: big downloads, 4K video Jitter: calls, where audio stutters |
| How to check it | Ping, browser network timings | Speed tests for bandwidth and throughput Repeated pings for jitter |
Response time is often used as a synonym for latency. In AI, time to first token is the latency before the first words of an answer appear.
The best tools to build low-latency voice AI agents
Voice AI agents are where latency becomes a buying criterion. We compared six tools from the stack: three platforms that run the whole agent, two speech model providers you can plug into any platform, and one open framework. The quick route: you want a phone agent without managing models, start with Retell or Bland; you want to pick each model yourself, Vapi or LiveKit (and if you're already on Vapi, here is how to switch from Vapi); you care about one stage, Deepgram for listening, ElevenLabs for speaking. Each card says who the tool is for, its limit and one tip, with a price order. For the full ranking, see the best AI voice agents; for voices alone, the best AI voice generators.
Retell AI: a phone agent platform that publishes its latency
Easiest startRetell runs the whole voice agent for you: telephony, transcription, the language model and the voice. You build the agent in a visual editor or through the API and connect it to your phone numbers.
Who it's for. Teams that want production phone calls fast, without assembling the stack themselves. Retell advertises about 600 ms of latency for its agents.
The limit. The bill is a sum of lines: voice infrastructure at $0.055 a minute, then the voice and the model you pick. The total runs from $0.07 to $0.31 a minute depending on those choices.
The tip. Most models come in a Standard and a Fast Tier, and the Fast Tier costs up to double: $0.064 against $0.128 a minute for GPT 5.6 Terra, $0.045 against $0.0675 for GPT 4.1. Start on Standard, time ten test calls, and only switch if the pauses are noticeable.
- Price
- Pay as you go, $0.07 to $0.31/min, $10 in free credits
- Best for
- Phone agents without managing models
- Limit
- Per-minute bill made of several lines
Vapi: a developer platform where you pick every model
Most flexibleVapi orchestrates the conversation and lets you choose each stage: the transcriber, the language model and the voice, from different providers. You build in code or in its dashboard.
Who it's for. Developers who want to swap models to trade speed against quality, stage by stage.
The limit. The advertised $0.05 a minute is Vapi's hosting fee only. Models are passed through at cost on top. Its own estimate for 1,000 minutes adds $10 of transcription, $8 to $45 of language model and $15 to $24 of voice to the $50 hosting fee, so $83 to $129 in all.
The tip. Test in the browser first: Daily WebRTC is free, while Twilio adds $0.008 a minute inbound and $0.014 outbound. You measure the agent's own latency without the phone network in the way.
- Price
- $0.05/min hosting plus models at cost, $5 free credits
- Best for
- Developers tuning each stage
- Limit
- Real cost is well above the headline rate
LiveKit: an open framework with a cloud to run it
For buildersLiveKit is an open-source framework for real-time voice and video agents, with a hosted cloud to run them. It handles the audio transport and gives you settings for turn detection and interruptions, the two places where conversational latency is won or lost.
Who it's for. Technical teams that want control over the pipeline and the option to self-host.
The limit. You write code. Its documentation is clear on the trade-off: voice activity detection alone is the option for minimal latency, while the default turn detector model predicts the end of a turn from both the meaning and the sound of speech and handles the widest range of conversations.
The tip. The pricing calculator breaks a minute into agent session, telephony, model, transcription, voice and observability: its default example comes to $0.0479 a minute. Use it to see which line to cut before touching anything else.
- Price
- Free plan at $0/month, paid plans from $50/month
- Best for
- Teams building their own pipeline
- Limit
- Requires development work
Deepgram: speech recognition and a voice agent API
For listeningDeepgram sells speech-to-text models, text-to-speech, and a Voice Agent API that bundles transcription, a managed language model and a voice in one connection.
Who it's for. Teams that need fast streaming transcription inside their own stack, or one API for the whole agent. For transcription tools beyond voice agents, see the best AI transcription software.
The limit. Prices are per minute, per second of audio or per 1,000 characters depending on the product, so budgeting takes a spreadsheet.
The tip. Bringing your own model or voice lowers the Voice Agent rate: Standard is $0.075 a minute, and $0.050 with your own language model and voice. Useful if you already pay for a fast model elsewhere.
- Price
- Pay as you go, $200 free credit; Flux STT $0.0077/min, Voice Agent $0.075/min ($0.050 with your own LLM and voice)
- Best for
- Streaming transcription in a custom stack
- Limit
- Many billing units to combine
ElevenLabs: low-latency voices for any agent
For speakingElevenLabs makes voice models and an agents product. Its real-time models are the ones voice platforms plug in when they want the agent to start speaking quickly.
Who it's for. Anyone whose agent must sound natural and start talking fast, whatever platform runs it.
The limit. The published delays are medians that exclude application and network latency: about 100 ms for Flash v2.5 and v4 Turbo, about 165 ms for v3 Conversational.
The tip. On the API, v4 Turbo costs $0.04 per 1,000 characters, half the $0.08 of v4, and is the faster of the two. For a phone agent, start with Turbo. Full review: ElevenLabs; plan by plan: ElevenLabs pricing.
- Price
- Free tier, Starter $6/month, API v4 Turbo $0.04 per 1,000 characters
- Best for
- Fast, natural voices
- Limit
- Covers the voice stage, not the whole call
Bland: one per-minute rate for phone agents
Simple pricingBland is a phone agent platform that bills talk time at one rate, instead of adding up transcription, model and voice lines.
Who it's for. Teams that want a predictable per-minute cost for calls.
The limit. The Build plan lowers the rate to $0.12 a minute but adds a $299 monthly platform fee, so it pays off only at volume.
The tip. The Start plan has no platform fee and needs no card: you get two credits and an inbound number, enough to time a few test calls before you commit.
- Price
- Start $0.14/min with no platform fee; Build $0.12/min plus $299/month
- Best for
- Predictable call costs
- Limit
- $299 monthly fee on the Build rate
Is your tool missing from this page? Email us: hello@hackceleration.com
Latency FAQ
Is latency the same as response time?
Mostly, yes: response time is often used as another name for latency, the time between a command and the expected response. The nuance is that response time usually includes the server's processing, while network latency is only the travel. In browser tools, that processing shows up as the "waiting" part of each request.
Why does my AI voice agent pause before answering?
Usually because one stage of the chain is slow, most often a long prompt, a tool call or a cautious turn detection setting. Run the call as a budget: time transcription, the model, the voice and the phone leg separately, then fix the biggest line first instead of swapping the voice.
Why track p95 latency instead of the average?
Because the p95 is the delay that 95% of requests stay under, so the slowest 5% sit above it. It matters more than the average for anything interactive: Retell, for instance, reports P50, P90, P95 and P99 for every call, because a median that looks fine can hide the slow turns people notice.
Is high or low latency better?
Low latency is better: it means less waiting between your action and the response. One exception is buffering on purpose, like a streaming player that holds a few seconds of video so a network hiccup doesn't freeze the picture.
What is the difference between latency and ping?
Ping is a tool, latency is what it measures. A ping sends a small ICMP echo packet and times the round trip, so the number it prints is the round-trip time, roughly double the one-way latency. Data can also take a different path each way, so ping is a rough reading, not an exact measurement.
Does more bandwidth reduce latency?
Not directly. Bandwidth is how much data fits through the pipe, latency is how long each piece takes to arrive. A faster plan helps only when the line was saturated: a video call on a congested connection queues packets, and that queueing shows up as latency.
What causes latency in Wi-Fi?
Mostly a weak signal and wireless interference, plus congestion when many devices share the same access point. Browser test presets put Wi-Fi at about 2 ms of minimum latency, so if you see far more than that to your own router, the problem is the local network, not your provider.
How much latency is acceptable on a call?
For calls, the reference is the ITU-T G.114 recommendation: keep one-way delay under 400 ms when planning a network, knowing that interactive tasks such as voice calls and video conferencing can suffer from much lower delays. Earlier versions of the same text noted that below 150 ms one way, most applications are not significantly affected, and that some interactive tasks feel delays under 100 ms. Those are one-way figures: a ping prints the round trip, roughly double, so 150 ms one way reads as about 300 ms in ping.