Overview
LiveKit agents can run locally, on your own infrastructure, or on LiveKit Cloud. To compare LiveKit with other voice agent platforms, deploy a LiveKit starter project to LiveKit Cloud and run the benchmark over WebRTC.
Configure your agent
Follow the Voice AI quickstart to create and deploy a starter project. Update the agent's instructions and tools for your scenarios. Keep the starter project's audio pipeline, including LiveKit Inference, turn detection, and noise cancellation.
Choose a deployment region near your test caller. Before a large run, review the quotas and limits for your plan.
Measure response latency
Measure response latency from the end of the caller's speech to the start of the reply in the audio the caller receives. Measure greeting latency separately, from the time the session connects to the first audio the caller hears.
To measure a LiveKit agent, run an audio simulation against your deployed agent. The dashboard and JSON export include caller-side latency. To see where the agent spent time during a turn, open the session in Agent insights.
Phone benchmarks
If your users dial in, run a separate benchmark over the phone. Phone calls include latency from the carrier and SIP path. Report phone and WebRTC results separately.
To receive test calls on LiveKit Cloud, assign a LiveKit phone number to a dispatch rule for your agent. Place test calls through the same outbound SIP trunk for each platform.
Report results
Use the same scenarios, transport, and caller-side measurement on each platform. Run each scenario more than once. Report the p50, p95, and p99 response latencies and the number of turns measured for each scenario.
Use the outcome to measure task success. If a scenario asks the agent to book a room, look for the reservation in the booking system after the call. To score on outcomes, see Grade on the final state.
If you publish results, include the agent code and configuration for each platform. Record the transport, caller location, and agent region.