Skip to main content

Session traces and metrics

Reference for the OpenTelemetry spans and latency stages that LiveKit Agents reports for each session.

Available inPython
|
Node.js

Overview

LiveKit Agents instruments each session with OpenTelemetry spans. The same spans appear in the Agent insights timeline and in any external tracing backend you export to. The names on this page are the names you query in either place.

The spans also record latency metrics for each stage of a turn. The latency charts use these stages, on the Metrics tab of a session and on the Agents  page for the project and for each agent.

Span reference

The main spans in the trace timeline are the following. A session can emit others, such as startup and shutdown spans, that aren't listed here:

SpanDescription
job_entrypointThe root span for the job, from the availability request to the end of shutdown. The return of your entrypoint function is an event on this span.
agent_sessionOne AgentSession, a child of job_entrypoint.
start_agent_activitySetup for an agent: model warm-up and toolset registration.
on_enterThe active agent's on_enter hook.
user_turnOne user turn, from the start of their speech to the point the agent accepts the turn as finished.
eou_waitThe wait for the user's turn to end, from their last speech to the turn decision. Carries the outcome as lk.eou.outcome.
eou_detectionOne end-of-turn prediction, inside eou_wait. Includes the turn-detector probability, the endpointing delay, and the conversation context it judged.
user_speakingOne continuous run of user speech, as detected by voice activity detection.
agent_turnOne agent reply, from the start of generation to the end of the reply. With preemptive generation, the span can start before the user's turn ends. A reply that calls tools stays one span, and each language model step is a generation event on it.
llm_nodeThe language model step of an agent turn, including tool selection.
llm_requestOne request to the language model. Each attempt, including the first, appears as a child llm_request_run span.
function_toolOne tool call, with its arguments, output, and error state.
realtime_inferenceOne generation from a realtime model, in place of the llm_node and tts_node steps.
realtime_metricsThe provider metrics for a realtime generation. Appears as its own span when they arrive after the turn ends.
tts_nodeThe speech synthesis step of an agent turn.
tts_requestOne request to the speech synthesis provider. Each attempt, including the first, appears as a child tts_request_run span.
agent_speakingOne continuous run of agent speech. A turn with no agent_speaking span never reached the user.
amdAnswering machine detection, on outbound telephony sessions that enable it. In Node.js, the span is named answering_machine_detection.
on_exitThe outgoing agent's on_exit hook, during a handoff or at session end.
drain_agent_activityShutdown for the outgoing agent, during a handoff or at session end. Runs on_exit, then waits for in-flight speech and tool calls to finish.
Speech-to-text runs continuously

Speech-to-text runs continuously rather than once per request, so it emits no span of its own. Transcription timing is reported on user_turn as lk.transcription_delay, and the audio it processed appears in the Speech usage tile for the session.

For the attributes on these spans, see Span attributes.

Latency stages

The session charts and the agent dashboards share most latency stages except where noted. A chart also shows a stage only when the session produced it, so pipeline turns and realtime turns contribute different stages:

StageWhereDescription
E2EBothEnd-to-end response time, from when the user stops speaking to when the reply begins playing. This is the total, not a stage.
STT delayBothTime to transcribe the user's speech into text after they stop speaking.
EOTBothEnd-of-turn detection: the time spent confirming the user has finished, beyond transcription.
Turn detectionSession chartsThe whole end-of-turn wait on a realtime turn, where nothing was transcribed. Replaces STT delay and EOT.
LLM TTFTBothTime for the language model to produce the first token of its reply.
TTS TTFBBothTime for speech synthesis to produce the first audio byte of the reply.
Tool callsBothHow long the agent's function calls took to return.
Realtime TTFTBothTime for a realtime model to produce the first audio byte of its reply. Replaces LLM TTFT and TTS TTFB in realtime sessions. The session charts label it Model TTFT.

Additional resources