Turn-taking tuning

Overview

Turn-taking in voice AI involves several stages of the agent pipeline:

• User activity detection decides when the user has finished a turn so the agent can reply. Options include turn detection mode, endpointing delays, and endpointing mode.
• Interruption handling decides when the user can cut the agent off mid-response. Options include enable/disable, detection mode, interruption thresholds, and false-interruption recovery.
• Preemptive generation lets the LLM (and optionally TTS) start work before the user's turn is fully confirmed. Options include enable/disable, preemptive TTS, max speech duration, and max retries.
• Audio pre-processing (noise cancellation, automatic gain control) cleans the input before any of these stages run. Options include voice isolation and background noise suppression.
• Agent speech scheduling controls the cadence of the agent's own utterances. Options include the minimum gap between agent utterances (Python only).

The defaults are reasonable for most apps, but tuning matters when you're chasing low latency, working in noisy environments, or seeing specific symptoms like the agent cutting users off. This page gives a recommended starting config, a full reference of the options that affect each stage, and a troubleshooting table mapping common symptoms to the options that fix them.

For a deeper reference on each parameter, see TurnHandlingOptions.

Configuration

The next two sections cover a recommended starting config and a full options reference.

Recommended starting config

A starting point for a voice agent that needs to respond quickly in environments with background noise or other speakers. See All options for what each parameter does.

from livekit.agents import AgentSession, TurnHandlingOptions, room_io
from livekit.plugins import ai_coustics, silero
from livekit.plugins.turn_detector.multilingual import MultilingualModel

session = AgentSession(
    turn_handling=TurnHandlingOptions(
        turn_detection=MultilingualModel(),
        endpointing={
            "mode": "fixed",
            "min_delay": 0.5,
            "max_delay": 3.0,
        },
        interruption={
            "mode": "adaptive",
            "min_duration": 0.5,
            "min_words": 0,
        },
        # preemptive_generation is enabled by default. Opt into preemptive TTS
        # for lower latency at the cost of wasted compute on cancellations.
        preemptive_generation={
            "preemptive_tts": False,
        },
    ),
    vad=silero.VAD.load(),
    # ... stt, tts, llm, etc.
)

await session.start(
    # ...,
    room_options=room_io.RoomOptions(
        audio_input=room_io.AudioInputOptions(
            noise_cancellation=ai_coustics.audio_enhancement(
                model=ai_coustics.EnhancerModel.QUAIL_VF_L,
            ),
        ),
    ),
)

import { voice } from '@livekit/agents';
import * as livekit from '@livekit/agents-plugin-livekit';
import * as silero from '@livekit/agents-plugin-silero';
import * as aiCoustics from '@livekit/plugins-ai-coustics';

const session = new voice.AgentSession({
  vad: await silero.VAD.load(),
  turnHandling: {
    turnDetection: new livekit.turnDetector.MultilingualModel(),
    endpointing: {
      minDelay: 500,
      maxDelay: 3000,
    },
    interruption: {
      mode: 'adaptive',
      minDuration: 500,
      minWords: 0,
    },
    // preemptiveGeneration is enabled by default. Opt into preemptive TTS
    // for lower latency at the cost of wasted compute on cancellations.
    preemptiveGeneration: {
      preemptiveTts: false,
    },
  },
  // ... stt, tts, llm, etc.
});

await session.start({
  // ...,
  inputOptions: {
    noiseCancellation: aiCoustics.audioEnhancement({ model: 'quailVfL' }),
  },
});

For quieter environments, drop the noise cancellation argument from session.start(). The rest of the config still applies.

For SIP participants, swap voice isolation for the telephony-tuned Krisp model: noise_cancellation.BVCTelephony() (Python) or TelephonyBackgroundVoiceCancellation() (Node.js). For multi-speaker rooms, use background noise suppression instead of voice isolation.

All options

The following table lists the options that affect turn-taking, grouped by pipeline stage.

Troubleshooting

The following table maps common turn-taking complaints to the options that affect them.

If you're tuning by feel, use agent observability to confirm changes actually move the metrics you care about. Preemptive generation in particular doesn't always reduce latency, and the metrics tell you whether your changes are pulling their weight.

Additional resources