Subagent delegation | LiveKit Documentation

Overview

Conversational latency and reasoning depth impose conflicting requirements. A low-latency model can respond within a few hundred milliseconds but performs limited reasoning. A frontier reasoning model produces higher-quality analysis but takes seconds to respond. In a voice conversation, a gap of several seconds can be indistinguishable from a dropped connection.

Subagent delegation separates these requirements across two models:

The primary agent runs a latency-optimized model and handles every conversational turn without blocking.
The subagent runs a slower, higher-capability model and performs the reasoning in the background.

The primary agent delegates through an async tool and acknowledges the request immediately, so the conversation continues. When the subagent completes, the result enters the conversation as a follow-up reply. The user hears continuous speech rather than silence.

Loading diagram…

This pattern is also referred to as the talker-reasoner pattern, or as a fast brain and slow brain architecture.

When to use subagent delegation

Use this pattern when a single model can't satisfy both reasoning quality and conversational latency. It applies to open-ended analysis, multi-step research, and planning: work that justifies a delay in the answer but not a pause in the conversation.

Each of the following alternatives is simpler, so evaluate them first:

A single agent with tools is sufficient when one model satisfies both latency and quality requirements.
An async tool on its own handles background work that doesn't require a second model, such as a slow API call or a long-running database query. Subagent delegation is the case where the background work is itself an LLM call.
The supervisor pattern is appropriate when the conversation should wait for a structured result. A specialist task takes temporary control and returns a typed value, which suits data collection and verification rather than open-ended reasoning.
Agent handoffs are appropriate when one agent completes its role and another takes over with different instructions and tools.

The distinction from the supervisor pattern is whether the user waits. With the supervisor pattern, the conversation blocks until a typed result returns. With subagent delegation, the conversation continues and the result arrives in a later turn.

Pattern anatomy

The pattern has three parts:

The primary agent. An Agent whose session uses a fast model. Its instructions define when to delegate and what to do while the subagent runs.
The subagent. A separate LLM instance with its own chat context, called through the standalone LLM interface. Keeping the contexts separate prevents reasoning history from inflating the token count on every conversational turn.
The delegation surface. An async function tool. Calling ctx.update() within the tool marks it non-blocking, which allows the conversation to continue while the subagent runs.

Delegating to a reasoning model

The following agent answers routine questions directly and delegates analytical ones to a reasoning model. The analyze tool calls ctx.update() before the slow work begins, which releases the conversation, and returns the result from the subagent on completion.

from livekit.agents import Agent, AgentSession, ChatContext, RunContext, function_tool, inference
from livekit.agents.llm import ToolFlag

# The subagent. A separate, more capable model with its own context, so deep
# reasoning never inflates the tokens on a conversational turn.
reasoner = inference.LLM(model="google/gemini-3.1-pro")


class ResearchAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions=(
                "You are a research assistant. Answer simple questions yourself, "
                "briefly and conversationally. For questions that need real "
                "analysis, call analyze. While it runs, say in one short sentence "
                "that you're looking into it, then keep helping with whatever else "
                "the user asks. Never answer the delegated question yourself while "
                "the analysis is running. When the analysis arrives, always deliver "
                "it and name the question it answers."
            ),
        )

    # CANCELLABLE lets the LLM stop the analysis when the user moves on, so
    # abandoned reasoning stops consuming tokens. Rejecting a repeat of the
    # same question keeps the agent from paying for the same reasoning twice.
    @function_tool(
        flags=ToolFlag.CANCELLABLE,
        on_duplicate="reject",
        duplicate_scope="name_and_args",
    )
    async def analyze(self, ctx: RunContext, question: str) -> str:
        """Analyze a question that needs careful, multi-step reasoning.

        Args:
            question: The question to analyze.
        """
        # This first update is what makes the tool non-blocking. The primary
        # agent voices an acknowledgement and the conversation continues.
        await ctx.update(f"Started analyzing: {question}")

        # The subagent gets a focused context, not the conversation history.
        reasoner_ctx = ChatContext()
        reasoner_ctx.add_message(
            role="system",
            content=(
                "Analyze the question thoroughly. Answer in at most three "
                "sentences, in plain language a voice agent can read aloud."
            ),
        )
        reasoner_ctx.add_message(role="user", content=question)

        # Filler speech covers the silent intervals while the subagent runs.
        async with ctx.with_filler(
            "Still working through it.", delay=5, interval=10, max_steps=3
        ):
            response = await reasoner.chat(chat_ctx=reasoner_ctx).collect()

        # The return value is voiced as a follow-up reply once the agent is idle.
        return response.text


session = AgentSession(
    # ... stt, tts, etc.
    # The primary agent. A latency-optimized model keeps every turn fast.
    llm=inference.LLM(model="google/gemma-4-31b-it"),
)

await session.start(agent=ResearchAssistant(), room=ctx.room)

import { inference, llm, voice } from '@livekit/agents';
import { z } from 'zod';

// The subagent. A separate, more capable model with its own context, so deep
// reasoning never inflates the tokens on a conversational turn.
const reasoner = new inference.LLM({ model: 'google/gemini-3.1-pro' });

const analyze = llm.tool({
  name: 'analyze',
  description: 'Analyze a question that needs careful, multi-step reasoning.',
  // CANCELLABLE lets the LLM stop the analysis when the user moves on, so
  // abandoned reasoning stops consuming tokens. Rejecting a second analysis
  // keeps the agent from paying for the same reasoning twice.
  flags: llm.ToolFlag.CANCELLABLE,
  onDuplicate: 'reject',
  parameters: z.object({
    question: z.string().describe('The question to analyze.'),
  }),
  execute: async ({ question }, { ctx, abortSignal }) => {
    // This first update is what makes the tool non-blocking. The primary
    // agent voices an acknowledgement and the conversation continues.
    await ctx.update(`Started analyzing: ${question}`);

    // The subagent gets a focused context, not the conversation history.
    const reasonerCtx = new llm.ChatContext();
    reasonerCtx.addMessage({
      role: 'system',
      content:
        'Analyze the question thoroughly. Answer in at most three ' +
        'sentences, in plain language a voice agent can read aloud.',
    });
    reasonerCtx.addMessage({ role: 'user', content: question });

    // chat() takes no abort signal, so hold the stream and close it when the
    // tool is cancelled. Without this the filler stops but the subagent keeps
    // running, and the conversation pays for reasoning nobody hears.
    const stream = reasoner.chat({ chatCtx: reasonerCtx });
    abortSignal?.addEventListener('abort', () => stream.close(), { once: true });

    // Filler speech covers the silent intervals while the subagent runs.
    const response = await ctx.filler(
      'Still working through it.',
      { delay: 5000, interval: 10000, maxSteps: 3, signal: abortSignal },
      () => stream.collect(),
    );

    // The return value is voiced as a follow-up reply once the agent is idle.
    return response.text;
  },
});

const researchAssistant = voice.Agent.create({
  instructions:
    'You are a research assistant. Answer simple questions yourself, ' +
    'briefly and conversationally. For questions that need real analysis, ' +
    "call analyze. While it runs, say in one short sentence that you're " +
    'looking into it, then keep helping with whatever else the user asks. ' +
    'Never answer the delegated question yourself while the analysis is ' +
    'running. When the analysis arrives, always deliver it and name the ' +
    'question it answers.',
  tools: [analyze],
});

const session = new voice.AgentSession({
  // ... stt, tts, etc.
  // The primary agent. A latency-optimized model keeps every turn fast.
  llm: new inference.LLM({ model: 'google/gemma-4-31b-it' }),
});

await session.start({ agent: researchAssistant, room: ctx.room });

Maintaining coherence

The primary agent responds using only the context available when it speaks, which precedes the result from the subagent. Instructions must account for this, otherwise the model produces a speculative answer.

Three controls govern this behavior:

Prohibit the agent from answering the delegated question. While the subagent runs, the primary agent acknowledges the request and continues the conversation. Without this constraint, the model might generate a low-quality answer that the subagent result could contradict.
Constrain how the agent presents results. The framework wraps each ctx.update() in a short instruction template before passing it to the LLM. Override update_template to control how the agent reports partial results. For details, see Prompt templates.
Require the agent to deliver pending results. If newer messages arrive before the subagent finishes, the framework uses reply_maybe_covered_template to generate the follow-up reply. This template allows the model to return an empty response if it believes the result was already covered. After several unrelated turns, a latency-optimized model might do this, causing the result to never be spoken. Agent instructions don't override the template because it provides the instructions for that reply. Override the template to require the agent to deliver pending results. For details, see Prompt templates.

Override the template on the session:

session = AgentSession(
    # ... stt, tts, etc.
    llm=inference.LLM(model="google/gemma-4-31b-it"),
    tool_handling={
        "async_options": {
            "reply_maybe_covered_template": (
                "New results arrived from background tool calls "
                "(call_ids: {call_ids}).\n"
                "Deliver these results to the user now, naming the question "
                "they answer.\n"
                "Never reply with an empty response, and never claim you "
                "already covered them.\n"
                "Do not repeat, word for word, details you have already stated."
            ),
        },
    },
)

const session = new voice.AgentSession({
  // ... stt, tts, etc.
  llm: new inference.LLM({ model: 'google/gemma-4-31b-it' }),
  toolHandling: {
    asyncOptions: {
      replyMaybeCoveredTemplate:
        'New results arrived from background tool calls (call_ids: {callIds}).\n' +
        'Deliver these results to the user now, naming the question they answer.\n' +
        'Never reply with an empty response, and never claim you already covered them.\n' +
        'Do not repeat, word for word, details you have already stated.',
    },
  },
});

Blocking for critical results

Certain questions require a verified answer rather than a fast one. Pricing, medical guidance, and any result the user acts on justify a pause in the conversation.

To make a delegated call blocking, omit ctx.update(). The signature is otherwise identical to the non-blocking tool. A tool that never calls ctx.update() behaves as a regular synchronous tool, and the conversation waits for the return value:

@function_tool()
async def check_price(ctx: RunContext, item: str) -> str:
    """Look up an exact price. Blocks until the answer is confirmed.

    Args:
        item: The item to price.
    """
    # No ctx.update() call, so this tool blocks the conversation.
    # confirm_price is your own pricing lookup.
    return await confirm_price(item)

const checkPrice = llm.tool({
  name: 'checkPrice',
  description: 'Look up an exact price. Blocks until the answer is confirmed.',
  parameters: z.object({
    item: z.string().describe('The item to price.'),
  }),
  // No ctx.update() call, so this tool blocks the conversation.
  // confirmPrice is your own pricing lookup.
  execute: async ({ item }) => confirmPrice(item),
});

The conversation is silent for the duration of a blocking call, so keep the work short. For anything longer, prefer the non-blocking path so the conversation continues.

Filler speech on a blocking tool

Filler speech is reliable on the non-blocking path, where the framework prompts the model to fold each update into the conversation. On a blocking tool, the filler line becomes the last assistant turn before the tool returns, and a latency-optimized model can treat its own filler as a user turn and acknowledge it instead of stating the result. Test the behavior with your primary model before adding filler to a blocking tool.

A single agent commonly uses both approaches: exploratory questions delegate to the background, and questions requiring a verified answer block.

Best practices

The following guidelines apply when implementing this pattern:

Give the subagent its own context. Passing the full conversation history spends tokens on context the reasoning task rarely requires. Pass the question alone, or a truncated copy when the subagent requires recent turns.
Constrain the subagent to voice-ready output. A reasoning model returns structured prose by default. Specify length and format in the system message so the agent can read the result aloud without further processing.
Make abandoned work cancellable. When the user changes topic, reasoning on the previous topic continues to consume tokens for a result the conversation no longer requires. Opt into cancellation with the CANCELLABLE flag so the LLM can stop it.
Reject duplicate delegations. A repeated question can trigger a second reasoning call while the first is still running, which pays twice for one answer. The preceding example sets on_duplicate to reject. In Python, duplicate_scope="name_and_args" narrows the match to the same question, so a genuinely different analysis still runs.
Preserve results across handoffs. An agent handoff drops pending updates. Wrap the tool in an AsyncToolset if the result should reach whichever agent is active on completion.
Test both paths. The delegated path and the immediate path produce different conversations. Cover each with the test framework, then validate the complete exchange with a simulation.

Test delegation with simulations

The behavior in this guide depends on instruction-following, which varies between models and between runs. A single manual test can pass while the same agent drops a completed result or refuses an unrelated request on the next call. Simulations run full conversations against your agent and judge each transcript, which catches these cases repeatably. Cover the delegated path, an interruption partway through, and the blocking path.

Additional resources

The following resources provide more information on the topics discussed in this guide.

Async tools

Handle long-running tools so agents can keep talking.

Supervisor pattern

Route work to specialist tasks while one agent stays in control.

Chat context

Manage the conversation history sent to each model.

LLM overview

Choose models and call them outside the voice pipeline.

Workflows

Compare the constructs for structuring a voice AI app.

Testing & evaluation

Test the delegated and immediate paths independently.