Tool definition and use

Overview

LiveKit Agents has full support for LLM tool use. This feature allows you to create a custom library of tools to extend your agent's context, create interactive experiences, and overcome LLM limitations.

Within a tool, you can:

• Generate agent speech with session.say() or session.generate_reply().
• Call methods on the frontend using RPC.
• Handoff control to another agent as part of a workflow.
• Store and retrieve session data from the context.
• Anything else that a Python function can do.
• Call external APIs or lookup data for RAG.

Tool types

Two types of tools are supported:

• Function tools: Tools that are defined as functions within your agent's code base and can be called by the LLM.
• Provider tools: Tools provided by a specific model provider (e.g. OpenAI, Gemini, etc.) and are executed internally by the provider's model server.

Provider tools

Provider tools are implemented and executed internally by a specific model provider (e.g., OpenAI, Gemini). They function similarly to function tools within the framework, but are vendor-specific and only compatible with the corresponding provider's models.

For example, you can use xAI's XSearch tool to query X for real-time information.

from livekit.plugins import xai

agent = MyAgent(
  llm=xai.realtime.RealtimeModel(),
  tools=[xai.realtime.XSearch()],
)

Currently we support provider tools for the following providers:

• Gemini
• xAI Grok Voice Agent API

Examples

The following additional examples show how to use tools in different ways:

In this section

Read more about each topic.

Additional resources

The following articles provide more information about the topics discussed in this guide: