Synthesia virtual avatar integration guide

Overview

Synthesia  provides interactive avatars that lip-sync your agent's speech in realtime. You can use the open source Synthesia integration for LiveKit Agents to add virtual avatars to your voice AI app.

The plugin loads up to five avatars for each session. You can change the avatar during a conversation without a new connection. See Swapping avatars for details.

Quick reference

This section includes a basic usage example and some reference material. For links to more detailed documentation, see Additional resources.

Installation

Install the plugin from PyPI:

uv add "livekit-agents[synthesia]~=1.8"

Authentication

The Synthesia plugin requires a Synthesia API key . Your plan must include interactive avatars.

Set SYNTHESIA_API_KEY in your .env file.

Avatar setup

The plugin requires at least one avatar ID from your workspace avatar gallery. Browse the gallery in Synthesia Studio  and copy the ID of an avatar that your workspace can access.

Pass one to five IDs to AvatarConfig. The session renders the first ID. The worker precomputes the other IDs so that you can swap to them later. An empty list, more than five IDs, or a single string raises a ValueError. An ID that your workspace can't access raises a SynthesiaError with type=ErrorType.UNKNOWN_AVATAR.

Usage

Use the Synthesia plugin in an AgentSession. For example, you can use this avatar in the Voice AI quickstart.

from livekit import agents
from livekit.agents import AgentServer, AgentSession
from livekit.plugins import synthesia

server = AgentServer()

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: agents.JobContext):
    session = AgentSession(
        # ... stt, llm, tts, etc.
    )

    avatar = synthesia.AvatarSession(
        synthesia.AvatarConfig(
            # IDs of the Synthesia avatars to use. See "Avatar setup" for details.
            avatar_ids=[
                "03cee7ec-ac90-45ec-8c20-74a399cf3dc4",
                "2d1e1b5f-8ce0-4c3f-b0d3-9c4a1a6e5f77",
            ],
        ),
    )

    # Start the avatar and wait for it to join
    await avatar.start(session, room=ctx.room)

    # Start your agent session with the user
    await session.start(
        # ... room, agent, room_options, etc.
    )

Preview the avatar in the Agent Console or a frontend starter app that you build.

Swapping avatars

Call swap_avatar on a started session to change the avatar without an interruption to the conversation. The ID must be one of the IDs that you passed to AvatarConfig, because the worker precomputes only those IDs. Pass "default" to return to the first ID in the list. The method returns the active avatar ID. To change how long the method waits for the swap, set the timeout parameter. The default is 15 seconds.

await avatar.swap_avatar("2d1e1b5f-8ce0-4c3f-b0d3-9c4a1a6e5f77")
await avatar.swap_avatar("default")  # back to the first ID

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

Error handling

Errors from the Synthesia API and the avatar session raise a SynthesiaError, a subclass of APIError. Use the type attribute to tell an invalid API key from an exhausted quota or a plan without interactive avatars. Some errors have no type, including a missing API key and a swap that the worker rejects. Use the retryable attribute to decide if you can try the same call again.

try:
    await avatar.start(session, room=ctx.room)
except synthesia.SynthesiaError as e:
    if e.type in (
        synthesia.ErrorType.QUOTA_EXCEEDED,
        synthesia.ErrorType.FEATURE_NOT_IN_PLAN,
    ):
        ...  # fall back to an audio-only session
    raise

Additional resources

The following resources provide more information about using Synthesia with LiveKit Agents.

Python package

The livekit-plugins-synthesia package on PyPI.

Plugin reference

Reference for the Synthesia avatar plugin.

GitHub repo

View the source or contribute to the LiveKit Synthesia avatar plugin.

Synthesia docs

Synthesia's full documentation site.

Agent Console

A virtual workbench to test your avatar agent.

Frontend starter apps

Ready-to-use frontend apps with avatar support.