Hedra virtual avatar integration guide

Overview

Hedra provides lifelike generative video with their Character-3 model. You can use the open source Hedra integration for LiveKit Agents to add virtual avatars to your voice AI app.

Quick reference

This section includes a basic usage example and some reference material. For links to more detailed documentation, see Additional resources.

Installation

Install the plugin from PyPI:

pip install "livekit-agents[hedra]~=1.0"

Authentication

The Hedra plugin requires a Hedra API key.

Set HEDRA_API_KEY in your .env file.

Usage

Use the plugin in an AgentSession. For example, you can use this avatar in the Voice AI quickstart.

from livekit import agents
from livekit.agents import AgentSession, RoomOutputOptions
from livekit.plugins import hedra

async def entrypoint(ctx: agents.JobContext):
   await ctx.connect()

   session = AgentSession(
      # ... stt, llm, tts, etc.
   )

   avatar = hedra.AvatarSession(
      avatar_id="...",  # ID of the Hedra avatar to use. See "Avatar setup" for details.
   )

   # Start the avatar and wait for it to join
   await avatar.start(session, room=ctx.room)

   # Start your agent session with the user
   await session.start(
      room=ctx.room,
      room_output_options=RoomOutputOptions(
         # Disable audio output to the room. The avatar plugin publishes audio separately.
         audio_enabled=False,
      ),
      # ... agent, room_input_options, etc....
   )

Preview the avatar in the Agents Playground or a frontend starter app that you build.

Avatar setup

The Hedra plugin requires a source image asset from which to generate the avatar. Avatars render as 512x512px square videos. Hedra automatically centers and crops around the face within the provided image. Hedra supports humanoid faces, in a range of styles from photorealistic to animated.

You can specify the avatar image by ID or by passing an image directly.

Pass avatar ID

To use an existing avatar, pass the avatar_id parameter to the plugin. You can find the ID in the Hedra web studio or upload it using the Hedra API.

Web studio

Generate or upload an image in the Hedra web studio. To find the ID to pass as avatar_id, download the image from the library. The avatar ID is the filename of the downloaded image, minus the extension.

API upload

To upload an image with the Hedra API, first create a new asset:

curl -X POST \
  -H "X-API-Key: <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"type":"image","name":"<your-avatar-name>"}' \
  https://api.hedra.com/web-app/public/assets

The response includes an asset id, which you need for the next step.

Then upload the image:

curl -X POST \
  -H "X-API-Key: <your-api-key>" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@<your-local-image-path>" \
  https://api.hedra.com/web-app/public/assets/<your-asset-id>/upload

You can now use the asset ID in the Hedra plugin as the avatar_id.

Pass image directly

To upload a new image directly in the plugin, pass a PIL Image object in the avatar_image parameter.

from PIL import Image

avatar_image = Image.open("/path/to/image.jpg")

avatar = hedra.AvatarSession(
   avatar_image=avatar_image,
)

The plugin uploads the image to Hedra and uses it for the avatar session. The image can come from anywhere, including your local filesystem, a remote URL, uploaded in realtime from your frontend, or generated by an external API or AI model.

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

Additional resources

The following resources provide more information about using Hedra with LiveKit Agents.

Python package

The livekit-plugins-hedra package on PyPI.

Plugin reference

Reference for the Hedra avatar plugin.

GitHub repo

View the source or contribute to the LiveKit Hedra avatar plugin.

Hedra API docs

Hedra's API docs.

Agents Playground

A virtual workbench to test your avatar agent.

Frontend starter apps

Ready-to-use frontend apps with avatar support.