Module livekit.plugins.atmee

Atmee avatar plugin for LiveKit Agents.

Bring your own LiveKit voice agent; Atmee renders its avatar into your room. Any Atmee avatar renders by its id, v1 or v2 (create avatars in the Atmee studio or with the Atmee API)::

from livekit.plugins import atmee

avatar = atmee.AvatarSession(avatar_id="...")       # ATMEE_API_KEY in the environment
await avatar.start(session, room=ctx.room)
await session.start(agent=..., room=ctx.room)       # the agent's TTS drives the avatar's video

See https://docs.livekit.io/agents/models/avatar/plugins/atmee/ for more information.

Classes

class AtmeeAPI (api_key: NotGivenOr[str] = NOT_GIVEN,
api_url: NotGivenOr[str] = NOT_GIVEN,
*,
conn_options: APIConnectOptions = APIConnectOptions(max_retry=3, retry_interval=2.0, timeout=10.0, liveness_timeout=6.0),
session: aiohttp.ClientSession | None = None)
Expand source code
class AtmeeAPI:
    """Async client for the Atmee API.

    Args:
        api_key: your ``sk_atmee_...`` key; defaults to ``ATMEE_API_KEY``.
        api_url: API base; defaults to ``ATMEE_API_URL`` or ``https://api.atmanity.us``.
        conn_options: retry/timeout policy. ``timeout`` bounds the connect of
            every request and each attempt of the calls that are safe to
            repeat (reading and ending a session, reading an avatar), which
            are retried ``max_retry`` times on transport failures and 5xx
            answers. Starting a session is never repeated and has a budget
            of its own.
        session: an ``aiohttp.ClientSession`` to reuse. Inside a LiveKit job the
            worker's shared session is used; outside (scripts, tests) a private
            one is created and closed by :meth:`aclose`.
    """

    def __init__(
        self,
        api_key: NotGivenOr[str] = NOT_GIVEN,
        api_url: NotGivenOr[str] = NOT_GIVEN,
        *,
        conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS,
        session: aiohttp.ClientSession | None = None,
    ) -> None:
        key = api_key or os.getenv("ATMEE_API_KEY")
        if not key:
            raise AtmeeException(
                "ATMEE_API_KEY must be set (or pass api_key=...); create one at "
                "https://www.atmee.ai/studio/api-keys"
            )
        self._api_key = key
        self._api_url = (api_url or os.getenv("ATMEE_API_URL") or DEFAULT_API_URL).rstrip("/")
        _check_api_url(self._api_url)
        self._conn_options = conn_options
        self._session = session
        self._owns_session = False

    # --- avatar sessions -------------------------------------------------

    async def create_avatar_session(
        self,
        avatar_id: str,
        *,
        livekit_url: str,
        livekit_token: str,
        agent_identity: str | None = None,
        max_duration_seconds: int | None = None,
        metadata: dict[str, Any] | None = None,
        wait_for: WaitFor = "initializing",
        avatar_version: NotGivenOr[str] = NOT_GIVEN,
        stop: asyncio.Event | None = None,
    ) -> AvatarSessionInfo:
        """Render ``avatar_id`` into the room ``livekit_token`` grants.

        ``livekit_token`` is a LiveKit access token you minted with YOUR
        project's secret for the avatar participant: ``kind=agent``, a
        ``roomJoin`` grant for your room, and the ``lk.publish_on_behalf``
        attribute naming your agent (:class:`AvatarSession` does this for
        you). Returns as soon as the rendering worker acknowledged the start,
        or once the avatar joined with ``wait_for="avatar_joined"``.

        Any avatar renders as what it is, v1 or v2; the request names no
        generation. The call first reads the avatar's generation to know how
        long the start may take: a v1 start may take up to 1 minute (3 with
        ``wait_for="avatar_joined"``). The first v2 session after a quiet
        period can take minutes, so a v2 start may take up to 5 (7.5), and
        while the API answers ``avatar_warming`` or ``avatar_start_interrupted``
        (nothing billed) it asks again after the ``Retry-After`` given.

        Once ``stop`` is set, the call asks no more: it raises an
        ``AtmeeException`` with ``code="closed"`` instead of asking again. A
        request already sent is still awaited, since it may open a session
        that must then be ended.
        """
        _check_wait_for(wait_for)
        _warn_avatar_version(avatar_version)
        payload: dict[str, Any] = {"livekitUrl": livekit_url, "livekitToken": livekit_token}
        if agent_identity:
            payload["agentIdentity"] = agent_identity
        if max_duration_seconds is not None:
            payload["maxDurationSeconds"] = int(max_duration_seconds)
        if metadata:
            payload["metadata"] = metadata
        params: dict[str, str] | None = (
            {"waitFor": wait_for} if wait_for != "initializing" else None
        )
        # Never retried on a timeout or a 5xx: the POST reserves and bills a
        # session as soon as the server accepts it, so asking again could open
        # a second render for the same room. Only the answers that say nothing
        # was reserved (`avatar_warming`, `avatar_start_interrupted`) are asked
        # again, within the start budget.
        loop = asyncio.get_running_loop()
        budget = await self._start_budget(avatar_id, wait_for)
        deadline = loop.time() + budget
        while True:
            if stop is not None and stop.is_set():
                raise AtmeeException("the avatar session start was stopped", code="closed")
            try:
                data = await self._request(
                    "POST",
                    f"/v1/avatars/{avatar_id}/avatar_sessions",
                    json=payload,
                    params=params,
                    retry=False,
                    total_timeout=max(1.0, deadline - loop.time()),
                )
                break
            except AtmeeAvatarWarmingError as e:
                pause = e.retry_after if e.retry_after is not None else _WARMING_RETRY_DEFAULT
                pause = min(max(pause, _WARMING_RETRY_BOUNDS[0]), _WARMING_RETRY_BOUNDS[1])
                if loop.time() + pause >= deadline:
                    raise
                logger.info(
                    "atmee avatar render not started yet; asking again",
                    extra={"code": e.code, "retry_in": pause},
                )
                # nothing is reserved while waiting: a stop ends the wait at once
                if stop is None:
                    await asyncio.sleep(pause)
                else:
                    try:
                        await asyncio.wait_for(stop.wait(), timeout=pause)
                    except asyncio.TimeoutError:
                        pass
        session_id = data.get("sessionId")
        if not isinstance(session_id, str) or not session_id:
            raise AtmeeException(
                f"the Atmee API accepted the avatar session but returned no sessionId: {data!r}",
                code="invalid_response",
            )
        return AvatarSessionInfo(
            session_id=session_id,
            status=str(data.get("status", "")),
            avatar_participant_identity=str(data.get("avatarParticipantIdentity", "")),
            agent_identity=str(data.get("agentIdentity", "")),
            room_name=str(data.get("roomName", "")),
            max_duration_seconds=int(data.get("maxDurationSeconds") or 0),
            raw=data,
        )

    async def _start_budget(self, avatar_id: str, wait_for: WaitFor) -> float:
        """How long a start of ``avatar_id`` may take, from its generation."""
        try:
            info = await self._get_avatar(
                avatar_id, deadline=asyncio.get_running_loop().time() + self._conn_options.timeout
            )
        except AtmeeException as e:
            # A key without read access to the avatar, or a slow read: allow
            # for the slower generation; the start reports real failures.
            logger.debug(
                "could not read the atmee avatar's generation; allowing for a v2 start",
                extra={"avatar_id": avatar_id, "error": str(e)},
            )
            return _START_TOTAL_TIMEOUT["v2"][wait_for]
        generation = "v2" if info.engine in _V2_ENGINES else "v1"
        return _START_TOTAL_TIMEOUT[generation][wait_for]

    async def end_avatar_session(self, session_id: str) -> dict[str, Any]:
        """Stop billing for a session now. Idempotent: ending an ended session
        answers normally with ``alreadyEnded: true``. Does not remove the avatar
        from your room — :meth:`AvatarSession.aclose` does that."""
        return await self._request("POST", f"/v1/avatar_sessions/{session_id}/end")

    async def get_avatar_session(self, session_id: str) -> dict[str, Any]:
        return await self._request("GET", f"/v1/avatar_sessions/{session_id}")

    # --- avatars -----------------------------------------------------------

    async def create_avatar(
        self,
        name: str,
        image: str | Path | bytes,
        *,
        description: str | None = None,
        content_type: str | None = None,
        avatar_version: NotGivenOr[str] = NOT_GIVEN,
    ) -> str:
        """Deprecated: create avatars in the Atmee studio or with the Atmee API.

        Creates a v1 avatar from a portrait (a file path, the image bytes or
        an ``https://`` URL) and returns its id.
        """
        logger.warning(
            "create_avatar is deprecated, create avatars in the Atmee studio or with the Atmee API"
        )
        if isinstance(image, str) and image.startswith(("http://", "https://")):
            manifest = {"schemaVersion": 1, "name": name, "assets": {"image": {"url": image}}}
            if description:
                manifest["description"] = description
            created = await self._request(
                "POST", "/v1/avatars", json=manifest, retry=False, total_timeout=120.0
            )
        else:
            if isinstance(image, bytes):
                blob = image
                filename = "portrait" + (mimetypes.guess_extension(content_type or "") or ".jpg")
            else:
                blob, filename = await asyncio.to_thread(Path(image).read_bytes), Path(image).name
            form = aiohttp.FormData()
            form.add_field("name", name)
            if description:
                form.add_field("description", description)
            ctype = content_type or mimetypes.guess_type(filename)[0] or "image/jpeg"
            form.add_field("file", blob, filename=filename, content_type=ctype)
            created = await self._request(
                "POST", "/v1/avatars", data=form, retry=False, total_timeout=120.0
            )
        return str(created.get("avatarId") or created.get("id") or "")

    async def get_avatar(self, avatar_id: str) -> AvatarInfo:
        return await self._get_avatar(avatar_id)

    async def _get_avatar(self, avatar_id: str, *, deadline: float | None = None) -> AvatarInfo:
        return _avatar_info(
            await self._request("GET", f"/v1/avatars/{avatar_id}", deadline=deadline)
        )

    # --- plumbing ------------------------------------------------------------

    def _ensure_session(self) -> aiohttp.ClientSession:
        if self._session is not None and not self._session.closed:
            return self._session
        try:
            self._session = utils.http_context.http_session()
        except RuntimeError:
            # Outside a LiveKit job (a script creating avatars, tests): own one.
            self._session = aiohttp.ClientSession()
            self._owns_session = True
        return self._session

    async def aclose(self) -> None:
        """Close the HTTP session if this client created it."""
        if self._owns_session and self._session is not None and not self._session.closed:
            await self._session.close()
        self._session = None
        self._owns_session = False

    async def __aenter__(self) -> AtmeeAPI:
        return self

    async def __aexit__(self, *exc: object) -> None:
        await self.aclose()

    async def _request(
        self,
        method: str,
        path: str,
        *,
        json: dict[str, Any] | None = None,
        data: aiohttp.FormData | None = None,
        params: dict[str, str] | None = None,
        retry: bool = True,
        total_timeout: float | None = None,
        deadline: float | None = None,
    ) -> dict[str, Any]:
        """One API call with the plugin's retry policy: transport errors and
        5xx answers are retried up to ``conn_options.max_retry`` times after the
        first attempt; a 503 ``no_capacity`` or ``avatar_warming`` and every
        4xx are final. ``retry=False`` for calls that are not idempotent
        (creating a session).

        Each attempt is bounded by ``total_timeout``, ``conn_options.timeout``
        when none is given; ``deadline`` (event-loop time) bounds the whole
        call, retries and the pauses between them included."""
        loop = asyncio.get_running_loop()
        attempts = self._conn_options.max_retry + 1 if retry else 1
        per_attempt = self._conn_options.timeout if total_timeout is None else total_timeout
        last_error: Exception | None = None
        for attempt in range(attempts):
            attempt_timeout = per_attempt
            if deadline is not None:
                remaining = deadline - loop.time()
                if remaining <= 0:
                    # the budget ran out: report a timeout, keep the last failure as its cause
                    raise AtmeeException(f"{method} {path} timed out") from last_error
                attempt_timeout = min(per_attempt, remaining)
            try:
                async with self._ensure_session().request(
                    method,
                    f"{self._api_url}{path}",
                    headers={"X-Api-Key": self._api_key, "Accept": "application/json"},
                    json=json,
                    data=data,
                    params=params,
                    timeout=aiohttp.ClientTimeout(
                        sock_connect=self._conn_options.timeout, total=attempt_timeout
                    ),
                ) as response:
                    if response.ok:
                        if response.status == 204 or response.content_length == 0:
                            return {}
                        try:
                            payload = await response.json(content_type=None)
                        except ValueError as e:
                            raise AtmeeException(
                                "the Atmee API returned an invalid JSON response",
                                status_code=response.status,
                                code="invalid_response",
                            ) from e
                        return payload if isinstance(payload, dict) else {"data": payload}
                    raise await _error_from_response(response)
            except AtmeeException as e:
                if e.status_code >= 500 and not isinstance(
                    e, AtmeeNoCapacityError | AtmeeAvatarWarmingError
                ):
                    last_error = e
                else:
                    raise
            except (aiohttp.ClientError, asyncio.TimeoutError) as e:
                last_error = AtmeeException(f"could not reach the Atmee API: {e}")
            if attempt < attempts - 1:
                logger.warning(
                    "atmee api call failed; retrying",
                    extra={"path": path, "attempt": attempt + 1, "error": str(last_error)},
                )
                pause = self._conn_options.retry_interval
                if deadline is not None:
                    pause = min(pause, max(0.0, deadline - loop.time()))
                await asyncio.sleep(pause)
        assert last_error is not None
        raise last_error

Async client for the Atmee API.

Args

api_key
your sk_atmee_… key; defaults to ATMEE_API_KEY.
api_url
API base; defaults to ATMEE_API_URL or https://api.atmanity.us.
conn_options
retry/timeout policy. timeout bounds the connect of every request and each attempt of the calls that are safe to repeat (reading and ending a session, reading an avatar), which are retried max_retry times on transport failures and 5xx answers. Starting a session is never repeated and has a budget of its own.
session
an aiohttp.ClientSession to reuse. Inside a LiveKit job the worker's shared session is used; outside (scripts, tests) a private one is created and closed by :meth:aclose.

Methods

async def aclose(self) ‑> None
Expand source code
async def aclose(self) -> None:
    """Close the HTTP session if this client created it."""
    if self._owns_session and self._session is not None and not self._session.closed:
        await self._session.close()
    self._session = None
    self._owns_session = False

Close the HTTP session if this client created it.

async def create_avatar(self,
name: str,
image: str | Path | bytes,
*,
description: str | None = None,
content_type: str | None = None,
avatar_version: NotGivenOr[str] = NOT_GIVEN) ‑> str
Expand source code
async def create_avatar(
    self,
    name: str,
    image: str | Path | bytes,
    *,
    description: str | None = None,
    content_type: str | None = None,
    avatar_version: NotGivenOr[str] = NOT_GIVEN,
) -> str:
    """Deprecated: create avatars in the Atmee studio or with the Atmee API.

    Creates a v1 avatar from a portrait (a file path, the image bytes or
    an ``https://`` URL) and returns its id.
    """
    logger.warning(
        "create_avatar is deprecated, create avatars in the Atmee studio or with the Atmee API"
    )
    if isinstance(image, str) and image.startswith(("http://", "https://")):
        manifest = {"schemaVersion": 1, "name": name, "assets": {"image": {"url": image}}}
        if description:
            manifest["description"] = description
        created = await self._request(
            "POST", "/v1/avatars", json=manifest, retry=False, total_timeout=120.0
        )
    else:
        if isinstance(image, bytes):
            blob = image
            filename = "portrait" + (mimetypes.guess_extension(content_type or "") or ".jpg")
        else:
            blob, filename = await asyncio.to_thread(Path(image).read_bytes), Path(image).name
        form = aiohttp.FormData()
        form.add_field("name", name)
        if description:
            form.add_field("description", description)
        ctype = content_type or mimetypes.guess_type(filename)[0] or "image/jpeg"
        form.add_field("file", blob, filename=filename, content_type=ctype)
        created = await self._request(
            "POST", "/v1/avatars", data=form, retry=False, total_timeout=120.0
        )
    return str(created.get("avatarId") or created.get("id") or "")

Deprecated: create avatars in the Atmee studio or with the Atmee API.

Creates a v1 avatar from a portrait (a file path, the image bytes or an https:// URL) and returns its id.

async def create_avatar_session(self,
avatar_id: str,
*,
livekit_url: str,
livekit_token: str,
agent_identity: str | None = None,
max_duration_seconds: int | None = None,
metadata: dict[str, Any] | None = None,
wait_for: WaitFor = 'initializing',
avatar_version: NotGivenOr[str] = NOT_GIVEN,
stop: asyncio.Event | None = None) ‑> livekit.plugins.atmee.api.AvatarSessionInfo
Expand source code
async def create_avatar_session(
    self,
    avatar_id: str,
    *,
    livekit_url: str,
    livekit_token: str,
    agent_identity: str | None = None,
    max_duration_seconds: int | None = None,
    metadata: dict[str, Any] | None = None,
    wait_for: WaitFor = "initializing",
    avatar_version: NotGivenOr[str] = NOT_GIVEN,
    stop: asyncio.Event | None = None,
) -> AvatarSessionInfo:
    """Render ``avatar_id`` into the room ``livekit_token`` grants.

    ``livekit_token`` is a LiveKit access token you minted with YOUR
    project's secret for the avatar participant: ``kind=agent``, a
    ``roomJoin`` grant for your room, and the ``lk.publish_on_behalf``
    attribute naming your agent (:class:`AvatarSession` does this for
    you). Returns as soon as the rendering worker acknowledged the start,
    or once the avatar joined with ``wait_for="avatar_joined"``.

    Any avatar renders as what it is, v1 or v2; the request names no
    generation. The call first reads the avatar's generation to know how
    long the start may take: a v1 start may take up to 1 minute (3 with
    ``wait_for="avatar_joined"``). The first v2 session after a quiet
    period can take minutes, so a v2 start may take up to 5 (7.5), and
    while the API answers ``avatar_warming`` or ``avatar_start_interrupted``
    (nothing billed) it asks again after the ``Retry-After`` given.

    Once ``stop`` is set, the call asks no more: it raises an
    ``AtmeeException`` with ``code="closed"`` instead of asking again. A
    request already sent is still awaited, since it may open a session
    that must then be ended.
    """
    _check_wait_for(wait_for)
    _warn_avatar_version(avatar_version)
    payload: dict[str, Any] = {"livekitUrl": livekit_url, "livekitToken": livekit_token}
    if agent_identity:
        payload["agentIdentity"] = agent_identity
    if max_duration_seconds is not None:
        payload["maxDurationSeconds"] = int(max_duration_seconds)
    if metadata:
        payload["metadata"] = metadata
    params: dict[str, str] | None = (
        {"waitFor": wait_for} if wait_for != "initializing" else None
    )
    # Never retried on a timeout or a 5xx: the POST reserves and bills a
    # session as soon as the server accepts it, so asking again could open
    # a second render for the same room. Only the answers that say nothing
    # was reserved (`avatar_warming`, `avatar_start_interrupted`) are asked
    # again, within the start budget.
    loop = asyncio.get_running_loop()
    budget = await self._start_budget(avatar_id, wait_for)
    deadline = loop.time() + budget
    while True:
        if stop is not None and stop.is_set():
            raise AtmeeException("the avatar session start was stopped", code="closed")
        try:
            data = await self._request(
                "POST",
                f"/v1/avatars/{avatar_id}/avatar_sessions",
                json=payload,
                params=params,
                retry=False,
                total_timeout=max(1.0, deadline - loop.time()),
            )
            break
        except AtmeeAvatarWarmingError as e:
            pause = e.retry_after if e.retry_after is not None else _WARMING_RETRY_DEFAULT
            pause = min(max(pause, _WARMING_RETRY_BOUNDS[0]), _WARMING_RETRY_BOUNDS[1])
            if loop.time() + pause >= deadline:
                raise
            logger.info(
                "atmee avatar render not started yet; asking again",
                extra={"code": e.code, "retry_in": pause},
            )
            # nothing is reserved while waiting: a stop ends the wait at once
            if stop is None:
                await asyncio.sleep(pause)
            else:
                try:
                    await asyncio.wait_for(stop.wait(), timeout=pause)
                except asyncio.TimeoutError:
                    pass
    session_id = data.get("sessionId")
    if not isinstance(session_id, str) or not session_id:
        raise AtmeeException(
            f"the Atmee API accepted the avatar session but returned no sessionId: {data!r}",
            code="invalid_response",
        )
    return AvatarSessionInfo(
        session_id=session_id,
        status=str(data.get("status", "")),
        avatar_participant_identity=str(data.get("avatarParticipantIdentity", "")),
        agent_identity=str(data.get("agentIdentity", "")),
        room_name=str(data.get("roomName", "")),
        max_duration_seconds=int(data.get("maxDurationSeconds") or 0),
        raw=data,
    )

Render avatar_id into the room livekit_token grants.

livekit_token is a LiveKit access token you minted with YOUR project's secret for the avatar participant: kind=agent, a roomJoin grant for your room, and the lk.publish_on_behalf attribute naming your agent (:class:AvatarSession does this for you). Returns as soon as the rendering worker acknowledged the start, or once the avatar joined with wait_for="avatar_joined".

Any avatar renders as what it is, v1 or v2; the request names no generation. The call first reads the avatar's generation to know how long the start may take: a v1 start may take up to 1 minute (3 with wait_for="avatar_joined"). The first v2 session after a quiet period can take minutes, so a v2 start may take up to 5 (7.5), and while the API answers avatar_warming or avatar_start_interrupted (nothing billed) it asks again after the Retry-After given.

Once stop is set, the call asks no more: it raises an AtmeeException with code="closed" instead of asking again. A request already sent is still awaited, since it may open a session that must then be ended.

async def end_avatar_session(self, session_id: str) ‑> dict[str, typing.Any]
Expand source code
async def end_avatar_session(self, session_id: str) -> dict[str, Any]:
    """Stop billing for a session now. Idempotent: ending an ended session
    answers normally with ``alreadyEnded: true``. Does not remove the avatar
    from your room — :meth:`AvatarSession.aclose` does that."""
    return await self._request("POST", f"/v1/avatar_sessions/{session_id}/end")

Stop billing for a session now. Idempotent: ending an ended session answers normally with alreadyEnded: true. Does not remove the avatar from your room — :meth:AvatarSession.aclose() does that.

async def get_avatar(self, avatar_id: str) ‑> livekit.plugins.atmee.api.AvatarInfo
Expand source code
async def get_avatar(self, avatar_id: str) -> AvatarInfo:
    return await self._get_avatar(avatar_id)
async def get_avatar_session(self, session_id: str) ‑> dict[str, typing.Any]
Expand source code
async def get_avatar_session(self, session_id: str) -> dict[str, Any]:
    return await self._request("GET", f"/v1/avatar_sessions/{session_id}")
class AtmeeAvatarNotReadyError (message: str, *, status_code: int = 0, code: str | None = None)
Expand source code
class AtmeeAvatarNotReadyError(AtmeeException):
    """The avatar cannot be rendered yet (HTTP 409 ``avatar_not_renderable``):
    it has no portrait, or its appearance is still being processed."""

The avatar cannot be rendered yet (HTTP 409 avatar_not_renderable): it has no portrait, or its appearance is still being processed.

Ancestors

  • livekit.plugins.atmee.api.AtmeeException
  • builtins.Exception
  • builtins.BaseException
class AtmeeAvatarWarmingError (message: str, *, code: str, retry_after: float | None = None)
Expand source code
class AtmeeAvatarWarmingError(AtmeeException):
    """The render could not start yet, and nothing was billed (HTTP 503):
    ``avatar_warming``, a v2 avatar is still getting ready to render, or
    ``avatar_start_interrupted``, the render ended before the avatar joined.

    :meth:`AtmeeAPI.create_avatar_session` asks again by itself within its
    start budget; this is raised only once that budget is spent.
    ``retry_after`` is the server's hint in seconds, when it sent one.
    """

    def __init__(self, message: str, *, code: str, retry_after: float | None = None) -> None:
        super().__init__(message, status_code=503, code=code)
        self.retry_after = retry_after

The render could not start yet, and nothing was billed (HTTP 503): avatar_warming, a v2 avatar is still getting ready to render, or avatar_start_interrupted, the render ended before the avatar joined.

:meth:AtmeeAPI.create_avatar_session() asks again by itself within its start budget; this is raised only once that budget is spent. retry_after is the server's hint in seconds, when it sent one.

Ancestors

  • livekit.plugins.atmee.api.AtmeeException
  • builtins.Exception
  • builtins.BaseException
class AtmeeException (message: str, *, status_code: int = 0, code: str | None = None)
Expand source code
class AtmeeException(Exception):
    """An Atmee API call failed.

    ``status_code`` is the HTTP status (0 when the request never got an
    answer), ``code`` the machine-readable ``error`` field of the response
    body when the API sent one (``invalid_livekit_token``,
    ``avatar_not_renderable``, ``insufficient_credits``, ...), and
    ``message`` its human-readable text.
    """

    def __init__(self, message: str, *, status_code: int = 0, code: str | None = None) -> None:
        super().__init__(message)
        self.status_code = status_code
        self.code = code
        self.message = message

    def __str__(self) -> str:
        parts = [self.message]
        if self.code:
            parts.append(f"[{self.code}]")
        if self.status_code:
            parts.append(f"(HTTP {self.status_code})")
        return " ".join(parts)

An Atmee API call failed.

status_code is the HTTP status (0 when the request never got an answer), code the machine-readable error field of the response body when the API sent one (invalid_livekit_token, avatar_not_renderable, insufficient_credits, …), and message its human-readable text.

Ancestors

  • builtins.Exception
  • builtins.BaseException

Subclasses

  • livekit.plugins.atmee.api.AtmeeAvatarNotReadyError
  • livekit.plugins.atmee.api.AtmeeAvatarWarmingError
  • livekit.plugins.atmee.api.AtmeeNoCapacityError
class AtmeeNoCapacityError (message: str, *, retry_after: float | None = None)
Expand source code
class AtmeeNoCapacityError(AtmeeException):
    """Every rendering worker is busy (HTTP 503 ``no_capacity``).

    Never retried by the client: a render slot frees up on the order of
    seconds to minutes, not the milliseconds a transport retry covers.
    ``retry_after`` is the server's hint in seconds, when it sent one.
    """

    def __init__(self, message: str, *, retry_after: float | None = None) -> None:
        super().__init__(message, status_code=503, code="no_capacity")
        self.retry_after = retry_after

Every rendering worker is busy (HTTP 503 no_capacity).

Never retried by the client: a render slot frees up on the order of seconds to minutes, not the milliseconds a transport retry covers. retry_after is the server's hint in seconds, when it sent one.

Ancestors

  • livekit.plugins.atmee.api.AtmeeException
  • builtins.Exception
  • builtins.BaseException
class AvatarInfo (avatar_id: str,
status: str,
kind: str | None = None,
name: str | None = None,
engine: str | None = None,
raw: dict[str, Any] = <factory>)
Expand source code
@dataclass
class AvatarInfo:
    """What ``GET /v1/avatars/{avatarId}`` reports about an avatar.

    ``kind`` (``conversational`` | ``render_only``) says whether the avatar has
    a voice and persona; ``engine`` is its generation as the API reports it
    (``"v1"``, ``"v2"`` or ``"v2hd"``; ``None`` when the API names none).
    """

    avatar_id: str
    status: str
    kind: str | None = None
    name: str | None = None
    engine: str | None = None
    raw: dict[str, Any] = field(default_factory=dict, repr=False)

    @property
    def ready(self) -> bool:
        """Whether it can be rendered: built, and for a v2 avatar, serving a take."""
        if self.status != "ready":
            return False
        return self.engine not in _V2_ENGINES or bool(self.raw.get("servingCandidateId"))

What GET /v1/avatars/{avatarId} reports about an avatar.

kind (conversational | render_only) says whether the avatar has a voice and persona; engine is its generation as the API reports it ("v1", "v2" or "v2hd"; None when the API names none).

Instance variables

var avatar_id : str
var engine : str | None
var kind : str | None
var name : str | None
var raw : dict[str, typing.Any]
prop ready : bool
Expand source code
@property
def ready(self) -> bool:
    """Whether it can be rendered: built, and for a v2 avatar, serving a take."""
    if self.status != "ready":
        return False
    return self.engine not in _V2_ENGINES or bool(self.raw.get("servingCandidateId"))

Whether it can be rendered: built, and for a v2 avatar, serving a take.

var status : str
class AvatarSession (*,
avatar_id: str,
api_url: NotGivenOr[str] = NOT_GIVEN,
api_key: NotGivenOr[str] = NOT_GIVEN,
avatar_participant_identity: NotGivenOr[str] = NOT_GIVEN,
avatar_participant_name: NotGivenOr[str] = NOT_GIVEN,
max_duration_seconds: int = 3600,
metadata: dict[str, Any] | None = None,
wait_for: WaitFor = 'initializing',
conn_options: APIConnectOptions = APIConnectOptions(max_retry=3, retry_interval=2.0, timeout=10.0, liveness_timeout=6.0),
http_session: aiohttp.ClientSession | None = None,
avatar_version: NotGivenOr[str] = NOT_GIVEN)
Expand source code
class AvatarSession(BaseAvatarSession[Literal["avatar_disconnected"]]):
    """An Atmee avatar session: render an Atmee avatar into your agent's room.

    Any Atmee avatar renders by its id: a **v1 avatar** is a talking head
    generated from a single portrait; a **v2 avatar** moves as a whole. Both
    are created in the Atmee studio or with the Atmee API. The
    first v2 session after a quiet period can take a few minutes to start, and
    :meth:`start` waits for it.

    The avatar joins the room as its own participant (``atmee-avatar-agent``)
    publishing on behalf of your agent; your agent's audio output is replaced
    by a data stream to it, and the avatar publishes synced video and audio.
    Billing runs while the avatar is in the room and stops when your agent
    leaves, the room closes, or :meth:`aclose` runs (which also removes the
    avatar participant); a ceiling of ``max_duration_seconds`` bounds it.

    Emits ``avatar_disconnected`` (the ``rtc.RemoteParticipant``) if the
    avatar participant leaves the room while the session is still running,
    for example because its rendering worker died.
    """

    def __init__(
        self,
        *,
        avatar_id: str,
        api_url: NotGivenOr[str] = NOT_GIVEN,
        api_key: NotGivenOr[str] = NOT_GIVEN,
        avatar_participant_identity: NotGivenOr[str] = NOT_GIVEN,
        avatar_participant_name: NotGivenOr[str] = NOT_GIVEN,
        max_duration_seconds: int = DEFAULT_MAX_DURATION_SECONDS,
        metadata: dict[str, Any] | None = None,
        wait_for: WaitFor = "initializing",
        conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS,
        http_session: aiohttp.ClientSession | None = None,
        avatar_version: NotGivenOr[str] = NOT_GIVEN,
    ) -> None:
        """
        Args:
            avatar_id: the Atmee avatar to render, v1 or v2.
            api_key: your ``sk_atmee_...`` key; defaults to ``ATMEE_API_KEY``.
            api_url: API base; defaults to ``ATMEE_API_URL`` or the Atmee cloud.
            avatar_participant_identity: identity the avatar joins with
                (``atmee-avatar-agent``). Set it when one room hosts several
                avatars.
            max_duration_seconds: hard ceiling of the session (default 3600).
            metadata: free-form JSON attached to the session on the Atmee side.
            wait_for: ``"initializing"`` (default) returns from :meth:`start`
                once the worker acknowledged the start; ``"avatar_joined"``
                blocks until the avatar is in the room. Anything else raises
                ``ValueError``.
            conn_options: connect timeout and retries of the API calls that
                are safe to repeat (reading the avatar before the start, ending
                the session); see :class:`AtmeeAPI`.
            http_session: an aiohttp session to reuse; the job's shared one by
                default, or a private one outside a job (released by
                :meth:`aclose`).
            avatar_version: deprecated and ignored; the avatar's own engine
                decides how it renders.
        """
        _check_wait_for(wait_for)
        _warn_avatar_version(avatar_version)
        if not avatar_id:
            raise AtmeeException("avatar_id is required")
        super().__init__()
        self._avatar_id = avatar_id
        self._max_duration_seconds = int(max_duration_seconds)
        self._metadata = metadata
        self._wait_for: WaitFor = wait_for
        self._conn_options = conn_options
        # Resolved lazily by AtmeeAPI: the job's shared session inside a
        # worker, a private one (closed by aclose) outside.
        self._api = AtmeeAPI(
            api_key=api_key,
            api_url=api_url,
            conn_options=conn_options,
            session=http_session,
        )

        self._avatar_participant_identity = avatar_participant_identity or _AVATAR_AGENT_IDENTITY
        self._avatar_participant_name = avatar_participant_name or _AVATAR_AGENT_NAME

        self.session_id: str | None = None
        """The Atmee avatar session id once :meth:`start` returned."""
        self.session_info: AvatarSessionInfo | None = None
        """The full start response once :meth:`start` returned."""

        self._end_task: asyncio.Task[None] | None = None
        self._end_lock = asyncio.Lock()
        self._close_lock = asyncio.Lock()
        self._ended = False
        self._started = False
        self._render_requested = False
        # aclose() was called: a start still waiting on its request then asks no
        # more (_stop_start) and ends what the request opens instead of wiring
        # it up. The request itself is never cancelled: the session it may open
        # could not be ended without the id its answer carries.
        self._closed = False
        self._stop_start = asyncio.Event()
        self._start_request: asyncio.Future[AvatarSessionInfo] | None = None
        self._settle_tasks: set[asyncio.Task[bool]] = set()
        self._room_for_events: rtc.Room | None = None
        self._agent_session_for_events: AgentSession[Any] | None = None
        self._aclose_task: asyncio.Task[None] | None = None

    @property
    def avatar_identity(self) -> str:
        return self._avatar_participant_identity

    @property
    def provider(self) -> str:
        return "atmee"

    async def start(
        self,
        agent_session: AgentSession[Any],
        room: rtc.Room,
        *,
        livekit_url: NotGivenOr[str] = NOT_GIVEN,
        livekit_api_key: NotGivenOr[str] = NOT_GIVEN,
        livekit_api_secret: NotGivenOr[str] = NOT_GIVEN,
    ) -> None:
        """Start rendering the avatar into ``room`` for ``agent_session``.

        Mints a LiveKit access token for the avatar participant with YOUR
        project's credentials (``LIVEKIT_URL`` / ``LIVEKIT_API_KEY`` /
        ``LIVEKIT_API_SECRET`` or the arguments), asks Atmee to render into
        the room, and routes the agent's audio to the avatar. Called before
        ``agent_session.start``, the avatar speaks from the agent's first
        word; called after it, the agent speaks with its own voice until the
        avatar takes over. A v1 avatar's start is acknowledged within seconds;
        the first v2 session after a quiet period can take a few minutes, and
        this waits for it.

        One-shot: each call creates a separately billed render, so a second
        ``start()`` on the same instance raises. After a failed start, create
        a new ``AvatarSession``. If :meth:`aclose` runs while the start is
        still waiting (the job shuts down, the agent session closes), the
        start asks no more, the render it opened is ended and this raises an
        ``AtmeeException`` with ``code="closed"``. If this call is cancelled,
        its request still finishes in the background and the render it opens
        is ended.
        """
        if self._started:
            raise AtmeeException(
                "AvatarSession.start() was already called; create a new AvatarSession "
                "for another render"
            )
        self._started = True
        await super().start(agent_session, room)
        try:
            await self._start_render(
                agent_session,
                room,
                livekit_url=livekit_url,
                livekit_api_key=livekit_api_key,
                livekit_api_secret=livekit_api_secret,
            )
        except BaseException:
            # Undo what super().start() set up (listeners, join task). Once the
            # render was requested its outcome is uncertain, so close fully:
            # end a session that may exist and remove the avatar participant.
            # Before that, no avatar of ours can be in the room, and removing
            # the participant could disconnect another session's avatar that
            # uses the same identity. Either way the instance stays spent.
            try:
                if self._render_requested:
                    await self.aclose()
                else:
                    await self._release_without_removing_participant()
            except Exception:
                # never let a teardown error replace the real start failure
                logger.warning("cleanup after a failed atmee avatar start failed", exc_info=True)
            raise

    async def _release_without_removing_participant(self) -> None:
        async with self._close_lock:
            room = self._room
            if room is not None:
                room.off("connection_state_changed", self._on_connection_state_changed)
                # the base aclose() removes the avatar identity from a
                # connected room; with no room it only drops its listener
                # and cancels the join task
                self._room = None
            await super().aclose()
            await self._api.aclose()

    async def _start_render(
        self,
        agent_session: AgentSession[Any],
        room: rtc.Room,
        *,
        livekit_url: NotGivenOr[str],
        livekit_api_key: NotGivenOr[str],
        livekit_api_secret: NotGivenOr[str],
    ) -> None:

        livekit_url = livekit_url or (os.getenv("LIVEKIT_URL") or NOT_GIVEN)
        livekit_api_key = livekit_api_key or (os.getenv("LIVEKIT_API_KEY") or NOT_GIVEN)
        livekit_api_secret = livekit_api_secret or (os.getenv("LIVEKIT_API_SECRET") or NOT_GIVEN)
        if not livekit_url or not livekit_api_key or not livekit_api_secret:
            raise AtmeeException(
                "livekit_url, livekit_api_key, and livekit_api_secret must be set "
                "by arguments or environment variables"
            )

        job_ctx = get_job_context(required=False)
        local_participant_identity = (
            job_ctx.local_participant_identity
            if job_ctx is not None
            else room.local_participant.identity
        )
        livekit_token = (
            api.AccessToken(api_key=livekit_api_key, api_secret=livekit_api_secret)
            .with_kind("agent")
            .with_identity(self._avatar_participant_identity)
            .with_name(self._avatar_participant_name)
            .with_grants(api.VideoGrants(room_join=True, room=room.name))
            # allow the avatar to publish audio and video on behalf of your local agent
            .with_attributes({ATTRIBUTE_PUBLISH_ON_BEHALF: local_participant_identity})
            .with_ttl(timedelta(seconds=self._max_duration_seconds) + _TOKEN_TTL_SLACK)
            .to_jwt()
        )

        logger.debug(
            "starting atmee avatar session",
            extra={"avatar_id": self._avatar_id, "lk.pii.room": room.name},
        )
        if self._closed:
            # closed before the render was requested: never open a billed render
            raise AtmeeException("the avatar session was closed before it started", code="closed")
        self._render_requested = True
        # The agent session can close without the job shutting down
        # (AgentSession.aclose(), an error), even while the start is still
        # waiting: end the render then too, or it would bill until
        # max_duration_seconds.
        self._agent_session_for_events = agent_session
        agent_session.on("close", self._on_agent_session_close)
        request = asyncio.ensure_future(
            self._api.create_avatar_session(
                self._avatar_id,
                livekit_url=livekit_url,
                livekit_token=livekit_token,
                agent_identity=local_participant_identity,
                max_duration_seconds=self._max_duration_seconds,
                metadata=self._metadata,
                wait_for=self._wait_for,
                stop=self._stop_start,
            )
        )
        self._start_request = request
        try:
            # waits without cancelling the request when start() itself is cancelled
            await asyncio.wait({request})
        except asyncio.CancelledError:
            # The caller gave up on start() (the job shuts down, a timeout of
            # its own): let the request finish and end what it opens.
            self._closed = True
            self._stop_start.set()
            task = asyncio.ensure_future(self._settle_closed_start(request))
            self._settle_tasks.add(task)
            task.add_done_callback(self._settle_tasks.discard)
            raise
        if self._closed:
            # aclose() ran while the request was waiting and left the teardown
            # to this start: end the session the API opened, if any, remove
            # the avatar and release the client, wiring nothing up.
            opened = await self._settle_closed_start(request)
            raise AtmeeException(
                "the avatar session was closed while it was starting"
                + ("; its render was ended" if opened else ""),
                code="closed",
            )
        info = request.result()
        self.session_info = info
        self.session_id = info.session_id
        logger.info(
            "atmee avatar session started",
            extra={
                "session_id": info.session_id,
                "status": info.status,
                "max_duration_seconds": info.max_duration_seconds,
            },
        )

        self._room_for_events = room
        room.on("participant_disconnected", self._on_participant_disconnected)

        agent_session.output.replace_audio_tail(
            _AvatarAudioOutput(
                room=room,
                destination_identity=self._avatar_participant_identity,
                sample_rate=SAMPLE_RATE,
                wait_remote_track=rtc.TrackKind.KIND_VIDEO,
                wait_playback_start=True,
            ),
        )

    def _on_participant_disconnected(self, participant: rtc.RemoteParticipant) -> None:
        if participant.identity != self._avatar_participant_identity or self._ended:
            return
        # The avatar left while we are still running: its worker died or was
        # removed. The worker reports its own end to Atmee; ending here too
        # closes the gap for a worker that could not (idempotent).
        logger.warning(
            "atmee avatar participant left the room",
            extra={"session_id": self.session_id, "lk.pii.identity": participant.identity},
        )
        self.emit("avatar_disconnected", participant)
        self._schedule_end()

    def _on_agent_session_close(self, _: Any) -> None:
        if self._aclose_task is None:
            self._aclose_task = asyncio.create_task(self.aclose())

    def _schedule_end(self) -> None:
        if self._end_task is None or self._end_task.done():
            self._end_task = asyncio.create_task(self._end_session())

    async def _settle_closed_start(self, request: asyncio.Future[AvatarSessionInfo]) -> bool:
        """Close everything once a start closed mid-request has its answer:
        end the session it opened, if any. Returns whether one was opened."""
        await asyncio.wait({request})
        info = None
        if not request.cancelled() and request.exception() is None:
            info = request.result()  # failed or stopped requests opened nothing
        if info is not None:
            self.session_id = info.session_id
        async with self._close_lock:
            await self._aclose_locked()
        return info is not None

    async def _end_session(self) -> None:
        """Best-effort ``POST /v1/avatar_sessions/{id}/end``.

        Serialized, and marked done only once the API confirmed the end (the
        call is idempotent), so a failed or cancelled attempt can be retried
        by a later :meth:`aclose`.
        """
        async with self._end_lock:
            if self._ended or not self.session_id:
                return
            try:
                await self._api.end_avatar_session(self.session_id)
            except AtmeeException as e:
                if 400 <= e.status_code < 500 and e.status_code not in (408, 429):
                    # The session is unknown or already settled on the Atmee
                    # side: nothing left to end, retrying would not help.
                    self._ended = True
                logger.warning(
                    "failed to end atmee avatar session; the worker's own report or the "
                    "session ceiling will settle it",
                    extra={"session_id": self.session_id, "error": str(e)},
                )
                return
            except Exception as e:  # never let a billing hint break teardown
                logger.warning(
                    "failed to end atmee avatar session; the worker's own report or the "
                    "session ceiling will settle it",
                    extra={"session_id": self.session_id, "error": str(e)},
                )
                return
            self._ended = True
            logger.debug("atmee avatar session ended", extra={"session_id": self.session_id})

    async def _finish_end(self) -> None:
        # an end already under way (the avatar left on its own) finishes first,
        # so none is still running once the HTTP client is released
        if self._end_task is not None and not self._end_task.done():
            try:
                await self._end_task
            except Exception:
                pass
        await self._end_session()

    async def aclose(self) -> None:
        """End the Atmee session and remove the avatar from the room.

        Registered as a job shutdown callback by :meth:`start`; call it
        yourself when running outside a job. Safe to call more than once and
        concurrently (the agent session's ``close`` event also triggers it):
        calls are serialized, so the HTTP session is never closed while
        another call is still ending the render.
        """
        self._closed = True
        self._stop_start.set()
        async with self._close_lock:
            await self._aclose_locked()

    async def _aclose_locked(self) -> None:
        if self._room_for_events is not None:
            self._room_for_events.off("participant_disconnected", self._on_participant_disconnected)
            self._room_for_events = None
        if self._agent_session_for_events is not None:
            self._agent_session_for_events.off("close", self._on_agent_session_close)
            self._agent_session_for_events = None
        if self._start_request is not None and not self._start_request.done():
            # A start is still waiting on its request, with no session to end
            # yet: once the request returns, the session it opened is ended and
            # everything here is closed (see _settle_closed_start).
            return
        # Remove the avatar participant and end the render side by side, so the
        # avatar leaves at once even when the end call is slow: at job shutdown
        # the process has only a few seconds.
        results = await asyncio.gather(super().aclose(), self._finish_end(), return_exceptions=True)
        for result in results:
            if isinstance(result, Exception):
                logger.warning("atmee avatar session cleanup step failed", exc_info=result)
        await self._api.aclose()

An Atmee avatar session: render an Atmee avatar into your agent's room.

Any Atmee avatar renders by its id: a v1 avatar is a talking head generated from a single portrait; a v2 avatar moves as a whole. Both are created in the Atmee studio or with the Atmee API. The first v2 session after a quiet period can take a few minutes to start, and :meth:start waits for it.

The avatar joins the room as its own participant (atmee-avatar-agent) publishing on behalf of your agent; your agent's audio output is replaced by a data stream to it, and the avatar publishes synced video and audio. Billing runs while the avatar is in the room and stops when your agent leaves, the room closes, or :meth:aclose runs (which also removes the avatar participant); a ceiling of max_duration_seconds bounds it.

Emits avatar_disconnected (the rtc.RemoteParticipant) if the avatar participant leaves the room while the session is still running, for example because its rendering worker died.

Args

avatar_id
the Atmee avatar to render, v1 or v2.
api_key
your sk_atmee_… key; defaults to ATMEE_API_KEY.
api_url
API base; defaults to ATMEE_API_URL or the Atmee cloud.
avatar_participant_identity
identity the avatar joins with (atmee-avatar-agent). Set it when one room hosts several avatars.
max_duration_seconds
hard ceiling of the session (default 3600).
metadata
free-form JSON attached to the session on the Atmee side.
wait_for
"initializing" (default) returns from :meth:start once the worker acknowledged the start; "avatar_joined" blocks until the avatar is in the room. Anything else raises ValueError.
conn_options
connect timeout and retries of the API calls that are safe to repeat (reading the avatar before the start, ending the session); see :class:AtmeeAPI.
http_session
an aiohttp session to reuse; the job's shared one by default, or a private one outside a job (released by :meth:aclose).
avatar_version
deprecated and ignored; the avatar's own engine decides how it renders.

Ancestors

  • livekit.agents.voice.avatar._types.AvatarSession
  • abc.ABC
  • EventEmitter
  • typing.Generic

Instance variables

prop avatar_identity : str
Expand source code
@property
def avatar_identity(self) -> str:
    return self._avatar_participant_identity

The participant identifier of the avatar

prop provider : str
Expand source code
@property
def provider(self) -> str:
    return "atmee"

The provider of the avatar

var session_id

The Atmee avatar session id once :meth:start returned.

var session_info

The full start response once :meth:start returned.

Methods

async def aclose(self) ‑> None
Expand source code
async def aclose(self) -> None:
    """End the Atmee session and remove the avatar from the room.

    Registered as a job shutdown callback by :meth:`start`; call it
    yourself when running outside a job. Safe to call more than once and
    concurrently (the agent session's ``close`` event also triggers it):
    calls are serialized, so the HTTP session is never closed while
    another call is still ending the render.
    """
    self._closed = True
    self._stop_start.set()
    async with self._close_lock:
        await self._aclose_locked()

End the Atmee session and remove the avatar from the room.

Registered as a job shutdown callback by :meth:start; call it yourself when running outside a job. Safe to call more than once and concurrently (the agent session's close event also triggers it): calls are serialized, so the HTTP session is never closed while another call is still ending the render.

async def start(self,
agent_session: AgentSession[Any],
room: rtc.Room,
*,
livekit_url: NotGivenOr[str] = NOT_GIVEN,
livekit_api_key: NotGivenOr[str] = NOT_GIVEN,
livekit_api_secret: NotGivenOr[str] = NOT_GIVEN) ‑> None
Expand source code
async def start(
    self,
    agent_session: AgentSession[Any],
    room: rtc.Room,
    *,
    livekit_url: NotGivenOr[str] = NOT_GIVEN,
    livekit_api_key: NotGivenOr[str] = NOT_GIVEN,
    livekit_api_secret: NotGivenOr[str] = NOT_GIVEN,
) -> None:
    """Start rendering the avatar into ``room`` for ``agent_session``.

    Mints a LiveKit access token for the avatar participant with YOUR
    project's credentials (``LIVEKIT_URL`` / ``LIVEKIT_API_KEY`` /
    ``LIVEKIT_API_SECRET`` or the arguments), asks Atmee to render into
    the room, and routes the agent's audio to the avatar. Called before
    ``agent_session.start``, the avatar speaks from the agent's first
    word; called after it, the agent speaks with its own voice until the
    avatar takes over. A v1 avatar's start is acknowledged within seconds;
    the first v2 session after a quiet period can take a few minutes, and
    this waits for it.

    One-shot: each call creates a separately billed render, so a second
    ``start()`` on the same instance raises. After a failed start, create
    a new ``AvatarSession``. If :meth:`aclose` runs while the start is
    still waiting (the job shuts down, the agent session closes), the
    start asks no more, the render it opened is ended and this raises an
    ``AtmeeException`` with ``code="closed"``. If this call is cancelled,
    its request still finishes in the background and the render it opens
    is ended.
    """
    if self._started:
        raise AtmeeException(
            "AvatarSession.start() was already called; create a new AvatarSession "
            "for another render"
        )
    self._started = True
    await super().start(agent_session, room)
    try:
        await self._start_render(
            agent_session,
            room,
            livekit_url=livekit_url,
            livekit_api_key=livekit_api_key,
            livekit_api_secret=livekit_api_secret,
        )
    except BaseException:
        # Undo what super().start() set up (listeners, join task). Once the
        # render was requested its outcome is uncertain, so close fully:
        # end a session that may exist and remove the avatar participant.
        # Before that, no avatar of ours can be in the room, and removing
        # the participant could disconnect another session's avatar that
        # uses the same identity. Either way the instance stays spent.
        try:
            if self._render_requested:
                await self.aclose()
            else:
                await self._release_without_removing_participant()
        except Exception:
            # never let a teardown error replace the real start failure
            logger.warning("cleanup after a failed atmee avatar start failed", exc_info=True)
        raise

Start rendering the avatar into room for agent_session.

Mints a LiveKit access token for the avatar participant with YOUR project's credentials (LIVEKIT_URL / LIVEKIT_API_KEY / LIVEKIT_API_SECRET or the arguments), asks Atmee to render into the room, and routes the agent's audio to the avatar. Called before agent_session.start, the avatar speaks from the agent's first word; called after it, the agent speaks with its own voice until the avatar takes over. A v1 avatar's start is acknowledged within seconds; the first v2 session after a quiet period can take a few minutes, and this waits for it.

One-shot: each call creates a separately billed render, so a second start() on the same instance raises. After a failed start, create a new AvatarSession. If :meth:aclose runs while the start is still waiting (the job shuts down, the agent session closes), the start asks no more, the render it opened is ended and this raises an AtmeeException with code="closed". If this call is cancelled, its request still finishes in the background and the render it opens is ended.

Inherited members

class AvatarSessionInfo (session_id: str,
status: str,
avatar_participant_identity: str,
agent_identity: str,
room_name: str,
max_duration_seconds: int,
raw: dict[str, Any] = <factory>)
Expand source code
@dataclass
class AvatarSessionInfo:
    """The ``202`` body of ``POST /v1/avatars/{avatarId}/avatar_sessions``."""

    session_id: str
    status: str
    avatar_participant_identity: str
    agent_identity: str
    room_name: str
    max_duration_seconds: int
    raw: dict[str, Any] = field(default_factory=dict, repr=False)

The 202 body of POST /v1/avatars/{avatarId}/avatar_sessions.

Instance variables

var agent_identity : str
var avatar_participant_identity : str
var max_duration_seconds : int
var raw : dict[str, typing.Any]
var room_name : str
var session_id : str
var status : str