Skip to main content
IntroductionBuild AgentsAgent FrontendsTest & EvaluateTelephonyRoboticsWebRTC TransportManage & DeployReferences
XYouTubeSlack

Rime Prism

How to use the Rime Prism speech-to-speech model with LiveKit Agents.

On this page

OverviewLiveKit Inference Usage Parameters Plugin InstallationAuthenticationUsageParameters Playback ConsiderationsAdditional resources

Overview

Prism  is a speech-to-speech model from Rime. It supports tool calls, task handoffs, and interruptions through the standard LiveKit realtime model interface.

Prism is available through LiveKit Inference and the Rime plugin. With LiveKit Inference, you use your LiveKit Cloud credentials and LiveKit Cloud bills you for usage. Use the plugin to bring your own Rime API key.

LiveKit Inference

Use LiveKit Inference to access Prism without a separate Rime API key. The model ID is rime/prism.

Usage

Use the RealtimeModel class from the inference module to run Prism through LiveKit Inference. You can use this model in the Voice AI quickstart:

from livekit.agents import AgentSession, inference


session = AgentSession(
    llm=inference.RealtimeModel(model="rime/prism"),
)

Authentication uses your LiveKit Cloud credentials. Set LIVEKIT_API_KEY and LIVEKIT_API_SECRET in your .env file, or pass api_key and api_secret directly.

Parameters

The following are the main parameters for configuring Prism with LiveKit Inference. For a complete list, see the API reference.

model
Required
string

The model to use. Set to rime/prism.

voicestring

Voice for speech output, such as astra.

api_keystring

LiveKit API key. Defaults to the LIVEKIT_API_KEY environment variable.

api_secretstring

LiveKit API secret. Defaults to the LIVEKIT_API_SECRET environment variable.

Plugin

Use the Rime plugin to connect directly to Prism with your own API key.

Available inPython

Installation

Install the plugin with the realtime extra. It requires Python 3.11 or later.

uv add "livekit-plugins-rime[realtime]"

Authentication

Set RIME_API_KEY in your .env file. Get a key from the Rime dashboard .

Usage

Pass the model to AgentSession as the llm. You can use it in the Voice AI quickstart.

from livekit.agents import AgentSession
from livekit.plugins import rime
from rimelabs_sdk.realtime import TurnDetection


session = AgentSession(
    llm=rime.realtime.RealtimeModel(
        turn_detection=TurnDetection(eagerness="low"),
    ),
)

Create one model per AgentSession.

Parameters

This section describes some of the available parameters.

voicestring

Voice for speech output. Defaults to the deployment's default voice.

turn_detectionTurnDetection

Server turn detection settings. eagerness controls how readily Prism ends a turn: low, medium, high, or auto. end_of_turn_timeout_ms sets the silence limit, from 1000 to 10000 milliseconds. Unset fields use the server defaults.

playbackLiteral['auto', 'estimated', 'discard']Default: auto

How the plugin reports playback completion: auto, estimated, or discard. discard drops audio and keeps text and tools, which is useful for tests. See Playback.

context_modeLiteral['strict', 'shared']Default: shared

How chat context updates apply. shared keeps one conversation across agent and task handoffs and preserves omitted items. strict replaces provider history with the updated context.

tool_fillersMapping[str, PrismFiller]

Filler speech per tool name while a tool runs. For example, {"lookup": PrismFiller(type="text", text="Checking your order.")}.

endpointstringDefault: wss://prism.api.rime.ai/v1/realtime

Endpoint URL for a custom deployment.

Playback

With playback="auto", wrap the audio output to report measured playback. Otherwise, the plugin uses Prism's estimates. Install the wrapper with model.wrap_audio_output() in on_enter, before the first reply.

Considerations

Prism has some differences from other realtime models:

  • Server turn detection only. You can't turn off Prism's turn detection. Tune it with turn_detection.
  • Audio output only. Prism can't generate text-only responses, so you can't pair it with a TTS plugin in a realtime hybrid setup.
  • No required tool choice. tool_choice supports auto and none only.
  • Text history only. Imported context supports text messages and completed tool calls. Images and video aren't supported.
  • No session resumption. The plugin doesn't reconnect a dropped session. Use a fallback adapter to recover.

Additional resources

The following resources provide more information about using Prism with LiveKit Agents.

Python plugin

Python plugin

ReferenceGitHubPyPI

Rime Prism docs

Rime Prism documentation.

Voice AI quickstart

Get started with LiveKit Agents and Prism.

Rime TTS plugin

Use Rime for text-to-speech.