Skip to main content

NVIDIA PersonaPlex plugin guide

How to use the NVIDIA PersonaPlex realtime model with LiveKit Agents.

On this page

OverviewInstallationAuthenticationUsageParametersConsiderationsAdditional resources
Available in
Python
Experimental feature

The PersonaPlex realtime model integration is experimental and the API might change in a future release.

Overview

PersonaPlex is a full-duplex speech-to-speech model from NVIDIA. It listens and speaks simultaneously, enabling natural conversational flow with interruptions and backchannels.

Installation

Install the NVIDIA plugin:

uv add "livekit-plugins-nvidia[personaplex]"

Authentication

PersonaPlex connects to a self-hosted server. Set the server URL as an environment variable in your .env file:

PERSONAPLEX_URL=ws://localhost:8998

If PERSONAPLEX_URL isn't set, the plugin defaults to localhost:8998.

Usage

Use PersonaPlex in an AgentSession. For example, you can use it in the Voice AI quickstart.

from livekit.plugins import nvidia


session = AgentSession(
    llm=nvidia.realtime.RealtimeModel(
        text_prompt="You are a helpful voice AI assistant.",
    ),
)

PersonaPlex uses text_prompt on the model for system instructions. Agent-level instructions aren't forwarded to PersonaPlex, so set your instructions here instead.

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

base_urlstrDefault: "localhost:8998"

WebSocket URL of the PersonaPlex server. If not set, reads from the PERSONAPLEX_URL environment variable.

voicestrDefault: "NATF2"

Voice to use for speech output. For a full list, see Voices.

text_promptstrDefault: "You are a helpful assistant."

System instruction for the model.

seedint

Seed for reproducible generation. See NVIDIA's prompting guide.

silence_threshold_msintDefault: 500

Duration of silence in milliseconds before the model finalizes a generation.

Considerations

PersonaPlex has some differences from other realtime model plugins:

  • Self-hosted only. Deploy a PersonaPlex server separately. See the NVIDIA PersonaPlex GitHub repo for setup instructions.
  • No tool calling. PersonaPlex doesn't support function calling or tools.
  • No user transcription. PersonaPlex doesn't provide transcriptions of user speech. If you need to log or display what the user said, add a separate STT plugin.
  • No conversation history. Loading prior conversation context isn't supported. Each session starts fresh.
  • Client-side silence detection only. Turn detection relies on the silence_threshold_ms parameter (default 500ms). There's no server-side turn detection or VAD integration.
  • Audio only. Video input isn't supported.

Additional resources

The following resources provide more information about using PersonaPlex with LiveKit Agents.

Icon for Python package

Python package

The livekit-plugins-nvidia package on PyPI.

Plugin reference

Reference for the PersonaPlex integration.

GitHub repo

View the source or contribute to the LiveKit NVIDIA plugin.

PersonaPlex research

NVIDIA's PersonaPlex research page and paper.

PersonaPlex model

PersonaPlex server source code and deployment instructions.

Voice AI quickstart

Get started with LiveKit Agents and PersonaPlex.