LiveKit docs › Partner spotlight › AWS › Amazon Nova Sonic

---

# Amazon Nova Sonic integration guide

> How to use the Amazon Nova Sonic model with LiveKit Agents.

Available in:
- [ ] Node.js
- [x] Python

## Overview

Amazon [Nova Sonic](https://aws.amazon.com/ai/generative-ai/nova/speech/) is a state-of-the-art speech-to-speech model with a bidirectional audio streaming API. Nova Sonic processes and responds to realtime speech as it occurs, enabling natural, human-like conversational experiences. LiveKit's AWS plugin includes support for both Nova Sonic 1.0 and Nova Sonic 2.0 on AWS Bedrock.

### Installation

Install the AWS plugin from PyPI with the `realtime` extra:

```shell
uv add "livekit-plugins-aws[realtime]"

```

### Authentication

The AWS plugin requires AWS credentials. Set the following environment variables in your `.env` file:

```shell
AWS_ACCESS_KEY_ID=<your-aws-access-key-id>
AWS_SECRET_ACCESS_KEY=<your-aws-secret-access-key>

```

### Usage

Use the Nova Sonic API within an `AgentSession`. For example, you can use it in the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai.md).

```python
from livekit.plugins import aws

# Nova Sonic 2.0 (default) - supports text input and 16 voices
session = AgentSession(
    llm=aws.realtime.RealtimeModel(),
)

# Or explicitly specify version
session = AgentSession(
    llm=aws.realtime.RealtimeModel.with_nova_sonic_2(
        voice="tiffany",
        turn_detection="MEDIUM"
    ),
)

# Nova Sonic 1.0 - audio-only, 11 voices
session = AgentSession(
    llm=aws.realtime.RealtimeModel.with_nova_sonic_1(
        voice="matthew"
    ),
)

```

### Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the [plugin reference](https://docs.livekit.io/reference/python/livekit/plugins/aws/experimental/realtime/realtime_model.html.md#livekit.plugins.aws.experimental.realtime.realtime_model.RealtimeModel).

- **`model`** _(string)_ (optional) - Default: `"amazon.nova-2-sonic-v1:0"`: Bedrock model ID for realtime inference. Use `"amazon.nova-sonic-v1:0"` for Nova Sonic 1.0.

- **`modalities`** _(string)_ (optional) - Default: `"mixed"`: Input/output mode. Valid values are:

- `"mixed"` enables audio and text input (Nova Sonic 2.0 only).
- `"audio"` enables audio-only (Nova Sonic 1.0).

- **`voice`** _(string)_ (optional) - Default: `"tiffany"`: Name of the Nova Sonic API voice. Nova Sonic 2.0 supports 16 voices across 8 languages. Nova Sonic 1.0 supports 11 voices. For a full list, see [Voices](https://docs.aws.amazon.com/nova/latest/userguide/available-voices.html).

- **`turn_detection`** _(string)_ (optional) - Default: `"MEDIUM"`: Turn-taking sensitivity. Valid values are `"HIGH"`, `"MEDIUM"`, and `"LOW"`. To learn more, see [Turn detection sensitivity](#turn-detection).

- **`region`** _(string)_ (optional) - Default: `"us-east-1"`: AWS region of the Bedrock runtime endpoint. Defaults to `"us-east-1"`.

- **`generate_reply_timeout`** _(float)_ (optional) - Default: `10.0`: Timeout in seconds for `generate_reply()` calls. This is the maximum time to wait before aborting the request.

## Turn detection sensitivity

Nova Sonic includes built-in VAD-based turn detection with configurable sensitivity:

- "HIGH": Fast responses, detects pauses quickly. Might interrupt slower speakers.
- "MEDIUM": Balanced turn-taking. Recommended for most use cases.
- "LOW": Patient, waits longer before responding. Best for thoughtful or hesitant speakers.

Configure turn detection when creating the model:

```python
session = AgentSession(
    llm=aws.realtime.RealtimeModel.with_nova_sonic_2(
        turn_detection="MEDIUM"
    ),
)

```

## Text input with Nova Sonic 2.0

Nova Sonic 2.0 supports text input via the `generate_reply()` method, enabling programmatic agent responses. This is useful for having the agent speak first, providing instructions, or simulating user input. Use `modalities="mixed"` to enable text input.

### Parameters

- **`instructions`** _(string)_ (optional): Instructions for the agent to use for the reply. This is sent as a system prompt to guide the model's response.

- **`user_input`** _(string)_ (optional): User input to respond to. This is sent as a user message to the model and added to Nova's conversation context.

### Examples

Make the agent greet users when they join:

```python
class Assistant(Agent):
    async def on_enter(self):
        await self.session.generate_reply(
            instructions="Greet the user warmly and introduce your capabilities"
        )

```

Send instructions to the agent to generate a response:

```python
await session.generate_reply(
    instructions="Greet the user and ask how you can help"
)

```

Send user input to the agent to generate a response:

```python
await session.generate_reply(
    user_input="Hello, I need help with my account"
)

```

### Timeout configuration

Configure a timeout for `generate_reply()` calls:

```python
session = AgentSession(
    llm=aws.realtime.RealtimeModel.with_nova_sonic_2(
        generate_reply_timeout=15.0  # seconds
    ),
)

```

## Function calling

To use [function tools](https://docs.livekit.io/agents/build/tools.md) with Nova Sonic, define tools with the `@function_tool` decorator and register them on your `Agent`. Nova Sonic invokes them during the conversation and incorporates the results into its spoken response.

Tool results are sent back to Nova Sonic in JSON format, so return a string or any JSON-serializable value such as a `dict`:

```python
from livekit.agents import ToolError, function_tool

@function_tool
async def get_weather(city: str, units: str = "fahrenheit") -> dict[str, str | int] | ToolError:
    """Retrieve the current weather for a city.

    Args:
        city: The city to get the weather for.
        units: Temperature units, either 'celsius' or 'fahrenheit'.
    """
    try:
        weather = await fetch_weather(city, units)
    except Exception as e:
        # After receiving this error, Nova Sonic will inform the user that the request wasn't successful and to try again
        return ToolError(f"Could not get weather for {city}: {e}")

    return {"city": city, "temperature": weather.temperature, "units": units}

```

When a tool returns (or raises) a `ToolError`, the message is delivered to Nova Sonic as `{"error": "<message>"}`, allowing the model to recover gracefully.

## Additional resources

The following resources provide more information about using Nova Sonic with LiveKit Agents.

- **[Python package](https://pypi.org/project/livekit-plugins-aws/)**: The `livekit-plugins-aws` package on PyPI.

- **[Plugin reference](https://docs.livekit.io/reference/python/livekit/plugins/aws/experimental/realtime/index.html.md)**: Reference for the Nova Sonic integration.

- **[GitHub repo](https://github.com/livekit/agents/tree/main/livekit-plugins/livekit-plugins-aws/livekit/plugins/aws/)**: View the source or contribute to the LiveKit AWS plugin.

- **[Nova Sonic docs](https://docs.aws.amazon.com/nova/latest/userguide/speech.html)**: Nova Sonic API documentation.

- **[Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai.md)**: Get started with LiveKit Agents and Amazon Nova Sonic.

- **[Joke Teller example](https://github.com/livekit/agents/blob/main/examples/voice_agents/realtime_joke_teller.py)**: Full example demonstrating Nova Sonic 2.0 features including text prompting, multilingual support, and function calling.

- **[AWS AI ecosystem guide](https://docs.livekit.io/agents/integrations/aws.md)**: Overview of the entire AWS AI and LiveKit Agents integration.

---

This document was rendered at 2026-06-07T11:35:38.458Z.
For the latest version of this document, see [https://docs.livekit.io/agents/models/realtime/plugins/nova-sonic.md](https://docs.livekit.io/agents/models/realtime/plugins/nova-sonic.md).

To explore all LiveKit documentation, see [llms.txt](https://docs.livekit.io/llms.txt).