Amazon Bedrock LLM integration guide

Overview

Amazon Bedrock is a fully managed service that provides a wide range of pre-trained models. With LiveKit's open source Bedrock integration and the Agents framework, you can build sophisticated voice AI applications using models from a wide variety of providers.

Quick reference

This section includes a basic usage example and some reference material. For links to more detailed documentation, see Additional resources.

Installation

Install the plugin from PyPI:

pip install "livekit-agents[aws]~=1.0"

Authentication

The AWS plugin requires AWS credentials. Set the following environment variables in your .env file:

AWS_ACCESS_KEY_ID=<your-aws-access-key-id>
AWS_SECRET_ACCESS_KEY=<your-aws-secret-access-key>

Usage

Use Bedrock within an AgentSession or as a standalone LLM service. For example, you can use this LLM in the Voice AI quickstart.

from livekit.plugins import aws

session = AgentSession(
    llm=aws.LLM(
        model="anthropic.claude-3-5-sonnet-20240620-v1:0",
        temperature=0.8,
    ),
    # ... tts, stt, vad, turn_detection, etc.
)

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

modelstring | TEXT_MODELOptionalDefault: anthropic.claude-3-5-sonnet-20240620-v1:0

The model to use for the LLM. For more information, see the documentation for the modelId parameter in the Amazon Bedrock API reference.

regionstringOptionalDefault: us-east-1

The region to use for AWS API requests.

temperaturefloatOptional

A measure of randomness in output. A lower value results in more predictable output, while a higher value results in more creative output.

Default values vary depending on the model you select. To learn more, see Inference request parameters and response fields for foundation models.

tool_choice[ToolChoice | Literal['auto', 'required', 'none']]OptionalDefault: auto

Specifies whether to use tools during response generation.

Additional resources

The following links provide more information about the Amazon Bedrock LLM plugin.

Python package

The livekit-plugins-aws package on PyPI.

Plugin reference

Reference for the Amazon Bedrock LLM plugin.

GitHub repo

View the source or contribute to the LiveKit Amazon Bedrock LLM plugin.

Bedrock docs

Amazon Bedrock docs.

Voice AI quickstart

Get started with LiveKit Agents and Amazon Bedrock.

AWS ecosystem guide

Overview of the entire AWS and LiveKit Agents integration.