Amazon Bedrock LLM integration guide

How to use the Amazon Bedrock LLM plugin for LiveKit Agents.

Overview

Amazon Bedrock is a fully managed service that provides a wide range of pre-trained models. With LiveKit's open source Bedrock integration and the Agents framework, you can build sophisticated voice AI applications using models from a wide variety of providers.

Quick reference

This section includes a basic usage example and some reference material. For links to more detailed documentation, see Additional resources.

Installation

Install the plugin from PyPI:

pip install "livekit-agents[aws]~=1.0"

Authentication

The AWS plugin requires AWS credentials. Set the following environment variables in your .env file:

AWS_ACCESS_KEY_ID=<your-aws-access-key-id>
AWS_SECRET_ACCESS_KEY=<your-aws-secret-access-key>

Usage

Use Bedrock within an AgentSession or as a standalone LLM service. For example, you can use this LLM in the Voice AI quickstart.

from livekit.plugins import aws
session = AgentSession(
llm=aws.LLM(
model="anthropic.claude-3-5-sonnet-20240620-v1:0",
temperature=0.8,
),
# ... tts, stt, vad, turn_detection, etc.
)

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

modelstring | TEXT_MODELOptionalDefault: anthropic.claude-3-5-sonnet-20240620-v1:0

The model to use for the LLM. For more information, see the documentation for the modelId parameter in the Amazon Bedrock API reference.

regionstringOptionalDefault: us-east-1

The region to use for AWS API requests.

temperaturefloatOptional

A measure of randomness in output. A lower value results in more predictable output, while a higher value results in more creative output.

Default values vary depending on the model you select. To learn more, see Inference request parameters and response fields for foundation models.

tool_choice[ToolChoice | Literal['auto', 'required', 'none']]OptionalDefault: auto

Specifies whether to use tools during response generation.

Additional resources

The following links provide more information about the Amazon Bedrock LLM plugin.