Fireworks AI LLM integration guide

How to use Fireworks AI Llama models with LiveKit Agents.

Overview

Fireworks AI provides access to Llama 3.1 instruction-tuned models through their inference API. These models are multilingual and text-only, making them suitable for a variety of agent applications.

Usage

Install the OpenAI plugin to add Fireworks AI support:

pip install "livekit-agents[openai]~=1.0"

Set the following environment variable in your .env file:

FIREWORKS_API_KEY=<your-fireworks-api-key>

Create a Fireworks AI LLM using the with_fireworks method:

from livekit.plugins import openai
session = AgentSession(
llm=openai.LLM.with_fireworks(
model="accounts/fireworks/models/llama-v3p3-70b-instruct",
temperature=0.7
),
# ... tts, stt, vad, turn_detection, etc.
)

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the method reference.

modelstrOptionalDefault: accounts/fireworks/models/llama-v3p3-70b-instruct

Model to use for inference. To learn more, see supported models.

temperaturefloatOptionalDefault: 1.0

Controls the randomness of the model's output. Higher values (e.g., 0.8) make the output more random, while lower values (e.g., 0.2) make it more focused and deterministic.

Valid values are between 0 and 1.5.

parallel_tool_callsboolOptional

Controls whether the model can make multiple tool calls in parallel. When enabled, the model can make multiple tool calls simultaneously, which can improve performance for complex tasks.

tool_choiceToolChoice | Literal['auto', 'required', 'none']OptionalDefault: auto

Controls how the model uses tools. Set to 'auto' to let the model decide, 'required' to force tool usage, or 'none' to disable tool usage.

The following links provide more information about the Fireworks AI LLM integration.