Fireworks AI LLM plugin guide | LiveKit Documentation

Available in

Python

Node.js

Overview

This plugin allows you to use Fireworks AI as an LLM provider for your voice agents. Fireworks AI compatibility is provided by the OpenAI plugin using the Fireworks AI Chat Completions API.

Usage

Install the OpenAI plugin to add Fireworks AI support:

uv add "livekit-agents[openai]~=1.4"

pnpm add @livekit/agents-plugin-openai@1.x

Set the following environment variable in your .env file:

FIREWORKS_API_KEY=<your-fireworks-api-key>

Create a Fireworks AI LLM using the with_fireworks method:

from livekit.plugins import openai

session = AgentSession(
    llm=openai.LLM.with_fireworks(
        model="accounts/fireworks/models/llama-v3p3-70b-instruct",
    ),
    # ... tts, stt, vad, turn_handling, etc.
)

import * as openai from '@livekit/agents-plugin-openai';

const session = new voice.AgentSession({
    llm: openai.LLM.withFireworks({
        model: "accounts/fireworks/models/llama-v3p3-70b-instruct",
    }),
    // ... tts, stt, vad, turnHandling, etc.
});

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference links in the Additional resources section.

modelstrDefault: accounts/fireworks/models/llama-v3p3-70b-instruct

Model to use for inference. To learn more, see supported models.

temperaturefloatDefault: 1.0

Sampling temperature that controls the randomness of the model's output. Higher values make the output more random, while lower values make it more focused and deterministic. Range of valid values can vary by model.

Valid values are between 0 and 1.5.

parallel_tool_callsbool

Controls whether the model can make multiple tool calls in parallel. When enabled, the model can make multiple tool calls simultaneously, which can improve performance for complex tasks.

tool_choiceToolChoice | Literal['auto', 'required', 'none']Default: auto

Controls how the model uses tools. String options are as follows:

'auto': Let the model decide.
'required': Force tool usage.
'none': Disable tool usage.

Additional resources

The following links provide more information about the Fireworks AI LLM integration.

Python plugin

Reference GitHub PyPI

Node.js plugin

Reference GitHub NPM

Fireworks AI docs

Fireworks AI API documentation.

Voice AI quickstart

Get started with LiveKit Agents and Fireworks AI.