Cerebras integration guide

An introduction to using the LiveKit Agents framework with Cerebras LLM.

On this page

OverviewQuick referenceEnvironment variablesLLM
Try the playground

Chat with a voice assistant built with LiveKit and Cerebras LLM

Try the playground

Overview

Cerebras provides high-throughput, low-latency AI inference for Meta's Llama LLM models. With LiveKit's Cerebras integration and the Agents framework, you can build responsive AI voice applications. For a demonstration of what you can build, check out the voice assistant built with Cerebras and LiveKit.

Note

If you're looking to build an AI voice assistant with Cerebras, check out our Voice Agent Quickstart guide and use the Cerebras LLM module as demonstrated below.

Quick reference

Environment variables

CEREBRAS_API_KEY=<your-cerebras-api-key>

LLM

LiveKit's Cerebras integration provides an OpenAI compatible LLM interface. This can be used in a VoicePipelineAgent. For a complete reference of all available parameters, see the plugin reference.

Usage

from livekit.plugins.openai import llm


cerebras_llm = llm.LLM.with_cerebras(
  model="llama3.1-8b",
  temperature=0.8,
)

Parameters

modelstringOptionalDefault: llama3.1-8b

ID of the model to use for inference. See supported models.

temperaturefloatOptionalDefault: 1.0

A measure of randomness of completions. A lower temperature is more deterministic. To learn more, see the Cerebras documentation.

GitHubSlack
Sign in