Overview
Cerebras provides access to Llama 3.1 and 3.3 models through their inference API. These models are multilingual and text-only, making them suitable for a variety of agent applications.
Usage
Install the OpenAI plugin to add Cerebras support:
pip install "livekit-agents[openai]~=1.0"
Set the following environment variable in your .env
file:
CEREBRAS_API_KEY=<your-cerebras-api-key>
Create a Cerebras LLM using the with_cerebras
method:
from livekit.plugins import openaisession = AgentSession(llm=openai.LLM.with_cerebras(model="llama3.1-8b",temperature=0.7),# ... tts, stt, vad, turn_detection, etc.)
Parameters
This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.
Model to use for inference. To learn more, see supported models.
A measure of randomness in output. A lower value results in more predictable output, while a higher value results in more creative output.
Valid values are between 0
and 1.5
. To learn more, see the Cerebras documentation.
Set to true to parallelize tool calls.
Specifies whether to use tools during response generation.
Links
The following links provide more information about the Cerebras LLM integration.
Python package
The livekit-plugins-openai
package on PyPI.
Plugin reference
Reference for the with_cerebras
method of the OpenAI LLM plugin.
GitHub repo
View the source or contribute to the LiveKit OpenAI LLM plugin.
Cerebras docs
Cerebras inference docs.
Voice AI quickstart
Get started with LiveKit Agents and Cerebras.