Skip to main content

Groq LLM plugin guide

How to use the Groq LLM plugin for LiveKit Agents.

On this page

OverviewQuick referenceInstallationAuthenticationUsageParametersAdditional resources
Available in
Python
|
Node.js
Try out Groq Talk

See our voice assistant playground using Groq STT, LLM, and TTS

Try out Groq Talk

Overview

This plugin allows you to use Groq as an LLM provider for your voice agents. Groq compatibility for Node.js is provided by the OpenAI plugin using the Groq Chat Completions API.

LiveKit Inference

Some Groq models are also available in LiveKit Inference, with billing and integration handled automatically. See the docs for more information.

Quick reference

This section includes a basic usage example and some reference material. For links to more detailed documentation, see Additional resources.

Installation

Install the plugin:

uv add "livekit-agents[groq]~=1.4"
pnpm add @livekit/agents-plugin-openai@1.x

Authentication

The Groq plugin requires a Groq API key.

Set GROQ_API_KEY in your .env file.

Usage

Use a Groq LLM in your AgentSession or as a standalone LLM service. For example, you can use this LLM in the Voice AI quickstart.

from livekit.plugins import groq


session = AgentSession(
    llm=groq.LLM(
        model="llama3-8b-8192"
    ),
    # ... tts, stt, vad, turn_detection, etc.
)
import * as openai from '@livekit/agents-plugin-openai';


const session = new voice.AgentSession({
    llm: openai.LLM.withGroq({
        model: "llama3-8b-8192"
    }),
    // ... tts, stt, vad, turn_detection, etc.
});

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

modelstringOptionalDefault: llama-3.3-70b-versatile

Name of the LLM model to use. For all options, see the Groq model list.

temperaturefloatOptionalDefault: 1.0

Controls the randomness of the model's output. Higher values, for example 0.8, make the output more random, while lower values, for example 0.2, make it more focused and deterministic.

parallel_tool_callsboolOptional

Controls whether the model can make multiple tool calls in parallel. When enabled, the model can make multiple tool calls simultaneously, which can improve performance for complex tasks.

tool_choiceToolChoice | Literal['auto', 'required', 'none']OptionalDefault: auto

Controls how the model uses tools. Set to 'auto' to let the model decide, 'required' to force tool usage, or 'none' to disable tool usage.

Additional resources

The following resources provide more information about using Groq with LiveKit Agents.

Python plugin

Python plugin

ReferenceGitHubPyPI
Node.js plugin

Node.js plugin

ReferenceGitHubNPM

Groq docs

Groq's official API documentation.

Voice AI quickstart

Get started with LiveKit Agents and Groq.

Groq ecosystem overview

Overview of the entire Groq and LiveKit Agents integration.