Groq integration guide

An introduction to using the LiveKit Agents framework with Groq STT and LLM.

Try out Groq STT live

See Groq's STT in action with our realtime transcription playground

Try out Groq STT live

Overview

Groq provides low-latency AI inference with deterministic results and automatic speech recognition. LiveKit's Groq integration provides both STT and LLM functionality through an OpenAI-compatible interface. Use Groq and the Agents framework to build AI voice assistants that are realistic and predictable with accurate transcriptions.

This guide walks you through the steps to build a live transcription application that uses LiveKit's Agents Framework and the Groq STT service. For a demonstration of the following application, see the LiveKit and Groq transcription app.

Note

If you're looking to build an AI voice assistant with Groq, check out our Voice Agent Quickstart guide and use the Groq integration as your STT and/or LLM provider.

Prerequisites

Instructions

Setup a LiveKit account and install the CLI

  1. Create an account or sign in to your LiveKit Cloud account.

  2. Install the LiveKit CLI and authenticate using lk cloud auth — (Optional).

Tip

The LiveKit CLI utility lk is a convenient way to setup and configure applications and manage your LiveKit services, but installing it isn't required.

Bootstrap an agent from template

  1. Clone a starter template for your preferred language using the CLI:

    lk app create \
    --template-url https://github.com/livekit-examples/transcription-groq-python

    If you aren't using the LiveKit CLI, clone the repository yourself:

    git clone https://github.com/livekit-examples/transcription-groq-python
  2. Enter your Groq API Key when prompted or manually add your environment variables:

    GROQ_API_KEY=<your-groq-api-key>
    LIVEKIT_API_KEY=<your-livekit-api-key>
    LIVEKIT_API_SECRET=<your-livekit-api-secret>
    LIVEKIT_URL=<your-livekit-url>
  3. Install dependencies and start your agent:

    cd <agent_dir>
    python3 -m venv venv
    source venv/bin/activate
    python3 -m pip install -r requirements.txt
    python3 main.py dev
Note

For more details on using the STT module to perform transcription outside of the context of a voice pipeline or multimodal agent, see the transcriptions documentation.

Create a minimal frontend with Next.js

  1. Clone the Transcription Frontend Next.js app starter template using the CLI:

    lk app create --template transcription-frontend

    If you aren't using the LiveKit CLI, clone the repository yourself:

    git clone https://github.com/livekit-examples/transcription-frontend
  2. Enter your your environment variables:

    LIVEKIT_API_KEY=<your-livekit-api-key>
    LIVEKIT_API_SECRET=<your-livekit-api-secret>
    NEXT_PUBLIC_LIVEKIT_URL=<your-livekit-url>
  3. Install dependencies and start your frontend application:

    cd <frontend_dir>
    pnpm install
    pnpm dev

Launch your app and talk to your agent

  1. Visit your locally-running application (by default, http://localhost:3000).
  2. Select Start voice transcription and begin speaking.

Quick reference

Environment variables

GROQ_API_KEY=<your-groq-api-key>

STT

LiveKit's Groq integration provides an OpenAI compatible speech-to-text (STT) interface. This can be used as the first stage in a VoicePipelineAgent or as a standalone transcription service as documented above. For a complete reference of all available parameters, see the plugin reference.

Usage

from livekit.plugins.openai import stt
groq_stt = stt.STT.with_groq(
model="whisper-large-v3-turbo",
language="en",
)

Parameters

modelstringOptionalDefault: whisper-large-v3-turbo

ID of the model to use for inference. See supported models.

languagestringOptionalDefault: en

Language of input audio in ISO-639-1 format.

detect_languageboolOptionalDefault: false

Whether or not language should be detected from the audio stream. Not every model supports language detection. See supported models.

LLM

LiveKit's Groq integration also provides an OpenAI compatible LLM interface. This can be used in a VoicePipelineAgent. For a complete reference of all available parameters, see the plugin reference.

Usage

from livekit.plugins.openai import llm
groq_llm = llm.LLM.with_groq(
model="llama3-8b-8192",
temperature=0.8,
)

Parameters

modelstringOptionalDefault: llama3-8b-8192

ID of the model to use for inference. For a complete list, see supported models.

temperaturefloatOptionalDefault: 1.0

A measure of randomness of completions. A lower temperature is more deterministic. To learn more, see chat completions.