Groq and LiveKit

Try Groq transcription

Experience Groq's fast STT in a LiveKit-powered playground

Groq ecosystem support

Groq provides fast AI inference in the cloud and on-prem AI compute centers. LiveKit Agents can integrate with the following Groq services:

• STT: Fast and cost-effective English or multilingual transcription based on whisper-large-v3.
• TTS: Fast English and Arabic text-to-speech based on playai-tts.
• LLM: Fast inference for open models like llama-3.1-8b-instant and more.

Getting started

Use the Voice AI quickstart to build a voice AI app with Groq. Select an STT-LLM-TTS pipeline model type and add the following components to build on Groq.

Install the Groq plugin:

pip install "livekit-agents[groq]~=1.0rc"

Add your Groq API key to your .env. file:

GROQ_API_KEY=<Your Groq API Key>

Use Groq components to initialize your AgentSession:

from livekit.plugins import groq

# ...

# in your entrypoint function
session = AgentSession(
    stt=groq.STT(
        model="whisper-large-v3-turbo",
        language="en",
    ),
    llm=groq.LLM(
        model="llama3-8b-8192"
    ),
    tts=groq.TTS(
      model="playai-tts",
      voice="Arista-PlayAI",
    ),
    # ... vad, turn_detection, etc.
)

LiveKit Agents overview

LiveKit Agents is an open-source framework for building realtime AI apps using WebRTC transport to end-user devices and WebSockets or HTTPS for backend services.

• Agent workflows: Build complex voice AI apps with discrete stages and handoffs.
• Telephony: Inbound and outbound calling using SIP trunks.
• Frontend SDKs: Full-featured SDKs and UI components for JavaScript, Swift, Kotlin, Flutter, React Native, and Unity.
• Python and Node.js: Build voice AI apps in Python or Node.js.
• Dispatch and load balancing: Built-in support for request distribution and load balancing.
• LiveKit Cloud: Fully-managed LiveKit server with global scale and low latency (you can also self-host).

Groq plugin documentation

The following links provide more information on each available Groq component in LiveKit Agents.