OpenAI integration guide

Overview

LiveKit's OpenAI integration provides support for LLM, STT, TTS, and the OpenAI Realtime API. The Quick reference section in this topic describes using OpenAI for LLM, STT, and TTS. For a guide on using the Realtime API, see the OpenAI Realtime API integration guide.

Use OpenAI STT, TTS, and LLM to create agents using the VoicePipelineAgent class. To use the Realtime API, you can create an agent using the MultimodalAgent class.

Quick reference

The following sections provide a quick reference for integrating OpenAI with LiveKit. For the complete reference, see the links provided in each section.

LLM

LiveKit's OpenAI plugin provides support for creating an instance of an LLM class to be used in a VoicePipelineAgent.

LLM class usage

Create an instance of OpenAI LLM:

from livekit.plugins.openai import llm

openai_llm = llm.LLM(
  model="gpt-4o",
  temperature=0.8,
)

LLM parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

STT

LiveKit's OpenAI plugin allows you to create an instance of OpenAI STT that can be used as the first stage in a VoicePipelineAgent or as a standalone transcription service.

STT usage

Create an OpenAI STT:

from livekit.plugins.openai import stt

openai_stt = stt.STT(
  language="en",
  model="whisper-1",
)

STT parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.

TTS

LiveKit's OpenAI plugin allows you to create an instance of OpenAI TTS that can be used in a VoicePipelineAgent or as a standalone speech generator.

TTS usage

Create an OpenAI TTS:

from livekit.plugins.openai import tts

openai_tts = tts.TTS(
  model="tts-1",
  voice="nova",
)

TTS parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference.