OpenAI and LiveKit

Try LiveKit.fm

Chat with OpenAI's latest models in a LiveKit demo inspired by OpenAI.fm

OpenAI ecosystem support

OpenAI  provides some of the most powerful AI models and services today, which integrate into LiveKit Agents in the following ways:

• GPT-Live: A full-duplex voice model that listens and speaks at the same time. It's available through LiveKit Inference and the OpenAI plugin.
• Realtime API: A turn-based speech-to-speech realtime model with live video input.
• GPT 4.1, GPT 5, GPT 4o, and more: Smart and creative models for voice AI.
• STT models: Realtime transcription with gpt-realtime-whisper, plus gpt-4o-transcribe and whisper-1.
• TTS models: Use OpenAI's latest gpt-4o-mini-tts to generate lifelike speech in a voice pipeline.

OpenAI LLM and realtime models are available in LiveKit Inference with billing and integration handled automatically. The OpenAI plugin for LiveKit Agents provides support for the full OpenAI developer platform  as well as Azure OpenAI Service .

Getting started

Use the following guide to speak to your own OpenAI-powered voice AI agent in less than 10 minutes.

LiveKit Agents overview

LiveKit Agents is an open source framework for building realtime AI apps in Python and Node.js. It supports complex voice AI workflows with multiple agents and discrete processing steps, and includes built-in load balancing.

LiveKit provides SIP support for telephony integration and full-featured frontend SDKs in multiple languages. It uses WebRTC transport for end-user devices, enabling high-quality, low-latency realtime experiences. To learn more, see LiveKit Agents.

Realtime API

LiveKit Agents serves as a bridge between your frontend (connected over WebRTC) and the OpenAI Realtime API (connected over WebSockets). LiveKit automatically converts Realtime API audio response buffers to WebRTC audio streams synchronized with text, and handles business logic like interruption handling automatically. You can add your own logic within your agent, and use LiveKit features for realtime state and data to coordinate with your frontend.

Additional benefits of LiveKit Agents for the Realtime API include:

• Noise cancellation: One line of code to remove background noise and speakers from your input audio.
• Telephony: Inbound and outbound calling using SIP trunks.
• Interruption handling: Automatically handles context truncation on interruption.
• Transcription sync: Realtime API text output is synced to audio playback automatically.

LiveKit Inference

Use the following code to use an OpenAI LLM in your AgentSession:

from livekit.agents import AgentSession

session = AgentSession(
    llm="openai/chat-latest",
)

import { AgentSession } from '@livekit/agents';

const session = new AgentSession({
    llm: "openai/chat-latest",
});

Use the following code to use an OpenAI realtime model in your AgentSession:

from livekit.agents import AgentSession, inference

session = AgentSession(
    llm=inference.RealtimeModel(model="openai/gpt-realtime-2.1"),
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    llm: new inference.RealtimeModel({ model: "openai/gpt-realtime-2.1" }),
});

Use the following code to use GPT-Live in your AgentSession:

from livekit.agents import AgentSession, inference

session = AgentSession(
    llm=inference.DuplexModel(model="openai/gpt-live-1"),
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    llm: new inference.DuplexModel({ model: "openai/gpt-live-1" }),
});

For more information, see the following topics:

OpenAI integration guides

The following links provide more information on each available OpenAI component in LiveKit Agents.