OpenAI STT plugin guide

Overview

This plugin allows you to use OpenAI  as an STT provider for your voice agents.

Installation

Install the plugin from PyPI:

uv add "livekit-agents[openai]~=1.5"

pnpm add @livekit/agents-plugin-openai@1.x

Authentication

The OpenAI plugin requires an OpenAI API key .

Set OPENAI_API_KEY in your .env file.

Usage

Use OpenAI STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

from livekit.plugins import openai

session = AgentSession(
  stt = openai.STT(
    model="gpt-4o-mini-transcribe",
  ),
  # ... llm, tts, etc.
)

import { inference, voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';

const vad = new inference.VAD({ model: 'silero' });

const session = new voice.AgentSession({
    stt: new openai.STT({
        model: 'gpt-realtime-whisper',
        vad,
    }),
    // ... llm, tts, etc.
});

const stt = new openai.STT({ useRealtime: false });

Using an OpenAI-compatible gateway

Realtime transcription streams over a WebSocket, which you can route through an OpenAI-compatible gateway or proxy, such as LiteLLM , Cloudflare AI Gateway, Helicone, or Portkey. Set the baseURL to your gateway endpoint and set the apiKey:

import { voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';
import * as silero from '@livekit/agents-plugin-silero';

const vad = await silero.VAD.load();

const session = new voice.AgentSession({
    stt: new openai.STT({
        model: 'gpt-realtime-whisper',
        // Connects to wss://your-gateway.example.com/v1/realtime?intent=transcription&model=gpt-realtime-whisper
        baseURL: 'https://your-gateway.example.com/v1',
        apiKey: process.env.GATEWAY_API_KEY,
        vad,
    }),
    // ... llm, tts, etc.
});

The plugin adds the model as a ?model= query parameter on the connection URL when the host isn't api.openai.com, so the gateway can route each connection to the correct model. An https:// base URL maps to wss://, and http:// maps to ws://.

Connecting directly to OpenAI omits the ?model= parameter by design. In that case, the plugin sends the model in the session configuration instead.

Parameters

This section describes some of the available parameters. See the plugin reference links in the Additional resources section for a complete list of all available parameters.

Additional resources

The following resources provide more information about using OpenAI with LiveKit Agents.

Python package

ReferenceGitHubPyPI

Node.js plugin

ReferenceGitHubNPM

OpenAI docs

OpenAI STT docs.

Voice AI quickstart

Get started with LiveKit Agents and OpenAI STT.

OpenAI ecosystem guide

Overview of the entire OpenAI and LiveKit Agents integration.