OpenAI STT

Overview

OpenAI speech-to-text is available in LiveKit Agents through LiveKit Inference and the OpenAI plugin. With LiveKit Inference, your agent runs on LiveKit's infrastructure to minimize latency. No separate provider API key is required, and usage and rate limits are managed through LiveKit Cloud. Use the plugin instead if you want to manage your own billing and rate limits. Pricing for LiveKit Inference is available on the pricing page .

LiveKit Inference

Use LiveKit Inference to access OpenAI STT without a separate OpenAI API key.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

Usage

To use OpenAI, use the STT class from the inference module:

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT(
        model="openai/gpt-live-transcribe",
        language="en"
    ),
    # ... llm, tts, vad, turn_handling, etc.
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    stt: new inference.STT({
        model: "openai/gpt-live-transcribe",
        language: "en"
    }),
    // ... llm, tts, vad, turnHandling, etc.
});

Audio format and turn detection

openai/gpt-live-transcribe accepts streaming audio only as 24 kHz PCM or 8 kHz μ-law. The Agents SDK streams PCM, and LiveKit Inference rejects other sample rates instead of resampling them, so if you set sample_rate (sampleRate in Node.js), set it to 24000.

The model has no built-in end-of-turn detection. It emits a final transcript each time your agent finalizes the current audio segment, which the Agents SDK does when it detects the end of speech.

Parameters

String descriptors

As a shortcut, you can also pass a model ID string directly to the stt argument in your AgentSession:

from livekit.agents import AgentSession

session = AgentSession(
    stt="openai/gpt-live-transcribe:en",
    # ... llm, tts, vad, turn_handling, etc.
)

import { AgentSession } from '@livekit/agents';

const session = new AgentSession({
    stt: "openai/gpt-live-transcribe:en",
    // ... llm, tts, vad, turnHandling, etc.
});

Plugin

LiveKit's plugin support for OpenAI  lets you connect directly to OpenAI's STT API with your own API key.

Installation

Install the plugin from PyPI:

uv add "livekit-agents[openai]~=1.8"

pnpm add @livekit/agents-plugin-openai@1.x

Authentication

The OpenAI plugin requires an OpenAI API key .

Set OPENAI_API_KEY in your .env file.

Usage

Use OpenAI STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

from livekit.plugins import openai

session = AgentSession(
  stt = openai.STT(
    model="gpt-4o-mini-transcribe",
  ),
  # ... llm, tts, etc.
)

import { inference, voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';

const vad = new inference.VAD({ model: 'silero' });

const session = new voice.AgentSession({
    stt: new openai.STT({
        model: 'gpt-realtime-whisper',
        vad,
    }),
    // ... llm, tts, etc.
});

const stt = new openai.STT({ useRealtime: false });

Using an OpenAI-compatible gateway

Realtime transcription streams over a WebSocket, which you can route through an OpenAI-compatible gateway or proxy, such as LiteLLM , Cloudflare AI Gateway, Helicone, or Portkey. Set the baseURL to your gateway endpoint and set the apiKey:

import { voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';
import * as silero from '@livekit/agents-plugin-silero';

const vad = await silero.VAD.load();

const session = new voice.AgentSession({
    stt: new openai.STT({
        model: 'gpt-realtime-whisper',
        // Connects to wss://your-gateway.example.com/v1/realtime?intent=transcription&model=gpt-realtime-whisper
        baseURL: 'https://your-gateway.example.com/v1',
        apiKey: process.env.GATEWAY_API_KEY,
        vad,
    }),
    // ... llm, tts, etc.
});

The plugin adds the model as a ?model= query parameter on the connection URL when the host isn't api.openai.com, so the gateway can route each connection to the correct model. An https:// base URL maps to wss://, and http:// maps to ws://.

Connecting directly to OpenAI omits the ?model= parameter by design. In that case, the plugin sends the model in the session configuration instead.

Parameters

This section describes some of the available parameters. See the plugin reference links in the Additional resources section for a complete list of all available parameters.

Additional resources

The following resources provide more information about using OpenAI with LiveKit Agents.

Python package

ReferenceGitHubPyPI

Node.js plugin

ReferenceGitHubNPM

OpenAI docs

OpenAI STT docs.

Voice AI quickstart

Get started with LiveKit Agents and OpenAI STT.

OpenAI ecosystem guide

Overview of the entire OpenAI and LiveKit Agents integration.