LiveKit docs › Partner spotlight › Azure › Azure OpenAI Realtime API --- # Azure OpenAI Realtime API and LiveKit > How to use the Azure OpenAI Realtime API with LiveKit Agents. Available in: - [x] Node.js - [x] Python ## Overview [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/) provides an implementation of OpenAI's Realtime API that enables low-latency, multimodal interactions with realtime audio and text processing through Azure's managed service. Use LiveKit's Azure OpenAI plugin to create an agent that uses the Realtime API. > ℹ️ **Note** > > Using the OpenAI platform instead of Azure? See our [OpenAI Realtime API guide](https://docs.livekit.io/agents/models/realtime/plugins/openai.md). ### Installation Install the OpenAI plugin from PyPI: **Python**: ```shell uv add "livekit-agents[openai]~=1.5" ``` --- **Node.js**: ```shell pnpm add @livekit/agents-plugin-openai@1.x ``` ### Authentication The Azure OpenAI plugin requires an [Azure OpenAI API key](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/create-resource) and your Azure OpenAI endpoint. Set the following environment variables in your `.env` file: ```shell AZURE_OPENAI_API_KEY= AZURE_OPENAI_ENDPOINT= ``` ### Usage Use the Azure OpenAI Realtime API within an `AgentSession`: **Python**: ```python from livekit.plugins import openai session = AgentSession( llm=openai.realtime.RealtimeModel.with_azure( azure_deployment="", azure_endpoint="wss://.openai.azure.com/", api_key="", ), ) ``` --- **Node.js**: ```typescript import * as openai from '@livekit/agents-plugin-openai'; const session = new voice.AgentSession({ llm: openai.realtime.RealtimeModel.withAzure({ azureDeployment: "", azureEndpoint: "wss://.openai.azure.com/", apiKey: "", }), }); ``` For a more comprehensive agent example, see the [Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai.md). ### Preview API deployments Azure deprecated the preview Realtime API on April 30, 2026. The `api_version` parameter and `OPENAI_API_VERSION` environment variable target the preview API and are no longer supported. Migrate to a GA deployment to avoid service disruption. To learn more, see Microsoft's [Realtime API migration guide](https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/realtime-audio-preview-api-migration-guide). ### Parameters This section describes the Azure-specific parameters. For a complete list of all available parameters, see the plugin reference links in the [Additional resources](#additional-resources) section. - **`azure_deployment`** _(string)_: Name of your model deployment. - **`entra_token`** _(string)_ (optional): Microsoft Entra ID authentication token. Required if not using API key authentication. To learn more see Azure's [Authentication](https://learn.microsoft.com/en-us/azure/ai-services/openai/realtime-audio-reference#authentication) documentation. - **`voice`** _(string)_ (optional) - Default: `alloy`: Voice to use for speech. To learn more, see [Voice options](https://developers.openai.com/docs/guides/text-to-speech#voice-options). - **`temperature`** _(float)_ (optional) - Default: `1.0`: Sampling temperature that controls the randomness of the model's output. Higher values make the output more random, while lower values make it more focused and deterministic. Range of valid values can vary by model. Valid values are between `0` and `2`. To learn more, see [chat completions](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create#(resource)%20chat.completions%20%3E%20(method)%20create%20%3E%20(params)%200.non_streaming%20%3E%20(param)%20temperature%20%3E%20(schema)). - **`instructions`** _(string)_ (optional) - Default: ``: Initial system instructions. - **`modalities`** _(list[api_proto.Modality])_ (optional) - Default: `["text", "audio"]`: Modalities to use, such as ["text", "audio"]. Set to `["text"]` to use the model in text-only mode with a [separate TTS plugin](#separate-tts). - **`turn_detection`** _(TurnDetection | None)_ (optional): Configuration for turn detection, see the section on [Turn detection](#turn-detection) for more information. ## Turn detection The Azure OpenAI Realtime API includes [voice activity detection (VAD)](https://learn.microsoft.com/en-us/azure/ai-services/openai/realtime-audio-reference#realtimeturndetection) to automatically detect when a user has started or stopped speaking. This feature is enabled by default There is one supported mode for VAD: - **Server VAD** (default) - Uses periods of silence to automatically chunk the audio ### Server VAD Server VAD is the default mode and can be configured with the following properties: **Python**: ```python from livekit.plugins.openai import realtime from openai.types.beta.realtime.session import TurnDetection session = AgentSession( llm=realtime.RealtimeModel( turn_detection=TurnDetection( type="server_vad", threshold=0.5, prefix_padding_ms=300, silence_duration_ms=200, create_response=True, interrupt_response=True, ) ), # ... vad, tts, stt, etc. ) ``` --- **Node.js**: ```typescript import * as openai from '@livekit/agents-plugin-openai'; const session = new voice.AgentSession({ llm: new openai.realtime.RealtimeModel({ turnDetection: { type: 'server_vad', threshold: 0.5, prefix_padding_ms: 300, silence_duration_ms: 200, create_response: true, interrupt_response: true, }, }), }); ``` - `threshold`: Higher values require louder audio to activate, better for noisy environments. - `prefix_padding_ms`: Amount of audio to include before detected speech. - `silence_duration_ms`: Duration of silence to detect speech stop (shorter = faster turn detection). ## Usage with separate TTS To use the Azure OpenAI Realtime API with a different [TTS instance](https://docs.livekit.io/agents/models/tts.md), configure it with a text-only response modality and include a TTS instance in your `AgentSession` configuration. This configuration allows you to gain the benefits of direct speech understanding while maintaining complete control over the speech output. **Python**: ```python session = AgentSession( llm=openai.realtime.RealtimeModel.with_azure( # ... endpoint and auth params ..., modalities=["text"] ), tts="cartesia/sonic-3" # Or other TTS instance of your choice ) ``` --- **Node.js**: ```typescript import * as openai from '@livekit/agents-plugin-openai'; const session = new voice.AgentSession({ llm: openai.realtime.RealtimeModel.withAzure({ // ... endpoint and auth params ..., modalities: ["text"] }), tts: "cartesia/sonic-3", // Or other TTS instance of your choice }); ``` ## Loading conversation history If you load conversation history into the model, it might respond with text output even if configured for audio response. To work around this issue, use the model [with a separate TTS instance](#separate-tts) and text-only response modality. You can use the [Azure OpenAI TTS plugin](https://docs.livekit.io/agents/models/tts/azure-openai.md) to continue using the same voices supported by the Realtime API. For additional workaround options, see the OpenAI [thread](https://community.openai.com/t/trouble-loading-previous-messages-with-realtime-api) on this topic. ## Additional resources The following resources provide more information about using Azure OpenAI with LiveKit Agents. - **[Azure OpenAI docs](https://learn.microsoft.com/en-us/azure/ai-services/openai/)**: Azure OpenAI service documentation. - **[Voice AI quickstart](https://docs.livekit.io/agents/start/voice-ai.md)**: Get started with LiveKit Agents and Azure OpenAI. - **[Azure ecosystem overview](https://docs.livekit.io/agents/integrations/azure.md)**: Overview of the entire Azure AI ecosystem and LiveKit Agents integration. --- This document was rendered at 2026-06-07T11:35:51.036Z. For the latest version of this document, see [https://docs.livekit.io/agents/models/realtime/plugins/azure-openai.md](https://docs.livekit.io/agents/models/realtime/plugins/azure-openai.md). To explore all LiveKit documentation, see [llms.txt](https://docs.livekit.io/llms.txt).