Azure AI Services and LiveKit

Azure AI ecosystem support

Microsoft's Azure AI Services is a large collection of cutting-edge production-ready AI services, which integrate with LiveKit in the following ways:

• Azure OpenAI: Run OpenAI models, including the Realtime API, with the security and reliability of Azure.
• Azure Speech: Speech-to-text and text-to-speech services.

LiveKit Inference and the LiveKit Agents OpenAI plugin support Azure OpenAI, and the Azure plugin supports Azure Speech.

Getting started

Use the voice AI quickstart to build a voice AI app with Azure OpenAI. Select a realtime model type and add the following components to use the Azure OpenAI Realtime API:

LiveKit Inference

Use an Azure OpenAI model in your agent session:

from livekit.agents import AgentSession, inference

session = AgentSession(
    llm=inference.LLM(
        model="openai/gpt-4.1-mini",
        provider="azure",
    ),
    # ... tts, stt, vad, turn_detection, etc.
)

import { AgentSession, inference } from '@livekit/agents';

session = new AgentSession({
    llm: new inference.LLM({
        model: "openai/gpt-4.1-mini",
        provider: "azure",
    }),
    // ... tts, stt, vad, turn_detection, etc.
});

Azure OpenAI Realtime API

Install the OpenAI plugin:

pip install "livekit-agents[openai]~=1.2"

Add your Azure OpenAI endpoint and API key to your .env. file:

AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint>
AZURE_OPENAI_API_KEY=<your-azure-openai-api-key>

Use the with_azure method to connect to Azure OpenAI:

from livekit.plugins import openai

# ...

# in your entrypoint function
session = AgentSession(
    llm=openai.realtime.RealtimeModel.with_azure(
        azure_deployment="<model-deployment>",
        api_version="2024-10-01-preview",
        voice="alloy",
    ),
    # ... vad, turn_detection, etc.
)

LiveKit Agents overview

LiveKit Agents is an open source framework for building realtime AI apps in Python and Node.js. It supports complex voice AI workflows with multiple agents and discrete processing steps, and includes built-in load balancing.

LiveKit provides SIP support for telephony integration and full-featured frontend SDKs in multiple languages. It uses WebRTC transport for end-user devices, enabling high-quality, low-latency realtime experiences. To learn more, see LiveKit Agents.

Azure plugin documentation