Azure AI Services and LiveKit

Azure AI ecosystem support

Microsoft's Azure AI Services is a large collection of cutting-edge production-ready AI services, which integrate with LiveKit in the following ways:

• Azure OpenAI: Run OpenAI models, including the Realtime API, with the security and reliability of Azure.
• Azure Speech: Speech-to-text and text-to-speech services.

The LiveKit Agents OpenAI plugin supports Azure OpenAI, and the Azure plugin supports Azure Speech.

Getting started

Use the voice AI quickstart to build a voice AI app with Azure OpenAI. Select a realtime model type and add the following components to use the Azure OpenAI Realtime API:

Install the OpenAI plugin:

pip install "livekit-agents[openai]~=1.0"

Add your Azure OpenAI endpoint and API key to your .env. file:

AZURE_OPENAI_ENDPOINT=<your-azure-openai-endpoint>
AZURE_OPENAI_API_KEY=<your-azure-openai-api-key>

Use the with_azure method to connect to Azure OpenAI:

from livekit.plugins import openai

# ...

# in your entrypoint function
session = AgentSession(
    llm=openai.realtime.RealtimeModel.with_azure(
        azure_deployment="<model-deployment>",
        api_version="2024-10-01-preview",
        voice="alloy",
    ),
    # ... vad, turn_detection, etc.
)

LiveKit Agents overview

LiveKit Agents is an open source framework for building realtime AI apps in Python and Node.js. It supports complex voice AI workflows with multiple agents and discrete processing steps, and includes built-in load balancing.

LiveKit provides SIP support for telephony integration and full-featured frontend SDKs in multiple languages. It uses WebRTC transport for end-user devices, enabling high-quality, low-latency realtime experiences. To learn more, see LiveKit Agents.

Azure plugin documentation