OpenAI API compatible LLMs

Customize the LLM for your voice agent

The OpenAI plugin provides methods that allow you to use OpenAI API compatible LLMs. In most cases, you can call the method with no parameter values by setting the required environment variables and accepting default values. The minimal syntax for each method assumes the environment variables for required values (for example, API keys) are set.

Support for additional LLMs is available through other LiveKit plugins.

Use the VoicePipelineAgent class and the OpenAI plugin to specify the LLM. In this example, use Groq as the LLM:

1. Set the GROQ_API_KEY environment variable:

   export GROQ_API_KEY=<your_groq_api_key>

   Copy

2. Create an agent:

   agent = VoicePipelineAgent(
      vad=ctx.proc.userdata["vad"],
      stt=deepgram.STT(),
      llm=openai.LLM.with_groq(),
      tts=cartesia.TTS(),
      chat_ctx=initial_ctx,
     )

   Copy

Supported LLMs

The OpenAI plugin offers support for the following LLMs:

Select an LLM in the dropdown menu to view parameters and syntax:

agent = VoicePipelineAgent(
     vad=ctx.proc.userdata["vad"],
     stt=deepgram.STT(),
     llm=openai.LLM.with_azure()
     tts=cartesia.TTS(),
     chat_ctx=initial_ctx,
    )