Gemini Live API

Overview

Google's Gemini Live API  enables low-latency, two-way interactions that use text, audio, and video input, with audio and text output. Gemini Live models are available in LiveKit Agents through LiveKit Inference and the Google plugin. With LiveKit Inference, you use your LiveKit Cloud credentials instead of a separate Google API key, and LiveKit Cloud bills you for usage. Use the plugin instead if you want to manage your own billing, use Vertex AI, or use a Gemini Live model that LiveKit Inference doesn't serve.

LiveKit Inference

Use LiveKit Inference to access Gemini Live models without a separate Google API key. The following table lists the Gemini Live models available through LiveKit Inference.

A checkmark in the EU endpoint column means the model has a dedicated EU endpoint, in addition to the global endpoint. To learn more, see Data residency.

Usage

Use the RealtimeModel class from the inference module to run Gemini Live models through LiveKit Inference. You can use this model in the Voice AI quickstart:

from livekit.agents import AgentSession, inference

session = AgentSession(
    llm=inference.RealtimeModel(model="google/gemini-3.8-live", voice="Puck"),
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
  llm: new inference.RealtimeModel({ model: 'google/gemini-3.8-live', voice: 'Puck' }),
});

Authentication uses your LiveKit Cloud credentials. Set LIVEKIT_API_KEY and LIVEKIT_API_SECRET in your .env file, or pass api_key and api_secret directly. In Node.js, these parameters are apiKey and apiSecret.

Parameters

The following are the main parameters for configuring Gemini Live models with LiveKit Inference. For a complete list, see the Python API reference.

String descriptors

As a shortcut, you can also pass the model ID directly to the llm argument in your AgentSession:

from livekit.agents import AgentSession

session = AgentSession(
    llm="google/gemini-3.8-live",
)

import { AgentSession } from '@livekit/agents';

const session = new AgentSession({
  llm: 'google/gemini-3.8-live',
});

Plugin

Use the Google plugin to connect directly to the Gemini Live API or Vertex AI with your own credentials. The plugin includes a RealtimeModel class for building voice agents.

Installation

Install the Google plugin:

uv add "livekit-agents[google]~=1.8"

pnpm add "@livekit/agents-plugin-google@1.x"

Authentication

The Google plugin requires authentication based on your chosen service:

• For Vertex AI, you must set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of the service account key file. For more information about mounting files as secrets when deploying to LiveKit Cloud, see File-mounted secrets.
• For the Google Gemini API, set the GOOGLE_API_KEY environment variable.

Usage

Use the Gemini Live API within an AgentSession. For example, you can use it in the Voice AI quickstart.

from livekit.plugins import google

session = AgentSession(
    llm=google.realtime.RealtimeModel(
        voice="Puck",
        temperature=0.8,
        instructions="You are a helpful assistant",
    ),
)

import * as google from '@livekit/agents-plugin-google';

const session = new voice.AgentSession({
   llm: new google.beta.realtime.RealtimeModel({
      model: "gemini-2.5-flash-native-audio-preview-12-2025",
      voice: "Puck",
      temperature: 0.8,
      instructions: "You are a helpful assistant",
   }),
});

Parameters

This section describes some of the available parameters. For a complete reference of all available parameters, see the plugin reference links in the Additional resources section.

Gemini 3.8

The gemini-3.8-live model is available on both the Gemini API and Vertex AI. To use it on Vertex AI, set vertexai to true and provide your Google Cloud project and location:

session = AgentSession(
    llm=google.realtime.RealtimeModel(
        model="gemini-3.8-live",
        vertexai=True,
        project="my-project",
        location="eu",
    ),
)

const session = new voice.AgentSession({
   llm: new google.beta.realtime.RealtimeModel({
      model: "gemini-3.8-live",
      vertexai: true,
      project: "my-project",
      location: "eu",
   }),
});

Gemini 3.1 compatibility

On current plugin versions, generate_reply(), update_instructions(), and update_chat_ctx() all work with 3.1 models, including agent handoffs that use session.update_agent() and multi-agent architectures that switch instructions mid-session. Basic voice conversations, tool calling, and audio I/O behave the same as on 2.5.

Other breaking changes

These changes also apply when migrating from Gemini 2.5 to 3.1:

• Affective dialog and proactive audio are not supported. Remove these options from your configuration.
• Asynchronous function calling is not supported. The model pauses and waits for your tool response before continuing.
• The thinkingConfig parameter uses thinkingLevel (options: "minimal", "low", "medium", "high") instead of thinkingBudget. The default is "minimal" for lowest latency.

Migrating from 2.5 to 3.1

To use the 3.1 model, update the model parameter:

session = AgentSession(
    llm=google.realtime.RealtimeModel(
        model="gemini-3.1-flash-live-preview",
        voice="Puck",
        instructions="You are a helpful assistant",
    ),
)

const session = new voice.AgentSession({
   llm: new google.beta.realtime.RealtimeModel({
      model: "gemini-3.1-flash-live-preview",
      voice: "Puck",
      instructions: "You are a helpful assistant",
   }),
});

Additional changes when migrating:

• Remove enable_affective_dialog and proactivity parameters if set.
• Replace thinkingBudget with thinkingLevel in your thinking_config.

For the full list of changes, see Google's migration guide .

Provider tools

See Gemini LLM provider tools for more information about tools that enable the model to use built-in capabilities executed on the model server.

Turn detection

The Gemini Live API includes built-in VAD-based turn detection, enabled by default. To use LiveKit's audio turn detector instead, configure the model to disable automatic activity detection. The audio turn detector operates directly on audio, so no separate STT model is required.

from google.genai import types
from livekit.agents import AgentSession, TurnHandlingOptions, inference

session = AgentSession(
   turn_handling=TurnHandlingOptions(
      turn_detection=inference.TurnDetector(),
   ),
   llm=google.realtime.RealtimeModel(
      realtime_input_config=types.RealtimeInputConfig(
         automatic_activity_detection=types.AutomaticActivityDetection(
            disabled=True,
         ),
      ),
   ),
)

import { inference } from '@livekit/agents';
import * as google from '@livekit/agents-plugin-google';

const session = new voice.AgentSession({
   llm: new google.beta.realtime.RealtimeModel({
      model: "gemini-2.5-flash-native-audio-preview-12-2025",
      realtimeInputConfig: {
         automaticActivityDetection: {
            disabled: true,
         },
      },
   }),
   turnHandling: {
      turnDetection: new inference.TurnDetector(),
   },
});

Thinking

Thinking lets the model reason internally before it responds. More thinking can improve responses to complex requests but adds latency. Configure it with the thinking_config parameter. The setting that controls how much the model thinks depends on the model:

The plugin doesn't include the model's thoughts in transcripts.

For more information, see Live API thinking  in the Gemini API docs.

Video input

The Gemini Live API supports live video input. When enabled, the agent receives frames from the user's camera or screen share and streams them natively within the Gemini realtime protocol alongside the audio session.

To enable video, set video_input=True in your RoomOptions:

class VideoAssistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a helpful voice assistant with live video input from your user.",
            llm=google.realtime.RealtimeModel(
                voice="Puck",
                temperature=0.8,
            ),
        )

server = AgentServer()

@server.rtc_session(agent_name="my-agent")
async def my_agent(ctx: JobContext):
    session = AgentSession()

    await session.start(
        agent=VideoAssistant(),
        room=ctx.room,
        room_options=room_io.RoomOptions(
            video_input=True,
        ),
    )

from livekit.agents import (
    Agent,
    AgentServer,
    AgentSession,
    JobContext,
    room_io,
)
from livekit.plugins import google

By default, the agent samples one frame per second while the user speaks and one frame every three seconds otherwise. To control the frame rate, pass a video_sampler to the AgentSession.

For more details on video input configuration, see Live video input.

Usage with separate TTS

You can combine Gemini Live API and a separate TTS instance to build a realtime hybrid architecture. This configuration allows you to gain the benefits of realtime speech comprehension while maintaining complete control over the speech output.

from google.genai.types import Modality

session = AgentSession(
    llm=google.realtime.RealtimeModel(modalities=[Modality.TEXT]),
    tts="fishaudio/s2.1-pro:fa4c9eb3dccc4806b382b40d61c6b10a",
)

import * as google from '@livekit/agents-plugin-google';

const session = new voice.AgentSession({
   llm: new google.beta.realtime.RealtimeModel({
      model: "gemini-2.5-flash-native-audio-preview-12-2025",
      modalities: [google.types.Modality.TEXT],
   }),
   tts: "fishaudio/s2.1-pro:fa4c9eb3dccc4806b382b40d61c6b10a",
});

Additional resources

The following resources provide more information about using Gemini with LiveKit Agents.

Python plugin

ReferenceGitHubPyPI

Node.js plugin

ReferenceGitHubNPM

Gemini docs

Gemini Live API documentation.

Voice AI quickstart

Get started with LiveKit Agents and Gemini Live API.

Google AI ecosystem guide

Overview of the entire Google AI and LiveKit Agents integration.

Gemini Live vision assistant

Build a vision-aware voice assistant with Gemini Live.