Gemini STT

Overview

Gemini speech-to-text is available in LiveKit Agents through LiveKit Inference and the Google plugin. With LiveKit Inference, your agent runs on LiveKit's infrastructure to minimize latency. No separate provider API key is required, and usage and rate limits are managed through LiveKit Cloud. Use the plugin instead if you want to manage your own billing and rate limits. Pricing for LiveKit Inference is available on the pricing page .

The two paths reach different Gemini APIs:

• LiveKit Inference serves Gemini Transcribe Live, a streaming transcription model on the Gemini Live API. It emits interim and final transcripts as the user speaks.
• The plugin transcribes with a Gemini multimodal model through generate_content. That API is non-streaming, so each transcript covers a complete segment of speech detected by a VAD.

For a realtime voice pipeline, prefer LiveKit Inference. Use the plugin when you need a specific Gemini model, your own Google credentials, or control over the transcription prompt.

LiveKit Inference

Use LiveKit Inference to access Gemini Transcribe Live without a separate Google API key.

Gemini Transcribe Live returns the transcript text and the recognized language. It doesn't return word timings, per-word confidence, or speaker labels.

Usage

To use Gemini Transcribe Live, use the STT class from the inference module:

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT(
        model="google/gemini-3.5-transcribe-live",
        language="en"
    ),
    # ... llm, tts, vad, turn_handling, etc.
)

import { AgentSession, inference } from '@livekit/agents';

const session = new AgentSession({
    stt: new inference.STT({
        model: "google/gemini-3.5-transcribe-live",
        language: "en"
    }),
    // ... llm, tts, vad, turnHandling, etc.
});

Parameters

Model parameters

Pass the following parameters inside extra_kwargs (Python) or modelOptions (Node.js):

The following example pins recognition to English and Spanish and biases it toward two product names:

from livekit.agents import AgentSession, inference

session = AgentSession(
    stt=inference.STT(
        model="google/gemini-3.5-transcribe-live",
        extra_kwargs={
            "language_codes": ["en-US", "es-ES"],
            "custom_vocabulary": ["LiveKit", "Agents Playground"],
        },
    ),
    # ... llm, tts, vad, turn_handling, etc.
)

Multilingual transcription

Gemini Transcribe Live identifies the spoken language on its own, and it continues to transcribe when the speaker changes language during the conversation. Leave both language and language_codes unset to use this behavior.

To limit detection to a known set of languages, pass language_codes in extra_kwargs. The model still detects the language, but only from the set you give it.

The model supports ar, cs, da, de, el, en, es, fi, fr, he, hi, id, it, ja, ko, nl, no, pl, pt, ru, sv, th, tr, uk, vi, and zh.

String descriptors

As a shortcut, you can also pass a model ID string directly to the stt argument in your AgentSession:

from livekit.agents import AgentSession

session = AgentSession(
    stt="google/gemini-3.5-transcribe-live:en",
    # ... llm, tts, vad, turn_handling, etc.
)

import { AgentSession } from '@livekit/agents';

const session = new AgentSession({
    stt: "google/gemini-3.5-transcribe-live:en",
    // ... llm, tts, vad, turnHandling, etc.
});

Regional availability

Gemini Transcribe Live is served from a single global deployment because Google publishes no regional Live API endpoints. A session that pins its inference request to a specific region doesn't route to this model.

Plugin

LiveKit's plugin support for Google lets you transcribe with a Gemini multimodal model using your own Google credentials.

Installation

Install the plugin from PyPI:

uv add "livekit-agents[google]~=1.5"

Authentication

Authenticate with one of the following methods:

• For the Gemini API, set the api_key argument or the GOOGLE_API_KEY environment variable.
• For Vertex AI, set vertexai=True and name a project with the project argument or the GOOGLE_CLOUD_PROJECT environment variable. Credentials come from GOOGLE_APPLICATION_CREDENTIALS or the credentials argument. For more information about mounting files as secrets when deploying to LiveKit Cloud, see File-mounted secrets.

Usage

Use Gemini STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

Gemini transcription is non-streaming, so the session needs a VAD to decide when a segment of speech is complete. AgentSession wraps the STT in StreamAdapter for you when a VAD is configured:

from livekit.agents import AgentSession, inference
from livekit.plugins import google

session = AgentSession(
    stt=google.STT(
        model="gemini-2.5-flash",
    ),
    vad=inference.VAD(model="silero"),
    # ... llm, tts, etc.
)

Without a VAD, the session raises an error. To use the STT outside of an AgentSession, wrap it in StreamAdapter yourself.

Parameters

This section describes the parameters that apply to Gemini transcription. See the plugin reference for a complete list of all available parameters.

Unsupported options

The Gemini backend shares the google.STT class with Google Cloud STT, but it doesn't accept the Cloud Speech recognition options. The plugin logs a warning and ignores adaptation, keywords, denoiser_config, endpointing_sensitivity, speech_start_timeout, and speech_end_timeout. Word time offsets and use_streaming are always off.

A single instance can't switch between the two backends. Passing a model from the other backend to update_options raises an error, so create a new STT instance instead.

Additional resources

The following resources provide more information about using Gemini STT with LiveKit Agents.

Python plugin

ReferenceGitHubPyPI

Gemini audio docs

Gemini Developer API docs for audio understanding.

Voice AI quickstart

Get started with LiveKit Agents and Gemini STT.

Google Cloud STT

Use Chirp and the other Google Cloud STT models instead.

Google ecosystem guide

Overview of the entire Google AI and LiveKit Agents integration.