Skip to main content
IntroductionBuild AgentsAgent FrontendsTelephonyRoboticsWebRTC TransportManage & DeployReferences
XYouTubeSlack

Sarvam STT plugin guide

How to use the Sarvam STT plugin for LiveKit Agents.

On this page

OverviewAuthenticationInstallationRealtime APILegacy APITroubleshootingAdditional resources
Available inPython
|
Node.js

Overview

Use the Sarvam STT plugin to add speech recognition for Indian languages, English, and code-mixed audio to your LiveKit Agents. It fits voice agents that need broad Indic coverage with low-latency transcription, plus the option to translate, transliterate, output verbatim text, or return code-mixed transcripts.

The plugin exposes one class for each of Sarvam's two speech-to-text APIs:

  • STTRealtime uses Sarvam's realtime API. Start here for new Python agents.
  • STT uses Sarvam's legacy API. This is the only path available in Node.js.

Authentication

The Sarvam plugin requires a Sarvam API key .

Set SARVAM_API_KEY in your .env file:

SARVAM_API_KEY=<your-sarvam-api-key>

Installation

Install the plugin:

uv add "livekit-agents[sarvam]~=1.7"
pnpm add @livekit/agents-plugin-sarvam@1.x

Realtime API

Only Available inPython

STTRealtime connects to Sarvam's realtime API, which requires a Sarvam subscription that includes realtime streaming access. It streams partial transcripts and uses Sarvam's own VAD for turn detection. The model is always saaras:v3-realtime and isn't configurable.

Usage

Use STTRealtime in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

from livekit.agents import AgentSession
from livekit.plugins import sarvam


session = AgentSession(
   stt=sarvam.STTRealtime(
      language="en-IN",
      stream_type="balanced",
   ),
   # ... llm, tts, etc.
)

With the default vad endpointing, Sarvam detects turns and the plugin forwards its VAD events. Set endpointing to manual to delimit turns from your application instead. The plugin then emits START_OF_SPEECH on the first audio frame of a turn and END_OF_SPEECH when you flush the stream.

Realtime streams don't reconnect after a socket failure, because Sarvam bills per connection. Create a new stream or restart the session if the connection drops.

Parameter reference

This section describes commonly used parameters. See the plugin reference links in the Additional resources section for a complete list of all available parameters.

languagestringDefault: en-IN

A supported BCP-47 language code, or auto for adaptive language identification. Valid values are as-IN, auto, bn-IN, brx-IN, doi-IN, en-IN, gu-IN, hi-IN, kn-IN, kok-IN, ks-IN, mai-IN, ml-IN, mni-IN, mr-IN, ne-IN, or-IN, pa-IN, sa-IN, sat-IN, sd-IN, ta-IN, te-IN, and ur-IN.

Odia is or-IN on this API, not od-IN.

stream_typestringDefault: balanced

Latency profile for the stream. One of fast (lowest-latency partial transcripts), balanced (partial transcripts at a moderate cadence), or simulated (final transcripts only, with no partials).

modestringDefault: transcribe

The task applied to final transcripts. Valid values are transcribe, translate, verbatim, translit, and codemix. See the legacy mode parameter for what each value does.

endpointingstringDefault: vad

How turn boundaries are determined. One of vad (Sarvam detects turns and sends VAD events) or manual (your application delimits turns by flushing the stream).

promptstring

Terminology or context hint used to bias decoding.

vad_sot_thresholdfloat

Speech activation threshold, from 0.0 to 1.0. Applies only when endpointing is vad. If unset, Sarvam applies its own default.

vad_min_speech_msinteger

Minimum speech duration, in milliseconds, before a turn opens. Applies only when endpointing is vad. If unset, Sarvam applies its own default.

vad_min_silence_msinteger

End-of-turn silence, in milliseconds. Applies only when endpointing is vad. If unset, Sarvam applies its own default.

Legacy API

STT connects to Sarvam's legacy speech-to-text API. It's the only path available in Node.js, and remains supported in Python for existing integrations.

Usage

Use STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

Set language and model explicitly rather than relying on the defaults. The default model differs between the two SDKs: use saaras:v4 in Python and saaras:v3 in Node.js.

from livekit.agents import AgentSession
from livekit.plugins import sarvam


session = AgentSession(
   stt=sarvam.STT(
      language="en-IN",
      model="saaras:v4",
      mode="transcribe",  # default
      sample_rate=16000,
      high_vad_sensitivity=True,
   ),
   # ... llm, tts, etc.
)
import { voice } from '@livekit/agents';
import * as sarvam from '@livekit/agents-plugin-sarvam';


const session = new voice.AgentSession({
    stt: new sarvam.STT({
        languageCode: "en-IN",
        model: "saaras:v3",  // default
        mode: "transcribe",  // default
        highVadSensitivity: true,
    }),
    // ... llm, tts, etc.
});

Parameter reference

This section describes commonly used parameters. See the plugin reference links in the Additional resources section for a complete list of all available parameters.

languageLanguageCodeDefault: en-IN

Language code for the input audio. Valid values are as-IN, bn-IN, brx-IN, doi-IN, en-IN, gu-IN, hi-IN, kn-IN, kok-IN, ks-IN, mai-IN, ml-IN, mni-IN, mr-IN, ne-IN, od-IN, pa-IN, sa-IN, sat-IN, sd-IN, ta-IN, te-IN, unknown, and ur-IN.

Odia is od-IN on this API, not or-IN.

In Node.js this parameter is called languageCode.

modelstring

The Sarvam STT model to use. Python supports saaras:v4 (the default) and saaras:v3. Node.js supports saaras:v3 (the default) but not saaras:v4.

saaras:v4 is the latest model.

modestringDefault: transcribe

The transcription mode. One of transcribe (a standard transcription in the source language), translate (translate the spoken input), verbatim (preserve more of the speaker's exact wording), translit (transliterated output), or codemix (optimized for code-mixed speech).

sample_rateintegerDefault: 16000
Only Available inPython

Input audio sample rate used for streaming sessions. Must be greater than 0.

high_vad_sensitivityboolean

Enables Sarvam's high VAD sensitivity option for streaming transcription. Use it if your agent needs to detect softer or shorter utterances.

In Node.js this parameter is called highVadSensitivity.

flush_signalboolean

Sends Sarvam's flush_signal streaming option when set.

In Node.js this parameter is called flushSignal.

Sarvam's fine-grained VAD options, such as positive_speech_threshold and min_speech_frames, are also available. Tune them only after validating the default behavior with your target microphone, room, telephony, or browser audio path. See the plugin reference links in the Additional resources section for the full list.

Troubleshooting

The following sections include common issues and their solutions.

No or delayed transcripts

Check the audio path first:

  • Confirm that the LiveKit participant is publishing audio.
  • Confirm that the agent session is using Sarvam as the configured stt provider.
  • Try disabling custom VAD options and retest with the defaults.
  • On the realtime API, create a new stream or restart the session if the connection fails. The plugin doesn't reconnect automatically.

Short utterances are missed

On the realtime API, lower vad_min_speech_ms. On the legacy API, enable high_vad_sensitivity. If you tune the fine-grained VAD options, change one value at a time and validate with representative audio.

Transcripts are in the wrong language or script

Set language explicitly instead of relying on the default. If your use case involves translation, transliteration, or code-mixed output, set the corresponding mode.

Additional resources

The following resources provide more information about using Sarvam with LiveKit Agents.

Python

Python

PyPIReferenceGitHub
Node.js

Node.js

NPMReferenceGitHub

Sarvam realtime STT WebSocket

Sarvam's realtime speech-to-text WebSocket API.

Sarvam legacy STT API reference

Sarvam's legacy speech-to-text WebSocket API.

Voice AI quickstart

Get started with LiveKit Agents and Sarvam.

Sarvam TTS

Guide to the Sarvam TTS plugin with LiveKit Agents.