Baseten STT plugin guide | LiveKit Documentation

Available inPython

Overview

This plugin allows you to use Baseten as an STT provider for your voice agents.

Installation

Install the plugin from PyPI:

uv add "livekit-agents[baseten]~=1.6"

Authentication

The Baseten plugin requires a Baseten API key .

Set the following in your .env file:

BASETEN_API_KEY=<your-baseten-api-key>

Model deployment

You must deploy a websocket-based STT model to Baseten to use it with LiveKit Agents. The plugin supports two model families, selected with the model parameter:

`model`	Deployment	Notes
`whisper` (default)	Whisper Large V3 Turbo Streaming	Realtime streaming Whisper.
`qwen3-asr`	Qwen3-ASR 1.7B Streaming	Streaming Qwen3-ASR, with optional word-level alignment.

The two families use different wire protocols and aren't interchangeable, so set model to match the deployment you created. The standard, non-streaming Whisper deployments in the Baseten library aren't suitable for realtime use. Deploy one of the streaming models in the table.

Your model endpoint may show as an HTTP URL such as https://model-<id>.api.baseten.co/environments/production/predict. The domain is correct but you must change the protocol to wss and the path to /environments/production/websocket to use it as the model_endpoint parameter:

wss://model-<your-model-id>.api.baseten.co/environments/production/websocket

Note the model- prefix on the hostname. Instead of building the URL yourself, pass model_id (or chain_id for a chain deployment) and the plugin constructs it for you.

Usage

Use Baseten STT within an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

from livekit.plugins import baseten

session = AgentSession(
   # Streaming Whisper, the default model.
   stt=baseten.STT(model_id="<your-model-id>"),
   # ... llm, tts, etc.
)

To use Qwen3-ASR instead, set model:

stt=baseten.STT(model="qwen3-asr", model_id="<your-model-id>")

Detection across all supported languages is unreliable on the one- to two-second utterances typical of telephony. Restrict detection to the languages your agent supports instead. This needs Baseten Whisper runtime v0.5.0 or newer:

stt=baseten.STT(
   model_id="<your-model-id>",
   language="auto",
   language_options=["en", "de"],
)

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

modelSTTModelsDefault: whisper

Which deployment protocol to use: whisper or qwen3-asr. The two aren't interchangeable. See Model deployment.

model_endpointstringEnv: BASETEN_MODEL_ENDPOINT

The endpoint URL for your deployed model. You can find this in your Baseten dashboard. Note that this must be a websocket URL (starts with wss://). Takes priority over model_id and chain_id. See Model deployment for more details.

model_idstring

Baseten truss model ID. The plugin builds the endpoint URL for you. Ignored when model_endpoint is set.

chain_idstring

Baseten chain ID. The plugin builds the endpoint URL for you. Ignored when model_endpoint is set.

languageLanguageCode

Language code for the input audio. For whisper, this is a BCP-47 code and defaults to en; use auto for detection. For qwen3-asr, it's a Qwen3-ASR canonical language name such as English or Cantonese, or auto (the default). Forcing a language on qwen3-asr also sets the output language and can translate, so use it only to pin what's actually spoken.

language_optionslist[LanguageCode]Default: []

Restrict automatic detection to this set of language codes, such as ["en", "de"]. This is usually more accurate than auto. Requires Baseten Whisper runtime v0.5.0 or newer. Empty, the default, leaves detection unrestricted. whisper only.

enable_partial_transcriptsboolDefault: True

Emit interim transcripts while the speaker is still talking.

partial_transcript_interval_sfloat

Interval in seconds between partial transcript updates. Defaults to 1.0 for whisper and 0.5 for qwen3-asr, matching each deployment's own default.

final_transcript_max_duration_sintDefault: 30

Maximum seconds of audio before the server forces a final transcript.

show_word_timestampsbool

Include word-level timestamps. Defaults to on for whisper and off for qwen3-asr, whose aligner is opt-in on the deployment (STREAM_ALIGNER=mms).

encodingstringDefault: pcm_s16le

Audio encoding: pcm_s16le or pcm_mulaw. whisper only.

buffer_size_secondsfloatDefault: 0.032

Audio buffer size in seconds. whisper only.

sample_rateintDefault: 16000

Audio sample rate in Hz. whisper only. qwen3-asr always uses 16 kHz PCM16.

vad_thresholdfloatDefault: 0.5

Server-side voice activity detection threshold, from 0.0 to 1.0.

vad_min_silence_duration_msint

Minimum duration of silence in milliseconds to consider speech ended. Defaults to 300 for whisper and 500 for qwen3-asr.

vad_speech_pad_msint

Duration in milliseconds to pad speech segments. Defaults to 30 for whisper and 100 for qwen3-asr.

Additional resources

The following resources provide more information about using Baseten with LiveKit Agents.

Python package

The livekit-plugins-baseten package on PyPI.

Plugin reference

Reference for the Baseten STT plugin.

GitHub repo

View the source or contribute to the LiveKit Baseten STT plugin.

Baseten docs

Baseten's full docs site.

Voice AI quickstart

Get started with LiveKit Agents and Baseten.

Baseten TTS

Guide to the Baseten TTS plugin with LiveKit Agents.