Soniox STT plugin guide

Overview

This plugin allows you to use Soniox  as an STT provider for your voice agents.

Installation

Install the plugin:

uv add "livekit-agents[soniox]~=1.5"

pnpm add @livekit/agents-plugin-soniox@1.x

Authentication

The Soniox plugin requires an API key from the Soniox console .

Set SONIOX_API_KEY in your .env file.

Usage

Use Soniox STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

Configure Soniox STT options when you create the STT instance:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v5",
         language_hints=["en"]
      )
   ),
# ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      languageHints: ['en'],
   }),
   // ... llm, tts, etc.
});

Speaker diarization

You can enable speaker diarization  so the STT assigns a speaker identifier to each word or segment. When enabled, each token includes a speaker field and the STT reports capabilities.diarization=True.

The following example enables speaker diarization:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v5",
         language_hints=["en"],
         enable_speaker_diarization=True,
      )
   ),
# ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      languageHints: ['en'],
      enableSpeakerDiarization: true,
   }),
   // ... llm, tts, etc.
});

You can use MultiSpeakerAdapter to detect the primary speaker and format the transcripts by speaker. To learn more, see Speaker diarization and primary speaker detection.

Realtime translation

To use realtime translation , pass a TranslationConfig to STTOptions. Soniox supports two translation modes: one-way and two-way.

One-way translation

To translate from any detected language into a single target language, set type to "one_way" and specify the target language. For example, to translate any spoken language into English:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v5",
         translation=soniox.TranslationConfig(
            type="one_way",
            target_language="en",
         ),
      )
   ),
   # ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      translation: {
         type: 'one_way',
         targetLanguage: 'en',
      },
   }),
   // ... llm, tts, etc.
});

Two-way translation

To translate back and forth between two languages, set type to "two_way" and specify the two languages. For example, to translate between English and Spanish:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v5",
         translation=soniox.TranslationConfig(
            type="two_way",
            language_a="en",
            language_b="es",
         ),
      )
   ),
   # ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      translation: {
         type: 'two_way',
         languageA: 'en',
         languageB: 'es',
      },
   }),
   // ... llm, tts, etc.
});

When translation is active, the first SpeechData in the alternatives list of each SpeechEvent (alternatives[0]) contains the translated text in the text field, and language holds the dominant target language.

Per-language segments

SpeechData splits the transcript by language. source_languages lists the language of each spoken segment and source_texts holds the matching text. In translation mode, target_languages and target_texts do the same for the translated output. In Node.js, the fields are sourceLanguages, sourceTexts, targetLanguages, and targetTexts.

For example, two-way English and Spanish translation of the spoken phrase "No hablo español, but I speak English." produces:

text = "I don't speak Spanish, pero hablo inglés."
source_languages = ["es", "en"]
source_texts = ["No hablo español, ", "but I speak English."]
target_languages = ["en", "es"]
target_texts = ["I don't speak Spanish, ", "pero hablo inglés."]

Outside translation mode, source_languages and source_texts identify the language of each segment in code-switched audio.

Endpoint detection

Soniox uses endpoint detection  to detect when a speech segment has ended and finalize its transcript. It identifies an endpoint as speech followed by enough silence. Finalizing sooner reduces latency, while waiting longer gives the model more context for accuracy. Three options let you tune this tradeoff:

• max_endpoint_delay_ms sets the maximum time Soniox waits after speech stops before it finalizes a segment.
• endpoint_sensitivity adjusts how readily the model commits an endpoint, where higher values finalize sooner.
• endpoint_latency_adjustment_level trades accuracy for lower endpoint latency, where higher values reduce latency.

endpoint_sensitivity and endpoint_latency_adjustment_level require the stt-rt-v5 model. Earlier models reject them. endpoint_sensitivity is available in Python only.

The following example tunes all three options. Set model to stt-rt-v5 because endpoint-related options require it:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v5",
         max_endpoint_delay_ms=2000,
         endpoint_sensitivity=0.5,
         endpoint_latency_adjustment_level=2,
      )
   ),
# ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v5',
      maxEndpointDelayMs: 2000,
      endpointLatencyAdjustmentLevel: 2,
   }),
   // ... llm, tts, etc.
});

Parameters

This section describes some of the available options. See the STTOptions reference for a complete list.

In Python, pass these as an STTOptions object via the params argument. In Node.js, pass them directly to the STT constructor using their camelCase names (for example, languageHints, enableSpeakerDiarization).

Additional resources

The following resources provide more information about using Soniox with LiveKit Agents.

Python package

The livekit-plugins-soniox package on PyPI.

Node.js package

The @livekit/agents-plugin-soniox package on npm.

Plugin reference

Reference for the Soniox STT plugin.

GitHub repo

View the source or contribute to the LiveKit Soniox STT plugin.

Soniox docs

Soniox's full docs site.

Voice AI quickstart

Get started with LiveKit Agents and Soniox.