Soniox STT plugin guide

Overview

This plugin allows you to use Soniox  as an STT provider for your voice agents.

Installation

Install the plugin:

uv add "livekit-agents[soniox]~=1.5"

pnpm add @livekit/agents-plugin-soniox@1.x

Authentication

The Soniox plugin requires an API key from the Soniox console .

Set SONIOX_API_KEY in your .env file.

Usage

Use Soniox STT in an AgentSession or as a standalone transcription service. For example, you can use this STT in the Voice AI quickstart.

Configure Soniox STT options when you create the STT instance:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v4",
         language_hints=["en"]
      )
   ),
# ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      languageHints: ['en'],
   }),
   // ... llm, tts, etc.
});

Speaker diarization

You can enable speaker diarization  so the STT assigns a speaker identifier to each word or segment. When enabled, each token includes a speaker field and the STT reports capabilities.diarization=True.

The following example enables speaker diarization:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v4",
         language_hints=["en"],
         enable_speaker_diarization=True,
      )
   ),
# ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      languageHints: ['en'],
      enableSpeakerDiarization: true,
   }),
   // ... llm, tts, etc.
});

You can use MultiSpeakerAdapter to detect the primary speaker and format the transcripts by speaker. To learn more, see Speaker diarization and primary speaker detection.

Realtime translation

To use realtime translation , pass a TranslationConfig to STTOptions. Soniox supports two translation modes: one-way and two-way.

One-way translation

To translate from any detected language into a single target language, set type to "one_way" and specify the target language. For example, to translate any spoken language into English:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v4",
         translation=soniox.TranslationConfig(
            type="one_way",
            target_language="en",
         ),
      )
   ),
   # ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      translation: {
         type: 'one_way',
         targetLanguage: 'en',
      },
   }),
   // ... llm, tts, etc.
});

Two-way translation

To translate back and forth between two languages, set type to "two_way" and specify the two languages. For example, to translate between English and Spanish:

from livekit.plugins import soniox

session = AgentSession(
   stt=soniox.STT(
      params=soniox.STTOptions(
         model="stt-rt-v4",
         translation=soniox.TranslationConfig(
            type="two_way",
            language_a="en",
            language_b="es",
         ),
      )
   ),
   # ... llm, tts, etc.
)

import { voice } from '@livekit/agents';
import * as soniox from '@livekit/agents-plugin-soniox';

const session = new voice.AgentSession({
   stt: new soniox.STT({
      model: 'stt-rt-v4',
      translation: {
         type: 'two_way',
         languageA: 'en',
         languageB: 'es',
      },
   }),
   // ... llm, tts, etc.
});

When translation is active, the first SpeechData in the alternatives list of each SpeechEvent (alternatives[0]) contains the translated text in the text field, and language holds the dominant target language.

Per-language segments

SpeechData splits the transcript by language. source_languages lists the language of each spoken segment and source_texts holds the matching text. In translation mode, target_languages and target_texts do the same for the translated output. In Node.js, the fields are sourceLanguages, sourceTexts, targetLanguages, and targetTexts.

For example, two-way English and Spanish translation of the spoken phrase "No hablo español, but I speak English." produces:

text = "I don't speak Spanish, pero hablo inglés."
source_languages = ["es", "en"]
source_texts = ["No hablo español, ", "but I speak English."]
target_languages = ["en", "es"]
target_texts = ["I don't speak Spanish, ", "pero hablo inglés."]

Outside translation mode, source_languages and source_texts identify the language of each segment in code-switched audio.

Parameters

This section describes some of the available options. See the STTOptions reference for a complete list.

In Python, pass these as an STTOptions object via the params argument. In Node.js, pass them directly to the STT constructor using their camelCase names (for example, languageHints, enableSpeakerDiarization).

Additional resources

The following resources provide more information about using Soniox with LiveKit Agents.

Python package

The livekit-plugins-soniox package on PyPI.

Node.js package

The @livekit/agents-plugin-soniox package on npm.

Plugin reference

Reference for the Soniox STT plugin.

GitHub repo

View the source or contribute to the LiveKit Soniox STT plugin.

Soniox docs

Soniox's full docs site.

Voice AI quickstart

Get started with LiveKit Agents and Soniox.