Skip to main content

Mistral TTS plugin guide

How to use the Mistral TTS plugin for LiveKit Agents.

On this page

Available inPython
|
Node.js

Overview

This plugin allows you to use Voxtral  as a TTS provider for your voice agents.

Installation

Install the LiveKit Mistral AI plugin from PyPI or npm:

uv add "livekit-agents[mistralai]~=1.8"
pnpm add @livekit/agents-plugin-mistralai

Authentication

The Mistral plugin requires a Mistral AI API key .

Set MISTRAL_API_KEY in your .env file.

Usage

Use Mistral TTS within an AgentSession or as a standalone speech generator. For example, you can use this TTS in the Voice AI quickstart.

from livekit.plugins import mistralai
session = AgentSession(
tts=mistralai.TTS(
model="voxtral-mini-tts-3",
voice="voice_447",
),
# ... llm, stt, etc.
)
import * as mistralai from '@livekit/agents-plugin-mistralai';
const session = new voice.AgentSession({
tts: new mistralai.TTS({
}),
// ... llm, stt, etc.
});

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

modelTTSModels | stringDefault: voxtral-mini-tts-3

Voxtral TTS model to use. To learn more, see TTS model options .

voiceTTSVoices | stringDefault: en_paul_neutral

Voxtral TTS voice to use. Voxtral 3 adds multilingual voices named voice_<n> (for example, voice_447 for an American English voice) and still accepts the older voices. Earlier models only accept the older voices. For the full list, see TTSVoxtral3Voices in the plugin source .

ref_audiostr

Base64-encoded audio sample (3–25 seconds) for zero-shot voice cloning. Mutually exclusive with voice.

response_formatRESPONSE_FORMATDefault: mp3

The audio format of synthesized speech: mp3, wav, pcm, opus, or flac. Ignored when use_realtime is enabled, which always streams PCM.

expressivenessfloat
Only Available inPython

Controls how expressive the speech is. Only supported by Voxtral 3.

Valid range: 1–5.

phoneme_hintsdict[str, list[str]]
Only Available inPython

Pronunciation hints that map a case-sensitive word to its ordered list of International Phonetic Alphabet (IPA) phones, for example {"tomato": ["t", "ə", "m", "ɑ", "t", "oʊ"]}. Only supported by Voxtral 3, and not with use_realtime.

use_realtimeboolDefault: False
Only Available inPython

Stream speech over the realtime websocket API while the LLM generates text, instead of synthesizing it sentence by sentence over HTTP. Only supported by Voxtral 3. To learn more, see Realtime streaming.

Realtime streaming

Available inPython

By default, the plugin synthesizes text over HTTP one sentence at a time. Speech starts as soon as the first sentence of the LLM reply is complete. Set use_realtime=True to stream text to Voxtral 3 over a websocket as the LLM generates it. The plugin keeps a pool of connections and reuses them across requests.

from livekit.plugins import mistralai
tts = mistralai.TTS(
model="voxtral-mini-tts-3",
use_realtime=True,
)

In realtime mode, audio is always PCM and phoneme_hints isn't supported. The plugin splits longer replies into multiple requests automatically. A single realtime request can't exceed about 3,500 characters.

Additional resources

The following resources provide more information about using Mistral with LiveKit Agents.