Background audio

Overview

To add more realism to your agent, or add additional sound effects, publish background audio. This audio is played on a separate audio track. The BackgroundAudioPlayer class supports on-demand playback of custom audio as well as automatic ambient and thinking sounds synchronized to the agent lifecycle.

For a complete example, see the following recipes:

A voice AI agent with background audio for thinking states and ambiance.

Background audio example in Node.js

A voice AI agent with background audio for ambiance.

Create the player

The BackgroundAudioPlayer class manages audio playback to a room. It can also play ambient and thinking sounds automatically during the lifecycle of the agent session, if desired.

ambient_soundAudioSource | AudioConfig | list[AudioConfig]

Ambient sound plays on a loop in the background during the agent session. See Supported audio sources and Multiple audio clips for more details.

thinking_soundAudioSource | AudioConfig | list[AudioConfig]

Thinking sound plays while the agent is in the "thinking" state. See Supported audio sources and Multiple audio clips for more details.

Create the player within your entrypoint function:

from livekit.agents import BackgroundAudioPlayer, AudioConfig, BuiltinAudioClip

# An audio player with automated ambient and thinking sounds
background_audio = BackgroundAudioPlayer(
    ambient_sound=AudioConfig(BuiltinAudioClip.OFFICE_AMBIENCE, volume=0.8),
    thinking_sound=[
        AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING, volume=0.8),
        AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING2, volume=0.7),
    ],
)

# An audio player with a custom ambient sound played on a loop
background_audio = BackgroundAudioPlayer(
    ambient_sound="/path/to/my-custom-sound.mp3",
)

# An audio player for on-demand playback only
background_audio = BackgroundAudioPlayer()

import { voice } from '@livekit/agents';

const backgroundAudio = new voice.BackgroundAudioPlayer({
    ambientSound: {
            source: voice.BuiltinAudioClip.OFFICE_AMBIENCE,
            volume: 0.8,
    },
    thinkingSound: [
        { source: voice.BuiltinAudioClip.KEYBOARD_TYPING, volume: 0.8 },
        { source: voice.BuiltinAudioClip.KEYBOARD_TYPING2, volume: 0.7 },
    ],
});

// An audio player with a custom ambient sound played on a loop
const backgroundAudio2 = new voice.BackgroundAudioPlayer({
    ambientSound: "/path/to/my-custom-sound.mp3",
});

// An audio player for on-demand playback only
const backgroundAudio3 = new voice.BackgroundAudioPlayer();

Start and stop the player

Call the start method after room connection and after starting the agent session. Pass the room and agent_session to start(). Ambient sounds, if any, begin playback immediately.

await background_audio.start(room=ctx.room, agent_session=session)

await backgroundAudio.start({ room: ctx.room, agentSession: session });

To stop and clean up the player, call the aclose (or close in Node.js) method. You must create a new player instance if you want to start again.

await background_audio.aclose()

await backgroundAudio.close();

Play audio on-demand

You can play audio at any time, after starting the player, with the play method.

audio

Required

AudioSource | AudioConfig | list[AudioConfig]

The audio source or a probabilistic list of sources to play. To learn more, see Supported audio sources and Multiple audio clips.

loopbooleanDefault: False

Set to True to continuously loop playback.

For example, if you created background_audio in the previous example, you can play an audio file like this:

background_audio.play("/path/to/my-custom-sound.mp3")

backgroundAudio.play("/path/to/my-custom-sound.mp3");

The play method returns a PlayHandle which you can use to await or cancel the playback.

The following example uses the handle to await playback completion:

# Wait for playback to complete
await background_audio.play("/path/to/my-custom-sound.mp3")

const handle = await backgroundAudio.play("/path/to/my-custom-sound.mp3");

The next example shows the handle's stop method, which stops playback early:

handle = background_audio.play("/path/to/my-custom-sound.mp3")
await(asyncio.sleep(1))
handle.stop() # Stop playback early

const handle = backgroundAudio.play("/path/to/my-custom-sound.mp3");
await new Promise(resolve => setTimeout(resolve, 1000));
handle.stop(); // Stop playback early

Multiple audio clips

You can pass a list of audio sources to any of play, ambient_sound, or thinking_sound. The player selects a single entry in the list based on the probability parameter. This is useful to avoid repetitive sound effects. To allow for the possibility of no audio at all, ensure the sum of the probabilities is less than 1.

AudioConfig has the following properties:

source

Required

AudioSource

The audio source to play. See Supported audio sources for more details.

volumefloatDefault: 1

The volume at which to play the given audio.

probabilityfloatDefault: 1

The relative probability of selecting this audio source from the list.

# Play the KEYBOARD_TYPING sound with an 80% probability and the KEYBOARD_TYPING2 sound with a 20% probability
background_audio.play([
    AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING, volume=0.8, probability=0.8),
    AudioConfig(BuiltinAudioClip.KEYBOARD_TYPING2, volume=0.7, probability=0.2),
])

// Play the KEYBOARD_TYPING sound with an 80% probability and the KEYBOARD_TYPING2 sound with a 20% probability
backgroundAudio.play([
    { source: voice.BuiltinAudioClip.KEYBOARD_TYPING, volume: 0.8, probability: 0.8 },
    { source: voice.BuiltinAudioClip.KEYBOARD_TYPING2, volume: 0.7, probability: 0.2 },
])

Supported audio sources

The following audio sources are supported:

Local audio file

Pass a string path to any local audio file. The player decodes files with FFmpeg via PyAV and supports all common audio formats including MP3, WAV, AAC, FLAC, OGG, Opus, WebM, and MP4.

WAV files

The player uses an optimized custom decoder to load WAV data directly to audio frames, without the overhead of FFmpeg. For small files, WAV is the highest-efficiency option.

Built-in audio clips

The following built-in audio clips are available by default for common sound effects:

BuiltinAudioClip.OFFICE_AMBIENCE: Chatter and general background noise of a busy office.
BuiltinAudioClip.KEYBOARD_TYPING: The sound of an operator typing on a keyboard, close to their microphone.
BuiltinAudioClip.KEYBOARD_TYPING2: A shorter version of KEYBOARD_TYPING.

Raw audio frames

Pass an AsyncIterator[rtc.AudioFrame] to play raw audio frames from any source.

Additional resources

Background audio

A voice AI agent with background audio for thinking states and ambiance.

Background audio example in Node.js

A voice AI agent with background audio for ambiance.

Speech & audio overview

Control agent speech, handle interruptions, and initiate speech.