Skip to main content

Enhanced noise cancellation

LiveKit Cloud offers AI-powered noise cancellation for realtime audio.

Overview

LiveKit Cloud includes advanced models licensed from Krisp to remove background noise and ensure the best possible audio quality. The models run locally, with no audio data sent to Krisp servers as part of this process and negligible impact on audio latency or quality.

The feature includes a background voice cancellation (BVC) model, which removes extra background speakers in addition to background noise, providing the best possible experience for voice AI applications. You can also use the standard NC model if desired.

The following comparison shows the effect of the models on the audio as perceived by a user, and also as perceived by a voice AI agent running an STT model (Deepgram Nova 3 in these samples). The segments marked with a strikethrough indicate unwanted content that would confuse the agent. These samples illustrate that BVC is necessary to achieve clean STT in noisy multi-speaker environments.

Try the free noise canceller tool with your LiveKit Cloud account to test your own audio samples.

Original Audio - Noisy Taxi Ride

Original Audio - Noisy Taxi Ride light waveform
STT by Deepgram Nova 3
Hi there, can you hear me alright? pretty bad Sorry it's pretty noisy in the taxi. sorry i can hear you pretty soon Okay so as I was saying I just heard about this platform called LiveKit - it's an all in one platform for voice AI agents with some pretty cool features. Have you heard about it?

With Noise Cancellation (NC)

With Noise Cancellation (NC) light waveform
STT by Deepgram Nova 3
Hi there, can you hear me alright? pretty bad Sorry it's pretty noisy in the taxi. sorry i'll get you there pretty soon Okay so as I was saying I just heard about this platform called LiveKit - it's an all in one platform for voice AI agents with some pretty cool features. Have you heard about it?

With Background Voice Cancellation (BVC)

With Background Voice Cancellation (BVC) light waveform
STT by Deepgram Nova 3
Hi there, can you hear me alright? Sorry it's pretty noisy in the taxi. Okay so as I was saying I just heard about this platform called LiveKit - it's an all in one platform for voice AI agents with some pretty cool features. Have you heard about it?

Supported platforms

You can apply the filter in the frontend ("outbound") with plugins for JavaScript, Swift, and Android, or directly inside of your agent code ("inbound"). The BVC model is available only within your agent, using the Python or Node.js plugins. LiveKit also offers an NC model for SIP-based telephony, which can be enabled with a flag in the trunk configuration.

The following table shows the support for each platform.

PlatformOutboundInboundBVCPackage
Web@livekit/krisp-noise-filter
SwiftLiveKitKrispNoiseFilter
Androidio.livekit:krisp-noise-filter
Flutterlivekit_noise_filter
React Native@livekit/react-native-krisp-noise-filter
UnityN/A
Pythonlivekit-plugins-noise-cancellation
Node.js@livekit/noise-cancellation-node
TelephonyLiveKit SIP documentation

Usage instructions

Use the following instructions to integrate the filter into your app, either inside of your agent code or in the frontend.

Tip

Leaving default settings on is strongly recommended. Learn more about these defaults in the Noise & echo cancellation docs.

Agent code ("inbound") implementation

The following examples show how to set up noise cancellation inside your agent code. This applies noise cancellation to inbound audio and is the recommended approach for most voice AI use cases.

Tip

When using noise or background voice cancellation in the agent code, do not enable Krisp noise cancellation in the frontend. Noise cancellation models are trained on raw audio and might produce unexpected results if the input has already been processed by Krisp in the frontend.

Standard noise cancellation and the separate echo cancellation feature can be left enabled.

Installation

Install the noise cancellation package from PyPI:

pip install "livekit-plugins-noise-cancellation~=0.2"

Usage in LiveKit Agents

Include the filter in RoomInputOptions when starting your AgentSession:

from livekit.plugins import noise_cancellation
# ...
await session.start(
# ...,
room_input_options=room_io.RoomInputOptions(
noise_cancellation=noise_cancellation.BVC(),
),
)
# ...
Agents v0.12 compatibility

In LiveKit Agents v0.12, pass the noise_cancellation parameter to the VoicePipelineAgent or MultimodalAgent constructor.

Usage with AudioStream

Apply the filter to any individual inbound AudioStream:

stream = rtc.AudioStream.from_track(
track=track,
noise_cancellation=noise_cancellation.NC(),
)

Available models

There are three noise cancellation models available:

# Standard enhanced noise cancellation
noise_cancellation.NC()
# Background voice cancellation (NC + removes non-primary voices
# that would confuse transcription or turn detection)
noise_cancellation.BVC()
# Background voice cancellation optimized for telephony applications
noise_cancellation.BVCTelephony()

Frontend ("outbound") implementation

The following examples show how to set up noise cancellation in the frontend. This applies noise cancellation to outbound audio.

Tip

When using noise or background voice cancellation in the frontend, do not enable Krisp noise cancellation in the agent code.

Standard noise cancellation and the separate echo cancellation feature can be left enabled.

Installation

npm install @livekit/krisp-noise-filter

This package includes the Krisp SDK but not the models, which downloads at runtime to minimize the impact on your application's bundle size.

React components usage

LiveKit Components includes a convenient useKrispNoiseFilter hook to easily integrate Krisp into your React app:

import { useKrispNoiseFilter } from '@livekit/components-react/krisp';
function MyKrispSetting() {
const krisp = useKrispNoiseFilter();
return (
<input
type="checkbox"
onChange={(ev) => krisp.setNoiseFilterEnabled(ev.target.checked)}
checked={krisp.isNoiseFilterEnabled}
disabled={krisp.isNoiseFilterPending}
/>
);
}

Base JS SDK usage

For other frameworks or advanced use cases, use the KrispNoiseFilter class directly:

import { type LocalAudioTrack, Room, RoomEvent, Track } from 'livekit-client';
const room = new Room();
// We recommend a dynamic import to only load the required resources when you enable the plugin
const { KrispNoiseFilter } = await import('@livekit/krisp-noise-filter');
room.on(RoomEvent.LocalTrackPublished, async (trackPublication) => {
if (
trackPublication.source === Track.Source.Microphone &&
trackPublication.track instanceof LocalAudioTrack
) {
if (!isKrispNoiseFilterSupported()) {
console.warn('Krisp noise filter is currently not supported on this browser');
return;
}
// Once instantiated, the filter will begin initializing and will download additional resources
const krispProcessor = KrispNoiseFilter();
console.log('Enabling LiveKit Krisp noise filter');
await trackPublication.track.setProcessor(krispProcessor);
// To enable/disable the noise filter, use setEnabled()
await krispProcessor.setEnabled(true);
// To check the current status use:
// krispProcessor.isEnabled()
// To stop and dispose of the Krisp processor, simply call:
// await trackPublication.track.stopProcessor()
}
});

Available models

The JavaScript noise filter supports only the standard noise cancellation (NC) model.

Compatibility

Not all browsers support the underlying Krisp SDK (including Safari <17.4). Use isKrispNoiseFilterSupported() to check if the current browser is supported.