Noise & echo cancellation

Overview

Your user's microphone is likely to pick up undesirable audio including background noise (like traffic, music, voices, etc) and may also pick up echoes from their own speakers. In both cases, this audio noise can reduce the quality of the audio heard by a remote participant. In AI voice applications, this can additionally interfere with and degrade the quality of transcription and turn detection, ultimately preventing an ideal user experience.

The best place to remove this audio is at the source, on the user's own device, before the audio ever hits the network. This ensures that every participant hears the exact same audio, and any post-processing can also be applied prior to any recording or egress.

LiveKit includes default client-side noise and echo cancellation based on the underlying open-source WebRTC implementations of echoCancellation and noiseSuppression. You can adjust these settings with the AudioCaptureOptions type in the LiveKit SDKs.

While those are suitable for many applications, you can achieve even cleaner results with AI-powered noise cancellation. LiveKit has partnered with Krisp to offer an enhanced noise cancellation feature to all LiveKit Cloud customers.

Comparison

Here's a comparison of the different noise cancellation options on an audio input with a large amount of noisy traffic in the background:

Original

WebRTC noiseSuppression

Krisp noise cancellation

Krisp noise cancellation

Krisp noise cancellation is supported for Web, iOS/macOS, and Android. Flutter and React Native support are coming soon.

Regardless of platform, you'll need install a separate package to enable the feature. This is because the Krisp SDK and models are fairly large (5-25MB combined depending on the platform).

LiveKit Cloud customers can enable Krisp in their application by following the instructions below.

npm install @livekit/krisp-noise-filter

import { useKrispNoiseFilter } from '@livekit/components-react/krisp';

function MyKrispSetting() {
  const krisp = useKrispNoiseFilter();
  return (
    <input
      type="checkbox"
      onChange={(ev) => krisp.setNoiseFilterEnabled(ev.target.checked)}
      checked={krisp.isNoiseFilterEnabled}
      disabled={krisp.isNoiseFilterPending}
    />
  );
}

import { type LocalAudioTrack, Room, RoomEvent, Track } from 'livekit-client';

const room = new Room();

// We recommend a dynamic import to only load the required resources when you enable the plugin
const { KrispNoiseFilter } = await import('@livekit/krisp-noise-filter');

room.on(RoomEvent.LocalTrackPublished, async (trackPublication) => {
  if (
    trackPublication.source === Track.Source.Microphone &&
    trackPublication.track instanceof LocalAudioTrack
  ) {
    if (!isKrispNoiseFilterSupported()) {
      console.warn('Krisp noise filter is currently not supported on this browser');
      return;
    }
    // Once instantiated, the filter will begin initializing and will download additional resources
    const krispProcessor = KrispNoiseFilter();
    console.log('Enabling LiveKit Krisp noise filter');
    await trackPublication.track.setProcessor(krispProcessor);

    // To enable/disable the noise filter, the recommended approach is:
    // await krispProcessor.setEnabled(enable)

    // To check the current status use:
    // krispProcessor.isEnabled()

    // To stop and dispose of the Krisp processor, simply call:
    // await trackPublication.track.stopProcessor()
  }
});