Benchmarking

Measuring performance

LiveKit can scale to many simulteneous rooms by running a distributed setup across multiple nodes. However, each room must fit within a single node. For this reason, benchmarks below will be focused on stressing the number of concurrent users in a room.

With WebRTC SFUs, a few factors determine the amount of work a server could perform:

  • Number of tracks published
  • Number of subscribers
  • Amount of data sent to each subscriber

An SFU needs to perform work to receive every track - this means receiving tens of packets per second. It then needs to forward that received data to every subscriber. That adds up to a significant amount of work in decryption and encryption, packet processing, and data forwarding.

Due to these variations, it can be difficult to understand the capacity of the SFU for an specific application. We provide tooling that help with simulating workload according to your specifications.

Load testing

The LiveKit CLI includes the lk load-test subcommand, which can simulate real-world loading conditions for various scenarios. It uses the Go SDK to simulate publishers and subscribers in a room.

When publishing, it could send both video and audio tracks:

  • video: looping video clips at 720p, with keyframes every ~3s (simulcast enabled)
  • audio: sends blank packets that aren't audible, but would simulate a target bitrate.

As a subscriber, it can simulate an application that takes advantage of adaptive stream, rendering a specified number of remote streams on-screen.

When benchmarking with the load tester, be sure to run it on a machine with plenty of CPU and bandwidth, and ensure it has sufficient file handles (ulimit -n 65535). You can also run the load tester from multiple machines.

caution:

Load testing traffic on your cloud instance will count toward your quotas, and is subject to the limits of your plan.

Benchmarks

We've run benchmarks for a few common scenarios to give a general understanding of performance. All benchmarks below are to demonstrate max number of participants supported in a single room.

All benchmarks were ran with the server running on a 16-core, compute optimized instance on Google Cloud. ( c2-standard-16)

In the tables below:

  • Pubs - Number of publishers
  • Subs - Number of subscribers

Audio only

This simulates an audio only experience with a large number of listeners in the room. It uses an average audio bitrate of 3kbps. In large audio sessions, only a small number of people are usually speaking (while everyone are on mute). We use 10 as the approximate number of speakers here.

Use casePubsSubsBytes/s in/outPackets/s in/outCPU utilization
Large audio rooms1030007.3 kBps / 23 MBps305 / 959,15680%

Command:

lk load-test \
--url <YOUR-SERVER-URL> \
--api-key <YOUR-KEY> \
--api-secret <YOUR-SECRET> \
--room load-test \
--audio-publishers 10 \
--subscribers 1000

Video room

Default video resolution of 720p was used in the load tests.

Use casePubsSubsBytes/s in/outPackets/s in/outCPU utilization
Large meeting15015050 MBps / 93 MBps51,068 / 762,74985%
Livestreaming13000233 kBps / 531 MBps246 / 560,96292%

To simulate large meeting:

lk load-test \
--url <YOUR-SERVER-URL> \
--api-key <YOUR-KEY> \
--api-secret <YOUR-SECRET> \
--room load-test \
--video-publishers 150 \
--subscribers 150

To simulate livestreaming:

lk load-test \
--url <YOUR-SERVER-URL> \
--api-key <YOUR-KEY> \
--api-secret <YOUR-SECRET> \
--room load-test \
--video-publishers 1 \
--subscribers 3000 \