Client / Receiving Tracks

Receiving media

On this page

Overview

Track subscription

Media playback

Speaker detection

Selective subscription

From client-side

From server-side

Adaptive stream

Enabling/disabling tracks

Simulcast controls

Overview

While connected to a room, a participant can subscribe to any tracks published to the room. When autoSubscribe is enabled (default), the server automatically subscribes new participants to every available track.

Track subscription

Receiving tracks from the server starts with a subscription.

As mentioned in our guide on publishing media, LiveKit models tracks with two constructs: TrackPublication and Track. Think of a TrackPublication as metadata for a track registered with the server and Track as the raw media stream. Track publications are always available to the client, even when the track is not subscribed to.

Track subscription callbacks provide your app with both the Track and TrackPublication objects.

Subscribed callback will be fired on both Room and RemoteParticipant objects.

import {
connect,
RoomEvent,
} from 'livekit-client';
room
.on(RoomEvent.TrackSubscribed, handleTrackSubscribed)
function handleTrackSubscribed(
track: RemoteTrack,
publication: RemoteTrackPublication,
participant: RemoteParticipant
) {
/* do things with track, publication or participant */
}

Media playback

Once subscribed to an audio or video track, it's ready for playback:

Note:

On some platforms (Swift, Android, and Flutter), once subscribed to an audio track, live playback begins automatically.

function handleTrackSubscribed(
track: RemoteTrack,
publication: RemoteTrackPublication,
participant: RemoteParticipant
) {
// attach track to a new HTMLVideoElement or HTMLAudioElement
const element = track.attach();
parentElement.appendChild(element);
// or attach to existing element
// track.attach(element)
}

Speaker detection

When audio tracks are published, LiveKit will detect participants whome are speaking. Speaker updates are sent for both local and remote participants. They will fire on both, Room and Participant objects.

room.on(RoomEvent.ActiveSpeakersChanged, (speakers: Participant[]) => {
// speakers contain all of the current active speakers
})
participant.on(ParticipantEvent.IsSpeakingChanged, (speaking: boolean) => {
console.log(`${participant.identity} is ${speaking ? "now" : "no longer"} speaking. audio level: ${participant.audioLevel}`)
})

Selective subscription

When autoSubscribe is disabled, LiveKit will rely on the client to decide which tracks it should subscribe to. This is more appropriate for spatial applications and/or applications that require precise control over what the clients are receiving.

Both client and server APIs are available to setup a connection for selective subscription and once configured, only explicitly subscribed tracks are sent down to the client.

From client-side

let room = await room.connect(url, token, {
autoSubscribe: false,
})
room.on(RoomEvent.TrackPublished, (track, publication, participant) => {
publication.setSubscribed(true)
})

From server-side

These controls are also available with the server APIs.

import { RoomServiceClient } from 'livekit-server-sdk';
const client = new RoomServiceClient("myhost", "api-key", "my secret")
// subscribe to new track
client.updateSubscriptions("myroom", "receiving-participant-identity", ["TR_TRACKID"], true)
// unsubscribe from existing track
client.updateSubscriptions("myroom", "receiving-participant-identity", ["TR_TRACKID"], false)

Adaptive stream

In an application, video elements where tracks are rendered could vary in size, and sometimes hidden. It would be extremely wasteful to fetch high-resolution videos but only to render it in a 150x150 box.

Adaptive stream allows a developer to build dynamic video applications without consternation for how interface design or user interaction might impact video quality. It allows us to fetch the minimum bits necessary for high-quality rendering and helps with scaling to very large sessions.

When adaptive stream is enabled, client SDK will monitor both size and visibility of the UI elements that the tracks are attached to. Then it'll automatically coordinate with the server to ensure the closest-matching simulcast layer that matches the UI element is sent back. If the element is hidden, the client informs the server, which pauses the associated track until the element's visibility is restored.

Note:

With JS SDK, you must use `Track.attach()` in order for adaptive stream to be effective.

Adaptive Stream

Enabling/disabling tracks

Client implementations seeking fine-grain control can enable or disable tracks at their discretion. This could be used to implement subscriber-side mute. (for example, muting a publisher in the room, but only for the current user).

When disabled, the client will not receive any new data for that track. If a disabled track is subsequently enabled, new data will be received by the client again.

The disable action is useful when optimzing for a client's bandwidth consumption. For example, if a particular user's video track is offscreen, disabling this track will reduce bytes from being sent by the LiveKit server until the track's data is needed again. (this is not needed with adaptive stream)

import {
connect,
RoomEvent,
} from 'livekit-client';
room.on(RoomEvent.TrackSubscribed, handleTrackSubscribed)
function handleTrackSubscribed(
track: RemoteTrack,
publication: RemoteTrackPublication,
participant: RemoteParticipant
) {
publication.setEnabled(false)
}

Note:

You may be wondering how `subscribe` and `unsubscribe` differs from `enable` and `disable`. A track must be subscribed to and enabled for data to be received by the client. If a track has not been subscribed to (or was unsubscribed) or disabled, the client performing these actions will not receive that track's data.

The difference between these two actions is *negotiation*. Subscribing requires a negotiation handshake with the LiveKit server, while enable/disable does not. Depending on one's use case, this can make enable/disable more efficient, especially when a track may be turned on or off frequently.

Simulcast controls

If a video track has simulcast enabled, a receiving client may want to manually specify the maximum receivable quality. This would result a quality and bandwidth reduction for the target track. This might come in handy, for instance, when an application's user interface is displaying a small thumbnail for a particular user's video track.

You can see how this is used in our React component.

import {
connect,
RoomEvent,
} from 'livekit-client';
connect('ws://your_host', token, {
audio: true,
video: true,
}).then((room) => {
room
.on(RoomEvent.TrackSubscribed, handleTrackSubscribed)
});
function handleTrackSubscribed(
track: RemoteTrack,
publication: RemoteTrackPublication,
participant: RemoteParticipant
) {
if (track.kind === Track.Kind.Video) {
publication.setVideoQuality(VideoQuality.LOW)
}
}