Scalable Video

Scalable video is a feature for routed sessions that improves video quality in multi-party sessions by letting each subscriber receive a video quality that matches their current network conditions, independently of every other subscriber.

Without scalable video, the OpenTok Media Router forwards the same video quality to every subscriber on a stream. When a subscriber's connection degrades, the router uses bandwidth estimation to tell the publisher to reduce its bitrate, which lowers the quality for all subscribers on that stream. With scalable video active, the router selects the most appropriate quality layer for each subscriber independently in real time, without affecting what other subscribers receive.

Scalable video requires a routed session (a session that uses the OpenTok Media Router). It is not available in relayed sessions and would not be useful there, since streams travel directly between clients with no router in the path to select or switch quality layers. See The OpenTok Media Router and media modes.

Scalable video is enabled by default. The project-level setting defaults to Auto, which means the OpenTok Media Router activates scalable video automatically when there are more than two clients in a session and all other conditions are met. You can also set it to On (always active) or Off. For most applications, no code changes are needed.

Codecs and Scalability Models

How scalable video works depends on the codec the publisher negotiates. Understanding the differences helps you choose the right codec for your use case, such as when you need to optimize for publisher CPU usage, upstream bandwidth, or subscriber-side quality adaptation.

VP8: Simulcast

With VP8, scalable video is implemented through simulcast. The publisher encodes and transmits multiple separate bitstreams at different resolutions and frame rates. For example, 1080p, 540p, and 270p, each with its own temporal (frame rate) layers. Each stream is self-contained and can be decoded on its own.

Because each stream is independent, the OpenTok Media Router can switch any subscriber to a different quality level with no re-encoding. This provides strong resilience to changing network conditions.

VP9: Scalable Video Coding (SVC)

With VP9, scalable video uses Scalable Video Coding (SVC). The publisher encodes a single bitstream that has multiple spatial (resolution) and temporal (frame rate) layers embedded within it. Each higher layer depends on the lower layers, so a decoder receiving only the base layer gets low quality, while one receiving all layers gets full quality, all from the same bitstream.

The publisher CPU cost for SVC encoding is higher than encoding a single stream, roughly comparable to VP8 simulcast, but SVC requires less upstream bandwidth because there is only one bitstream to upload instead of multiple independent streams. From the single encoded stream, the OpenTok Media Router can extract and forward the right subset of layers for each subscriber, providing efficient and flexible quality adaptation that is also more resilient to packet loss.

SVC requires the publisher endpoint and the OpenTok Media Router to both support VP9 SVC. Firefox supports VP9 but not SVC. A Firefox publisher sends VP9 without layers, so the OpenTok Media Router cannot adapt quality for that stream.

For a full explanation of SVC layers, scalability modes (L1T3, L2T3, L3T3), archiving behavior, and device support, see VP9 Scalable Video Coding for routed sessions.

Key Difference at a Glance

H.264: Scalable Video Not Supported

The H.264 codec is fully supported in the Vonage Video API for publishing and subscribing, but scalable video is not available for H.264 streams. The OpenTok Media Router cannot switch quality layers for H.264 streams, and scalable video settings have no effect when H.264 is negotiated. If your application requires scalable video, use VP8 or VP9 instead. See Video codecs.

Scalable Video Support

Scalable video is supported for VP8 simulcast and VP9 SVC. It is not supported for H.264 streams. It is only available in routed sessions.

The following clients support scalable video:

• Web SDK - Chrome, Firefox, Safari, Samsung Internet, WebView Android, WebView on iOS, and Chromium-based Edge. Note: Firefox supports VP8 simulcast but not VP9 SVC.
• Android SDK on supported devices
• iOS SDK on supported devices
• Windows SDK
• Linux SDK
• macOS SDK
• React Native SDK on supported devices

For VP9 SVC device and browser compatibility details, see VP9 Scalable Video Coding for routed sessions.

How to Use Scalable Video

Project-Level Setting

Scalable video has three modes you can set for a project in your Video API account:

1. Go to your Video API account and select the project from the list of projects in the left-hand menu.
2. Under Project settings, find Scalable Video and select the setting for the project:
   • Auto (recommended) - The OpenTok Media Router enables scalable video when there are more than two clients in a session. Leave this selected unless you have a specific reason to override it.
   • On - Scalable video is always enabled (in supported clients) for all sessions in this project.
   • Off - Scalable video is disabled for all sessions in this project. Use this to lock the resolution and frame rate, or to reduce publisher CPU and bandwidth usage.
3. Click Save.

Scalable Screen-Sharing Streams

By default, scalable video is disabled for screen-sharing streams and enabled for camera and custom video source streams. Screen-sharing content typically changes less frequently than camera video, so the additional encoding cost of scalable video is usually unnecessary. However, enabling it can be beneficial in sessions where screen-share subscribers have varying network conditions. You can override the default per publisher:

How the Content Hint Affects Screen-Sharing Layers

When scalable video is enabled for a VP8 screen-sharing stream in a routed session, the video content hint you set on the publisher determines how the publisher structures the simulcast layers it sends. This is why the number of streams (SSRCs) you observe for a screen-sharing publisher in a tool such as chrome://webrtc-internals depends on the content hint:

• detail or text — The publisher optimizes for preserving fine detail and readability (text, line art, static content). It sends two streams at the same spatial resolution that differ only in frame rate: a full-resolution stream at the regular frame rate, and a second full-resolution stream at a lower frame rate. This keeps the shared content sharp and legible while still giving the OpenTok Media Router a lower-bitrate layer to fall back to when a subscriber's network degrades — so degradation reduces the frame rate rather than blurring the detail. Two layers at the full resolution give a good balance between resilience during degradation and maintaining the original detailed resolution.

• motion — The publisher optimizes for smooth movement (for example, sharing video playback). It behaves like camera simulcast, sending multiple streams at different spatial resolutions, so the OpenTok Media Router can drop a subscriber to a lower resolution to keep motion fluid under constrained network conditions.

Choose the content hint that matches your shared content: detail or text prioritizes keeping detail sharp (degrading frame rate first), while motion prioritizes smooth motion (degrading resolution first).

Setting Subscriber Preferred Frame Rate and Resolution

When a stream is published with scalable video, subscribers can signal a preferred quality to the OpenTok Media Router. The OpenTok Media Router selects the closest available layer that fits the subscriber's actual network conditions.

How to Verify Scalable Video Is Working

There is no single "scalable video active" flag in the SDK, but you can confirm it is operating using the following approaches.

Video Inspector

The Video Inspector tool in the Quality Metrics module displays the codec, resolution, and frame rate. Hover over any point on a plotted line to view the codec currently in use. When scalable video is enabled, the resolution and/or frame rate for one or more subscribers may adjust dynamically in response to changing network conditions.

WebRTC Stats

Each SDK exposes the underlying WebRTC stats report. On the publisher side, inspect RTCOutboundRtpStreamStats:

• With VP8 simulcast, you will see multiple ssrc entries with different frameWidth and frameHeight values, one per simulcast layer.
• With VP9 SVC, you will see a single ssrc with a scalabilityMode property set, for example L3T3.

SDK methods to access the stats report:

• Web SDK - Publisher.getRtcStatsReport() and Subscriber.getRtcStatsReport()
• Linux - see Getting stream stats

Checklist: Conditions Required for Scalable Video to Be Active

If you are not observing adaptive quality behavior, verify the following:

• The session is routed, not relayed.
• Scalable video is not set to Off at the project level.
• The negotiated codec is VP8 or VP9, not H.264.
• The publisher is running on a supported client.

Common Misconceptions and FAQs

Is Scalable Video the Same as VP9 SVC?

No. "Scalable video" is the name of the Vonage Video API feature. The technical mechanism depends on the codec:

• VP8 implements scalable video through simulcast. The publisher sends multiple independent streams.
• VP9 implements scalable video through SVC. The publisher sends one stream with embedded layers.

Both mechanisms let the OpenTok Media Router adapt the quality each subscriber receives. When documentation mentions "scalable video" without specifying a codec, it refers to the feature as a whole.

Does Setting a Preferred Codec Force Scalable Video On?

No. Selecting a preferred codec is independent of scalable video. Whether scalable video activates depends on all of the following:

1. The scalable video project setting (On, Off, or Auto).
2. The session being routed.
3. The negotiated codec supporting scalable video (VP8 or VP9, not H.264).
4. The publisher's client or browser supporting scalable video.

Why Is Subscriber.setPreferredResolution() Not Adapting Quality?

The subscriber-side setPreferredResolution() and setPreferredFrameRate() are hints to the OpenTok Media Router, not direct commands to the publisher's encoder. The OpenTok Media Router can only act on these hints when it is actively selecting between scalable video layers for that subscriber. If any of the following conditions is true, there are no layers to select from and the hints have no effect:

• Scalable video is disabled at the project level or on the publisher stream.
• The negotiated codec is H.264, which does not support scalable video.
• The publisher's client does not produce scalable layers. For example, a Firefox publisher sending VP9 without SVC layers.
• The publisher has a CPU or bandwidth constraint and is not sending higher layers for the router to select from.
• The session is relayed. In a relayed session, the OpenTok Media Router does not forward streams, so layer selection cannot occur. Note that in relayed (P2P) sessions, the publisher's encoder adapts its output directly based on the subscriber's network conditions, but this is not the same as scalable video layer selection.

Can I Mix Scalable and Non-Scalable Publishers in the Same Session?

Yes. Scalable video is determined per stream, not per session. A session can simultaneously have publishers using scalable video and publishers that are not, for example H.264 publishers, or clients publishing with scalable video disabled. The OpenTok Media Router handles each stream independently.

Does Scalable Video Affect Archiving?

For VP8 simulcast, the archiver records using the highest available quality layer.

For VP9 SVC, individual archives store each participant's stream as VP9 SVC-encoded WebM. Composed archives are always transcoded to H.264/AAC MP4 regardless of the session codec.

Playback of VP9 SVC WebM files may not work in all media players. For playback instructions and transcoding commands, see Notes on archiving with VP9 videos in the VP9 guide.

For more information, see this support article.