Audio fallback

Use the audio fallback API to dynamically prioritize audio in response to network quality.

Audio fallback keeps your calls running for all participants, regardless of connection issues or poor network conditions by temporarily disabling video for the affected participant. The audio fallback capability reacts to network conditions in real time, ensuring participants won’t drop out when faced with poor coverage. When conditions improve, a video stream is reestablished through our video-recovery capability.

The publisher and subscriber audio fallback features combine to provide optimal call quality. A single subscriber can fall back to audio-only (if only the subscriber's conditions warrant it) or a publisher can fall back to audio-only for poor network conditions.

When the publisher and subscribe audio fallback features are enabled, the SDK dispatches events to indicate changing quality conditions. You can use these events to trigger notifications in the client, or the SDK can display the default UI notifications.

Subscriber audio fallback

The subscriber audio fallback feature improves call quality by automatically turning off incoming video video streams resulting in an audio-only mode in response to deteriorating network conditions on the subscriber-side. Other clients will continue to receive video regardless of the subscriber's network conditions. This feature is handled through the Media Router and is only available in routed sessions. See the session creation docs to understand the distinction between routed and relayed sessions.

Publisher audio fallback

The publisher audio fallback feature improves call quality by having a publisher switch to audio-only mode when the publishing client's network conditions cannot support video. The publisher will resume video when network conditions improve.

The publisher audio fallback feature is supported in both relayed and routed sessions (see The Media Router and media modes).

The publisher audio fallback features includes enhancements to monitor the bandwidth and congestion of a published stream. The publisher will prioritize audio communication and fallback to audio when the bandwidth congestion cannot support video communications.

With the publisher audio fallback feature, the publishing client receives quality feedback (on packet loss, bandwidth, and congestion) from the subscriber (in a relayed session) or the Media Router (in a routed session).

Enabling and disabling audio fallback

The client SDKs include events for publisher and subscriber audio fallback. These events indicate when the publisher or subscriber video is enabled or disabled, and when there is a warning for the video being disabled, due to audio fallback (and video recovery).

You can enable and disable audio fallback when you publish a stream:

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Note: The audioFallbackEnabled prop of the OTPublisher component will be deprecated. Please use the audioFallback.subscriber setting instead.

Enabling and disabling audio-only fallback

Set the audioFallback property of the properties prop you pass into the OTPublisher component:

The audioFallback object includes two Boolean properties:

publisher — Whether to enable (true) or disable (false) publisher audio fallback. With publisher audio fallback enabled, when the stream's quality has degraded significantly (for example, because of network conditions), the publisher disables video in order to preserve audio quality. The default is false (publisher audio fallback is disabled).
subscriber — Whether to enable (true) or disable (false) subscriber audio fallback. This setting only applies in routed sessions (sessions that use the Media Router). Subscriber audio fallback is not supported in relayed session. With subscriber audio fallback enabled, when the Media Router determines that a stream's quality has degraded significantly for a specific subscriber, it disables the video in that subscriber in order to preserve audio quality. The default is true (subscriber audio fallback is enabled). This setting replaces the the audioFallbackEnabled property, which will be deprecated.

Publisher audio fallback events

When publisher audio fallback is enabled, callback methods of the OTPublisher component are invoked in response to changing quality conditions:

videoDisableWarning() — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
videoDisableWarningLifted() — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
videoDisabled() — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
videoEnabled() — Called with the reason property set to 'quality' when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code adds event listeners for audio fallback-related events (so that you can provide user interface notifications):

Subscriber audio fallback events The OTSubscriber component includes callback methods that are invoked based on events related to the video being enabled or disabled for the subscriber's stream:

videoEnabled() — Called when the video has been enabled after it was previously disabled. videoDisabled() — Called when the video has been disabled. The reason property of the event object indicates why the video was disabled. (This event object is an VideoEnabledChangedEvent object.) videoDisableWarning() — Called when the Media Router determines that the stream quality has degraded and the video will be disabled if the quality degrades more. If the quality degrades further, the Subscriber disables the video and calls the videoDisabled() callback. This event may also be dispatched when using the publisher audio fallback feature if the publisher's stream quality if degraded. videoDisableWarningLifted() — Called when video has been enabled after it was previously disabled. The OTSubscriber videoDisableWarning() and videoDisableWarningLifted() callback methods are only invoked in sessions that use the Media Router (sessions with the media mode set to routed).

For example the following code adds event listeners for audio fallback-related events (so that you can provide user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Note: The audioFallbackEnabled property of the options passed into the OT.initPublisher() method will be deprecated. Please use the audioFallback.subscriber property of the options object instead.

Enabling and disabling audio-only fallback

When initializing the Publisher object, set the audioFallback property of the option you pass into the OT.initPublisher() method:

The audioFallback property of the options you pass into the OT.initPublisher() method includes two Boolean properties:

publisher — Whether to enable (true) or disable (false) publisher audio fallback. The default is false (publisher audio fallback is disabled).
subscriber — Whether to enable (true) or disable (false) subscriber audio fallback. This setting only applies in routed sessions (sessions that use the Media Router). Subscriber audio fallback is not supported in relayed session. The default is true (subscriber audio fallback is enabled). This setting replaces the the audioFallbackEnabled property, which will be deprecated.

Audio fallback events and UI indications

When publisher audio fallback is enabled, the Publisher object dispatches these events in response to changing quality conditions:

videoDisableWarning — Dispatched when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
videoDisableWarningLifted — Dispatched when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
videoDisabled — Dispatched when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
videoEnabled — Dispatched with reason: 'quality' when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled.

By default, the Publisher displays icons when the videoDisableWarning and videoDisabled events occur.

The style property of the options parameter for OT.initPublisher() now includes a videoDisabledDisplayMode property. You can set the videoDisabledDisplayMode property to one of the following string values control how the default user interface elements are displayed:

auto (the default) — The icons are automatically displayed when the video is disabled or in risk of being disabled due to poor stream quality.
off — The icons are not displayed. You can display your own user interface notifications based on the events described above.
on — The icons are automatically displayed when the video is disabled or in risk of being disabled due to poor stream quality.

For example the following code disables the default video disabled user interface elements, and handles the related events (so that you can provide your own user interface notifications):

You can also set the videoDisabledDisplayMode style dynamically by calling the Publisher.setStyle() method:

Subscriber audio fallback events and UI indicators

A Subscriber object dispatches the following events related to the video being enabled or disabled for the subscriber's stream:

videoEnabled — Dispatched when the video has been enabled after it was previously disabled.
videoDisabled — Dispatched when the video has been disabled. The reason property of the event object indicates why the video was disabled. (This event object is an VideoEnabledChangedEvent object.)
videoDisableWarning — Dispatched when the Media Router determines that the stream quality has degraded and the video will be disabled if the quality degrades more. If the quality degrades further, the Subscriber disables the video and dispatches a videoDisabled event. This event may also be dispatched when using the beta publisher audio fallback feature if the publisher's stream quality if degraded.
videoDisableWarningLifted — The video has been enabled after it was previously disabled.

The Subscriber videoDisableWarning and videoDisableWarningLifted events are only available in sessions that use the Media Router (sessions with the media mode set to routed).

By default, the Subscriber displays a video disabled warning indicator and a video disabled indicator when the videoDisableWarning and videoDisableWarningLifted events are dispatched. You can disable the default display of the indicator by setting the videoDisabledDisplayMode style setting of the Subscriber object.

The following example uses the videoDisabledDisplayMode style setting to have the video disabled warning indicator and a video disabled indicator blink every one second when the videoDisableWarning and videoDisableWarningLifted events are dispatched:

You can also set the videoDisabledDisplayMode style to 'off' and add your own user interface elements based on the videoDisableWarning, videoDisabled, videoDisableWarningLifted, and videoEnabled events.

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Notes: The PublisherKit.setAudioFallbackEnabled() and PublisherKit.setAudioFallbackEnabled() methods will be deprecated. Please use the PublisherKit.publisherAudioFallbackEnabled() method instead.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, call the PublisherKit.Builder.publisherAudioFallbackEnabled() function when creating the publisher.

To enable and disable subscriber audio fallback (for all subscribers to the stream), call the PublisherKit.Builder.subscriberAudioFallbackEnabled() function when creating the publisher. Subscriber audio fallback is only supported in routed sessions (sessions that use the Media Router). Subscriber audio fallback is enabled by default (in routed sessions) for streams with a camera video source.

Publisher audio fallback events

When publisher audio fallback is enabled, callback methods of the PublisherKit.VideoListener are invoked for publisher audio fallback-related events:

PublisherKit.VideoListener.onVideoDisableWarning() — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
PublisherKit.VideoListener.onVideoDisableWarningLifted() — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
PublisherKit.VideoListener.onVideoDisabled() — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI. PublisherKit.VideoListener.onVideoEnabled() — Called with reason "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled. For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, callback methods of the SubscriberKit.VideoListener are invoked for subscriber audio fallback-related events:

SubscriberKit.VideoListener.onVideoDisableWarning() — Called when the Subscriber determines that the stream quality has degraded and the video will be disabled if the quality degrades more. SubscriberKit.VideoListener.onVideoDisableWarningLifted() — Called when the Subscriber determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk. SubscriberKit.VideoListener.onVideoDisabled() — Called when the Subscriber determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Subscriber still displays the subscriber video (such as the camera image) in the publishing client's UI. SubscriberKit.VideoListener.onVideoEnabled() — Called with reason "quality" when the Subscriber determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Notes: The OTPublisherKit.audioFallbackEnabled property will be deprecated. Please use the OTPublisherKit.subscriberAudioFallbackEnabled property instead.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, set the OTPublisherKitSettings.publisherAudioFallbackEnabled property when creating the publisher.

To enable and disable subscriber audio fallback (for all subscribers to the stream), set the OTPublisherKitSettings.subscriberAudioFallbackEnabled property when creating the publisher. Subscriber audio fallback is only supported in routed sessions (sessions that use the Media Router). Subscriber audio fallback is enabled by default (in routed sessions) for streams with a camera video source.

Publisher audio fallback events

When publisher audio fallback is enabled, the PublisherKitDelegate object will send the following messages pertaining to publisher audio fallback-related events:

[OTPublisherKitDelegate publisherVideoDisableWarning:] — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
[OTPublisherKitDelegate publisherVideoDisableWarningLifted:] — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
[OTPublisherKitDelegate publisherVideoDisabled:reason:] — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
[OTPublisherKitDelegate publisherVideoEnabled:reason:] — Called with reason "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled. For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, the SubscriberKitDelegate object will send the following messages pertaining to subscriber audio fallback-related events:

[OTSubscriberKitDelegate subscriberVideoDisableWarning:] — Called when the Subscriber determines that the stream quality has degraded and the video will be disabled if the quality degrades more. [OTSubscriberKitDelegate subscriberVideoDisableWarningLifted:] — Called when the Subscriber determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk. [OTSubscriberKitDelegate subscriberVideoDisabled:reason:] — Called when the Subscriber determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Subscriber still displays the subscriber video (such as the camera image) in the publishing client's UI. [OTSubscriberKitDelegate subscriberVideoEnabled:reason:] — Called with reason "quality" when the Subscriber determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Notes: The OTPublisherKit.audioFallbackEnabled property will be deprecated. Please use the OTPublisherKit.subscriberAudioFallbackEnabled property instead.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, set the OTPublisherKitSettings.publisherAudioFallbackEnabled property when creating the publisher.

Publisher audio fallback events

When publisher audio fallback is enabled, the PublisherKitDelegate object will send the following messages pertaining to publisher audio fallback-related events:

[OTPublisherKitDelegate publisherVideoDisableWarning:] — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
[OTPublisherKitDelegate publisherVideoDisableWarningLifted:] — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
[OTPublisherKitDelegate publisherVideoDisabled:reason:] — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
[OTPublisherKitDelegate publisherVideoEnabled:reason:] — Called with reason "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled. For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, the SubscriberKitDelegate object will send the following messages pertaining to subscriber audio fallback-related events:

For example the following code handles the related events (so that you can provide your own user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, set the Publisher.Builder.PublisherAudioFallback property when creating a Publisher object.

To enable and disable subscriber audio fallback (for all subscribers to the stream), call the Publisher.Builder.SubscriberAudioFallback property when creating a Publisher object. Subscriber audio fallback is only supported in routed sessions (sessions that use the Media Router). Subscriber audio fallback is enabled by default (in routed sessions) for streams with a camera video source.

Publisher audio fallback events

When publisher audio fallback is enabled, the Publisher object dispatches audio fallback-related events:

Publisher.VideoDisableWarning — Sent when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more. Publisher.VideoDisableWarningLifted — Sent when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
Publisher.VideoDisabled — Sent when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
Publisher.VideoEnabled — Sent with reason set to "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, the Subscriber object dispatches subscriber audio fallback-related events:

Subscriber.VideoDisableWarning — Sent when the Subscriber determines that the stream quality has degraded and the video will be disabled if the quality degrades more. Subscriber.VideoDisableWarningLifted — Sent when the Subscriber determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
Subscriber.VideoDisabled — Sent when the Subscriber determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Subscriber still displays the subscriber video (such as the camera image) in the publishing client's UI.
Subscriber.VideoEnabled — Sent with reason set to "quality" when the Subscriber determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Notes: The otc_publisher_set_audio_fallback_enabled() and otc_publisher_get_audio_fallback_enabled() functions will be deprecated. Please use the otc_publisher_settings_set_publisher_audio_fallback_enabled instead.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, call the otc_publisher_settings_set_publisher_audio_fallback_enabled() function.

To enable and disable subscriber audio fallback (for all subscribers to the stream), call the otc_publisher_settings_set_subscriber_audio_fallback_enabled() function. Subscriber audio fallback is only supported in routed sessions (sessions that use the Media Router). Subscriber audio fallback is enabled by default (in routed sessions) for streams with a camera video source.

Publisher audio fallback events

When publisher audio fallback is enabled, callback functions in the otc_publisher_callbacks struct are invoked for publisher audio fallback-related events:

otc_publisher_callbacks.on_video_disable_warning() — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
otc_publisher_callbacks.on_video_disable_warning_lifted() — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
otc_publisher_callbacks.on_video_disabled() — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
otc_publisher_callbacks.on_video_enabled() — Called with reason "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled. For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, callback functions in the otc_subscriber_callbacks struct are invoked for subscriber audio fallback-related events:

otc_subscriber_callbacks.on_video_disable_warning() — Called when the Subscriber determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
otc_subscriber_callbacks.on_video_disable_warning_lifted() — Called when the Subscriber determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
otc_subscriber_callbacks.on_video_disabled() — Called when the Subscriber determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Subscriber still displays the subscriber video (such as the camera image) in the publishing client's UI.
otc_subscriber_callbacks.on_video_enabled() — Called with reason "quality" when the Subscriber determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):

Use the audio fallback API to dynamically prioritize audio in response to network quality.

For conceptual information, see the audio fallback overview.

Enabling and disabling audio-only fallback

To enable publisher audio fallback, call the otc_publisher_settings_set_publisher_audio_fallback_enabled() function.

Publisher audio fallback events

When publisher audio fallback is enabled, callback functions in the otc_publisher_callbacks struct are invoked for publisher audio fallback-related events:

otc_publisher_callbacks.on_video_disable_warning() — Called when the Publisher determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
otc_publisher_callbacks.on_video_disable_warning_lifted() — Called when the Publisher determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
otc_publisher_callbacks.on_video_disabled() — Called when the Publisher determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Publisher still displays the publisher video (such as the camera image) in the publishing client's UI.
otc_publisher_callbacks.on_video_enabled() — Called with reason "quality" when the Publisher determines that the stream quality has improved and outgoing video transport has been re-enabled. For example the following code handles the related events (so that you can provide your own user interface notifications):

Subscriber audio fallback events

When subscriber audio fallback is enabled, callback functions in the otc_subscriber_callbacks struct are invoked for subscriber audio fallback-related events:

otc_subscriber_callbacks.on_video_disable_warning() — Called when the Subscriber determines that the stream quality has degraded and the video will be disabled if the quality degrades more.
otc_subscriber_callbacks.on_video_disable_warning_lifted() — Called when the Subscriber determines that the stream quality has improved to the point at which the video being disabled is not an immediate risk.
otc_subscriber_callbacks.on_video_disabled() — Called when the Subscriber determines that the stream quality has degraded and the outgoing video transport has been disabled. Note: while the video is disabled, the Subscriber still displays the subscriber video (such as the camera image) in the publishing client's UI.
otc_subscriber_callbacks.on_video_enabled() — Called with reason "quality" when the Subscriber determines that the stream quality has improved and outgoing video transport has been re-enabled.

For example the following code handles the related events (so that you can provide your own user interface notifications):