Adjusting audio and video — Linux

You can make audio and video adjustments to published and subscribed streams:

Publishing audio or video only

You can toggle audio and video on or off, by calling the otc_publisher_set_publish_audio() and otc_publisher_set_publish_video() functions, passing in OTC_FALSE or OTC_FALSE as the publish_audio and publish_audio parameters. For example, the following code turns audio off:

otc_publisher_set_publish_audio(publisher, OTC_FALSE);

Publishing in a voice-only session

To set up a voice-only session, instantiate an otc_publisher_settings struct, call the otc_publisher_settings_set_video_track() function, passing in OTC_FALSE as the enabled parameter. Then use the otc_publisher_new_with_settings() function to create. For example, the following code creates a publisher for a voice-only session:


otc_publisher_settings publisher_settings = otc_publisher_settings_new();
otc_publisher_settings_set_video_track(publisher_settings, OTC_FALSE);
otc_publisher_callbacks publisher_callbacks = {0};
// Set callbacks for the publisher. Then:
otc_publisher publisher = otc_publisher_new_with_settings(publisher_callbacks, publisher_settings);

Using a custom video capturer

The otc_publisher_new() function includes a capturer parameter. If you set this to NULL, the SDK uses the default system camera as the source for the published video stream. However, you can use a custom video capturer for the publisher by defining a otc_video_capturer_callbacks struct and passing it in as the capturer parameter of the otc_publisher_new() function. The otc_video_capturer_callbacks struct includes function pointers to callback functions that are invoked upon events related to the publisher video, such as when the video capturer needs a new video frame.

To see an example, see the Custom Video Capturer sample of the vonage-linux-sdk-samples repo.

Mirroring the local display of a publisher's video

By default, the local renderer for published video mirrors the video. You can set the mirror_on_local_render property of the otc_video_capturer_settings instance obtained by calling the get_capture_settings(capturer, settings) function. Set it to true to have the video mirrored in the local renderer of the publisher. Set it to false to have it not be mirrored.

This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.

Using a custom audio driver

You can define a custom audio device, to supply audio to all publishers in the client and to handle the mixed audio stream from all streams the client subscribes to.

A custom audio device is represented by an otc_audio_device struct. And an otc_audio_device_callbacks struct includes function pointers to functions that act as the audio-related callbacks that the OpenTok Linux SDK invokes.

To see an example, see the Custom Audio Device sample in the vonage-linux-sdk-samples repo.

Setting the audio volume for a subscriber

You can individually set the audio volume for each subscriber by calling the otc_subscriber_set_audio_volume() function.

Subscribing to audio or video only

By default, a subscriber is initialized to subscribe to audio and video, if they are available. You can toggle audio on or off by calling the otc_subscriber_set_subscribe_to_audio(otc_subscriber *subscriber, otc_bool subscribe_to_audio) :

// audio off:
otc_subscriber_set_subscribe_to_video(subscriber, OTC_FALSE);
// audio on:
otc_subscriber_set_subscribe_to_video(subscriber, OTC_TRUE);

You can toggle video on or off by calling the otc_subscriber_set_subscribe_to_video(otc_subscriber *subscriber, otc_bool subscribe_to_video) :

// video off:
otc_subscriber_set_subscribe_to_video(subscriber, OTC_FALSE);
// video on:
otc_subscriber_set_subscribe_to_video(subscriber, OTC_TRUE);

Detecting whether a stream has audio or video

You can call the otc_stream_has_audio() and otc_stream_has_audio() member functions of an otc_stream instance to see if it has audio or video.

Tuning audio quality

To fine tune audio quality for a publisher stream the following methods can be used:

Audio bitrate

otc_publisher_set_max_audio_bitrate(otc_publisher* publisher, uint32_t bitrate);

The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, OpenTok automatically assigns an audio bitrate for the stream.

The following are recommended settings:

  • 8,000 - 12,000 for narrowband (NB) speech
  • 16,000 - 20,000 for wideband (WB) speech
  • 28,000 - 40,000 for full-band (FB) speech
  • 48,000 - 64,000 for full-band (FB) music

Disable Audio Processing

otc_publisher_settings_set_disable_audio_processing (otc_publisher_settings* settings, otc_bool disabled);

Whether to disable echo cancellation, automatic gain control, and noise suppression for the published audio. You may want to set this to true when publishing high-quality audio. The default value is false.

Automatic Gain Control

otc_publisher_settings_set_auto_gain_control(otc_publisher_settings* settings, otc_bool enabled);

Whether to enable automatic gain control for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing.

Echo Cancellation

otc_publisher_settings_set_echo_cancellation(otc_publisher_settings* settings, otc_bool enabled);

Whether to enable echo cancellation for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing

Noise Suppression

otc_publisher_settings_set_noise_suppression(otc_publisher_settings* settings, otc_bool enabled);

Whether to enable noise suppression for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you if you disable audio processing.

Stereo

otc_publisher_settings_set_stereo(otc_publisher_settings* settings, otc_bool enabled);

Whether to publish stereo audio. The default value is false.

Setting publisher video degradation preference

When bandwidth or CPU are limited, the video engine can reduce the frame rate and/or resolution of the publisher video to adjust to the limitations.

The degradation preference is a configuration setting that allows tweaking this behaviour.

Developers are responsible for defining and applying degradation preferences according to their use case and requirements.

It works as a preference, meaning that the video engine will try to follow the selected preference, but there're no guarantees on the result.

This preference can behave differently when using scalable video, where the video engine can decide to remove layers instead of degrading the resolution.

Content Hint and degradation preference serve different but related purposes: Content Hint describes the type of content being transmitted, while degradation preference controls the encoding strategy. When you set a Content Hint, the video engine automatically determines the appropriate degradation preference (for example, "text" automatically selects to maintain resolution). An explicitly set degradation preference will override this automatic selection.

You can set this value using any of the provided otc_degradation_preference values:

Not set

otc_publisher_set_degradation_preference(publisher, OTC_DEGRADATION_PREFERENCE_NOT_SET);

The default value. The video engine will decide the optimal degradation preference.

Maintain frame rate and resolution

otc_publisher_set_degradation_preference(publisher, OTC_DEGRADATION_PREFERENCE_MAINTAIN_FRAMERATE_AND_RESOLUTION);

The video engine will try to keep the frame rate steady and not reduce the resolution.

Maintain frame rate

otc_publisher_set_degradation_preference(publisher, OTC_DEGRADATION_PREFERENCE_MAINTAIN_FRAMERATE);

The video engine will try to keep the frame rate steady but might reduce resolution if necessary.

Maintain resolution

otc_publisher_set_degradation_preference(publisher, OTC_DEGRADATION_PREFERENCE_MAINTAIN_RESOLUTION);

The video engine will not reduce the resolution but might reduce the frame rate if necessary.

Balanced

otc_publisher_set_degradation_preference(publisher, OTC_DEGRADATION_PREFERENCE_BALANCED);

The video engine will try to reach a balance between reducing resolution and frame rate when necessary.