Publisher Settings
Overview
The Vonage Video Client SDKs allow for participants to publish audio and video in a session. The Client SDKs allow for configuring the Publisher object based on your preferences and use case.
This how-to will go over:
- Publishing audio or video only
- Changing a publisher's video settings
- Changing a publisher's audio settings
Publishing Audio or Video Only
When you create a Publisher object, you can specify whether to initially publish only audio or only video. For example, the following code creates an audio-only Publisher:
To toggle audio and video on and off, set the publishAudio and publishVideo properties of the properties prop passed into the OTPublisher component:
By default, these are set to true (both audio and video are published).
Once you have created a Publisher object, you can toggle audio and video on or off, by calling the publishAudio() and publishVideo() methods (passing in a Boolean value). For example, the following code turns audio off:
When you toggle audio or video on or off, the Session object in each connected client dispatches a streamPropertyChanged event. There are also a number of user interface optimizations that you can make in a voice-only session. See the Voice guide.
Publishing in a voice-only session
To set up a voice-only session, set the videoSource property to null or false when you create each Publisher object in the session. For example, the following code creates a Publisher for a voice-only session:
To set up a voice-only session, call the videoTrack() method of the Publisher.Builder object, passing in false, when you instantiate each Publisher object in the session. For example, the following code creates a publisher for a voice-only session:
When you set the videoSource property to null, the publishing client does not request access to the camera, and no video is published.
Publishing audio only for Publishers with low bandwidth
Publishers can be configured to disable video, keeping audio enabled, in low bandwidth situations. Video publishing will resume when the Publisher's bandwidth improves.
For more information, see the Audio fallback developer guide.
Changing a Publisher’s Video Settings
Setting the Resolution and Frame Rate for a Video
You can set the frame rate and resolution for a publisher's stream by setting a resolution property of the options you pass into the OT.initPublisher() method. See Setting the video resolution of a stream and Setting the frame rate of a stream.
To set a recommended video resolution for a published stream, set the resolution property of the properties prop of the OTPublisher component:
This resolution property is a string, defining the desired resolution of the video. The format of the string is "widthxheight", where the width and height are represented in pixels. Valid values are "1920x1080", "1280x720", "640x480", and "320x240".
The default resolution for a stream (if you do not specify a resolution) is 640x480 pixels. If the client system cannot support the resolution you requested, the stream will use the next largest setting supported.
It is best to try to match the resolution to the size that the video will be displayed. If you are only displaying the video at 320x240 pixels then there is no point in streaming at 1280x720 or 1920x1080. Reducing the resolution can save bandwidth and reduce congestion and connection drops.
Note: See the 1080p developer guide for considerations about using 1080p resolution.
To set a recommended frame rate for a published stream, set the frameRate property of the properties prop of the OTPublisher component:
Set the value to the desired frame rate, in frames per second, of the video. Valid values are 30 (the default), 15, 7, and 1.
Note that in sessions that use the Vonage Video Media Router (sessions with the media mode set to routed), lowering the frame rate proportionally reduces the bandwidth the stream uses. However, in sessions that have the media mode set to relayed, lowering the frame rate does not reduce the stream's bandwidth.
You can set the resolution and frame rate for a publisher's stream by configuring the VideoCapturer object you use when you instantiate the Publisher object.
There a a number of ways to instantiate a VideoCapturer object. One is to call the OpenTok.VideoCapturer.EnumerateDevices() method to obtain a list of available video devices. Then use one of CreateVideoCapturer() methods of the device you select to instantiate a VideoCapturer object. For example, the following code creates a Publisher that uses the default system video capturer with the resolution set to 720p (VideoCapturer.Resolution.High) and the frame rate set to 30 frames per second:
If the selected video device does not support the selected resolution or frame rate, the video capturer will use the closest available settings.
Note that in sessions that use the Vonage Video Media Router (sessions with the media mode set to routed), lowering the frame rate proportionally reduces the bandwidth the stream uses. However, in sessions that have the media mode set to relayed, lowering the frame rate does not reduce the stream's bandwidth.
Switching the Camera Used by a Publisher
You can switch the video input device (camera) used as the video source for a Publisher by calling the Publisher.cycleVideo() or Publisher.setVideoSource() method:
The Publisher.cycleVideo() method lets you cycle through the available video sources (cameras). For example, on a mobile device you can switch between the front and back camera. The method returns a promise that resolves when the operation completes successfully. The promise resolves with an object that has a deviceId property set to the device ID of the camera used:
If there is an error, the promise is rejected. This can occur in the following conditions:
- The user denied access to the video input device.
- The publisher is not using a camera video source. This method does not work for a screen-sharing publisher, a publisher that uses a MediaStreamTrack source, or a publisher that does not include a video track (an audio-only publisher).
- There are no video input devices (cameras) available.
- There was an error acquiring video from the video input device.
Note that this method only works for a publisher that is using a camera video source.
Publisher.setVideoSource()
The Publisher.setVideoSource() method lets you pass in the device ID of the new video source (camera). The method returns a promise that resolves when the operation completes:
The following will result in errors:
- If the
videoSourceIdparameter is not a string or the device ID for a valid video input device, the promise will reject with an error with thenameproperty set to'OT_INVALID_VIDEO_SOURCE'. - If the publisher does not currently use a camera input, the promise will reject with an error with the
nameproperty set to'OT_SET_VIDEO_SOURCE_FAILURE'.
You can use the OT.getDevices() method to enumerate the available video input devices (and get their device IDs).
You can use the Publisher.getVideoSource() method to get the current video source and its device ID (if it is a video input device).
You can have the publisher use the rear-facing camera of the device by setting a properties prop of the OTPublisher component and setting the cameraPosition property of that object to "back":
You can cycle between cameras used to publish a stream by calling the cycleCamera() method of the Publisher object:
If you are using a custom video capturer, use the BaseVideoCapturer.CaptureSwitch interface to define the behavior of the Publisher.cycleCamera() method.
When the camera used by the Publisher changes, the changedCamera(Publisher publisher, int newCameraId)) method of the Publisher.CameraListener object is called.
When the camera used to publish a stream changes, the publisher delegate sends an OTPublisher publisher(_:didChangeCameraPosition:) message:
When the camera used to publish a stream changes, the publisher delegate sends an [OTPublisher publisher:didChangeCameraPosition:] message:
Mirroring the Local Display of a Publisher’s Video
You can set the mirror property of the options passed into the OT.initPublisher() method to have the publisher's locally rendered video mirrored (true) or not (false). By default, video is mirrored for a publisher that has a camera video source, and not mirrored for a screen-sharing video.
This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.
You can set the mirrorInLocalRender property of the CaptureSettings object returned by calling the getCaptureSettings() method of the capturer property of a PublisherKit object. Set this to true to have the publisher's locally rendered video mirrored or to false to not have it mirrored. You may want to mirror video when using a front-facing camera.
You can also set the mirrorInLocalRender property of the CaptureSettings object when creating a custom video capturer
This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.
The built-in capturer of a Publisher has the locally rendered video mirrored for when using a front-facing camera, but not when using a rear-facing camera.
You can set the MirrorOnLocalRender property of the VideoCaptureSettings object to have the publisher's locally rendered video mirrored (true) or not (false). Set this property in the implementation of the GetCaptureSettings() method of the custom video capturer for the Publisher object.
This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.
By default, the local renderer for published video mirrors the video. You can set the mirror_on_local_render property of the otc_video_capturer_settings instance obtained by calling the get_capture_settings(capturer, settings) function. Set it to true to have the video mirrored in the local renderer of the publisher. Set it to false to have it not be mirrored.
This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.
By default, the local renderer for published video mirrors the video. You can set the mirror_on_local_render property of the otc_video_capturer_settings instance obtained by calling the get_capture_settings(capturer, settings) function.
Set it to true to have the video mirrored in the local renderer of the publisher. Set it to false to have it not be mirrored.
This setting only affects the rendered video in the publisher's client application. It has no effect on the video in subscribing clients.
Changing a Publisher’s Audio Settings
Switching the Audio Source Used by a Publisher
You can switch the microphone or MediaStreamTrack object used as the audio source for a Publisher by calling the setAudioSource() method of the Publisher object.
Pass a device ID for a microphone or an audio MediaStreamTrack object into the Publisher.setAudioSource() method. The method returns a Promise that is rejected on error (see the reference documentation for setAudioSource()).
For example, the following code shows you how to implement a cycleMicrophone() function that cycles through the microphones available:
The Publisher.setAudioSource() method only works for a publisher that has an audio source. If you set audioSource to null (or false) when calling OT.initPublisher(), you cannot later add an audio source to the publisher.
The Publisher.getAudioSource() method returns the MediaStreamTrack object used as the current audio input source for the publisher.
The OT.getDevices() method enumerates the audio and video input devices available to the browser.
You can change the audio input source (microphone) used by publishers. Call the static AudioDevice.EnumerateInputAudioDevices() method to get a list of AudioDevice.InputAudioDevice objects, representing available input devices. Call the AudioDevice.SetInputAudioDevice(audioInput) method to set the audio input device used by publishers.
Detecting When a Publisher Switches Audio Input Devices (Web)
By default, the SDK automatically handles audio input device switching for any Publisher object when an audio input device is added or removed, unless audio input device management has been disabled. See Disabling default audio input device management if you wish to disable automatic audio input device switching.
The Publisher object dispatches an audioInputDeviceChanged event when the SDK automatically changes the audio input. This event will be dispatched when a new audio input device is added, the current audio input device is removed, or when the Publisher's operating system switches audio input devices. The dispatched event has a device property that contains information about the audio input device. You may want to let your users know their microphone has changed:
Switching the Audio Output Used by a Publisher
You can switch the audio output device (a speaker or headphones) used to play audio from all publishers and subscribers (in all Vonage Video sessions in the browser).
The OT.getAudioOutputDevices() method enumerates the audio and video input devices available to the browser.
The OT.getActiveAudioOutputDevice() method identifies the currently active audio output device.
Use the OT.setAudioOutputDevice() method to set the audio output device.
For example, the following code shows you how to implement a cycleAudioOutput() function that cycles through the available audio output devices:
By default, the Android SDK uses the device loudspeaker (instead of the headset speaker) for playing audio. This is preferable for apps that include both video and audio. However, in a voice-only session, it is preferable to have the audio played back using the headset speaker. You can have the app do this by making the following call:
Tuning Audio Quality
The OT.initPublisher() method includes options for tuning audio quality. This lets you publish streams that use high-quality (or lower quality) audio:
audioBitrate (Number) — The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio).
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
If you do not set this option, OpenTok.js automatically sets the audio bitrate for the stream.
Currently, this setting is not supported in streams published in Firefox.
autoGainControl (Boolean) — Whether to enable automatic gain control for the published audio. You may want to set this to false when publishing high-quality audio (by setting the audioBitrate property of the OT.initPublisher() options). The default value is true. This setting is ignored if you set disableAudioProcessing to false (which disables echo cancellation, automatic gain control, and noise suppression for the published stream). You can also utilize the advancedNoiseSuppression audio filter, which uses the Vonage Media library (see this topic).
disableAudioProcessing (Boolean) — Whether to disable echo cancellation, automatic gain control, and noise suppression for the published audio. You may want to set this to true when publishing high-quality audio (by setting the audioBitrate property of the OT.initPublisher() options). The default value is false.
echoCancellation (Boolean) — Whether to enable echo cancellation for the published audio. You may want to set this to false when publishing high-quality audio (by setting the audioBitrate property of the OT.initPublisher() options). The default value is true. This setting is ignored if you set disableAudioProcessing to false (which disables echo cancellation, automatic gain control, and noise suppression for the published stream).
Note: Some browsers (such as Chrome 73+) do not support echo cancellation for stereo audio (see this Chrome issue report).
enableStereo (Boolean) — Whether to publish stereo audio. The default value is false.
noiseSuppression (Boolean) — Whether to enable noise suppression for the published audio. You may want to set this to false when publishing high-quality audio (by setting the audioBitrate property of the OT.initPublisher() options). The default value is true. This setting is ignored if you set disableAudioProcessing to false (which disables echo cancellation, automatic gain control, and noise suppression for the published stream).
To set the audio bitrate for a publisher's audio stream, set the audioBitrate property of the properties prop passed into the OTPublisher component:
The audioBitrate setting is the desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio).
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
To fine tune audio quality for a publisher stream the following approaches can be used:
You adjust the audio bitrate for a publisher stream by calling the audioBitrate() method of the PublisherKit.Builder object you use to instantiate a Publisher.
The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, OpenTok automatically assigns an audio bitrate for the stream.
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
To fine tune audio quality for a publisher stream the following approaches can be used:
Audio bitrate as a publisher settings
The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, Vonage automatically assigns an audio bitrate for the stream.
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
To fine tune audio quality for a publisher stream the following approaches can be used:
Audio bitrate as a publisher settings
The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, Vonage automatically assigns an audio bitrate for the stream.
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
The Publisher.Builder class includes properties for tuning audio quality. This lets you publish streams that use high-quality (or lower quality) audio:
AudioBitrate (Number) — The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio).
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
If you do not set this option, Voange automatically sets the audio bitrate for the stream.
AutoGainControl (Boolean) — Whether to enable automatic gain control for the published audio. You may want to set this to false when publishing high-quality audio (by setting the AutoGainControl property of the Publisher.Builder). The default value is true. This setting is ignored if you set DisableAudioProcessing to true (which disables echo cancellation, automatic gain control, and noise suppression for the published stream).
DisableAudioProcessing (Boolean) — Whether to disable echo cancellation, automatic gain control, and noise suppression for the published audio. You may want to set this to true when publishing high-quality audio (by setting the DisableAudioProcessing property of the Publisher.Builder). The default value is false.
EchoCancellation (Boolean) — Whether to enable echo cancellation for the published audio. You may want to set this to false when publishing high-quality audio (by setting the EchoCancellation property of the Publisher.Builder). The default value is true. This setting is ignored if you set DisableAudioProcessing to true (which disables echo cancellation, automatic gain control, and noise suppression for the published stream).
Stereo (Boolean) — Whether to publish stereo audio. The default value is false.
NoiseSuppression (Boolean) — Whether to enable noise suppression for the published audio. You may want to set this to false when publishing high-quality audio (by setting the NoiseSuppression property of the Publisher.Builder). The default value is true. This setting is ignored if you set DisableAudioProcessing to true (which disables echo cancellation, automatic gain control, and noise suppression for the published stream).
To fine tune audio quality for a publisher stream the following methods can be used:
Audio bitrate
otc_publisher_set_max_audio_bitrate(otc_publisher* publisher, uint32_t bitrate);
The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, Vonage automatically assigns an audio bitrate for the stream.
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
Disable Audio Processing
otc_publisher_settings_set_disable_audio_processing (otc_publisher_settings* settings, otc_bool disabled);
Whether to disable echo cancellation, automatic gain control, and noise suppression for the published audio. You may want to set this to true when publishing high-quality audio. The default value is false.
Automatic Gain Control
otc_publisher_settings_set_auto_gain_control(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable automatic gain control for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing.
Echo Cancellation
otc_publisher_settings_set_echo_cancellation(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable echo cancellation for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing
Noise Suppression
otc_publisher_settings_set_noise_suppression(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable noise suppression for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you if you disable audio processing.
Stereo
otc_publisher_settings_set_stereo(otc_publisher_settings* settings, otc_bool enabled);
Whether to publish stereo audio. The default value is false. For a practical example, see this sample implementation
You can individually set the audio volume for each subscriber by calling the otc_subscriber_set_audio_volume() function.
To fine tune audio quality for a publisher stream the following methods can be used:
Audio bitrate
otc_publisher_set_max_audio_bitrate(otc_publisher* publisher, uint32_t bitrate);
The desired bitrate for the published audio, in bits per second. The supported range of values is 6,000 - 510,000. (Invalid values are ignored.) Set this value to enable high-quality audio (or to reduce bandwidth usage with lower-quality audio). If you do not set this option, Vonage automatically assigns an audio bitrate for the stream.
The following are recommended settings:
- 8,000 - 12,000 for narrowband (NB) speech
- 16,000 - 20,000 for wideband (WB) speech
- 28,000 - 40,000 for full-band (FB) speech
- 48,000 - 64,000 for full-band (FB) music
Disable Audio Processing
otc_publisher_settings_set_disable_audio_processing (otc_publisher_settings* settings, otc_bool disabled);
Whether to disable echo cancellation, automatic gain control, and noise suppression for the published audio. You may want to set this to true when publishing high-quality audio. The default value is false.
Automatic Gain Control
otc_publisher_settings_set_auto_gain_control(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable automatic gain control for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing.
Echo Cancellation
otc_publisher_settings_set_echo_cancellation(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable echo cancellation for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you disable audio processing
Noise Suppression
otc_publisher_settings_set_noise_suppression(otc_publisher_settings* settings, otc_bool enabled);
Whether to enable noise suppression for the published audio. You may want to set this to false when publishing high-quality audio. The default value is true. This setting is ignored if you if you disable audio processing.
Stereo
otc_publisher_settings_set_stereo(otc_publisher_settings* settings, otc_bool enabled);
Whether to publish stereo audio. The default value is false.
Reducing Audio Bandwidth With Opus DTX
Opus DTX (Discontinuous Transmission) is an audio codec that can reduce the bandwidth usage when a participant is not speaking. This can be useful in large sessions with many audio participants.
You enable Opus DTX by setting the enableDtx property when initializing a Publisher. For more information, see this Vonage Video API knowledge base article.