The Vonage Video API makes it easy for developers to add video to their applications. If you don’t have very specific needs for your video conferencing application or need low-level control over the video elements, you can use the default UI that Vonage Video API provides, reducing complexity for your development team. However, if you are used to working with mediaStreams and handle your video elements or because it fits better with the design principles of how your video application is built, we've got you covered.

Prerequisites

1. A Vonage Video API account. Click Sign Up to create one if you don't have one already.

2. This blog post assumes you are familiar with Vonage Video API, so we’ll only focus on the specific needs to publish and subscribe to your video elements.

Custom video elements

To make it easy for developers to use the Vonage Video API, the client SDK creates a video element by default that you can attach to the DOM. However, you can provide your own HTML video element if you want further control. For this blog post, we will refer to this React application as an example of how you access the mediaStream and use it as srcObject for your custom video element. Feel free to clone it and give it a try.

Publisher

First of all, in our publisher properties, we need to disable the default UI, as we mention in the developer documentation. The publish function is defined in the publisher hook

//hooks/publisher.jsx 
async function publish(name, extraData) {
    try {
      if (!mSession.session) throw new Error('You are not connected to session');
      const options = {
        insertMode: 'append',
        name: name,
        resolution: '1280x720',
        publishAudio: user.defaultSettings.publishAudio,
        publishVideo: user.defaultSettings.publishVideo,
        audioSource: user.defaultSettings.audioSource,
        videoSource: user.defaultSettings.videoSource,
        insertDefaultUI: false,
        audioFallback: {
          publisher: true,
        },
      };
      const finalOptions = Object.assign({}, options, extraData);
      setPublisherOptions(finalOptions);
      console.log(finalOptions);
      const newPublisher = OT.initPublisher(null, finalOptions);

      publishAttempt(newPublisher, 1);
      publisher.current = newPublisher;
   
    } catch (err) {
      console.log(err.stack);
    }
  }

The logic to publish into the session is defined in the publishAttempt function. But for simplicity, we’re going to hardcode one single retry. Note that we are not passing any target element to the publish function since we want to access the underlying mediaStream and use it with our video element.

async function publishAttempt(publisher, attempt = 1, noRetry = true) {
    console.log(`Attempting to publish in ${attempt} try`);

    publisher.on('destroyed', handleDestroyed);
    publisher.on('streamDestroyed', handleStreamDestroyed);
  publisher.on('videoElementCreated', handleVideoElementCreated);

    const { retry, error } = await new Promise((resolve, reject) => {
      mSession.session.publish(publisher, (err) => {
        if (err && noRetry) {
          resolve({ retry: undefined, error: err });
        }
        if (err && attempt < 3) {
          resolve({ retry: true, error: err });
        }
        if (err && attempt > 3) {
          resolve({ retry: false, error: err });
        } else {
          resolve({ retry: false, error: undefined });
        }
      });
    });

    if (retry) {
      // Wait for 2 seconds before attempting to publish again
      await delay(2000 * attempt);
      await publishAttempt(publisher.current, attempt + 1);
    } else if (error) {
      if (noRetry) return;
      alert("Publish error");
      mSession.disconnect();
      setIsPublishing(false);
      publisher.current = null;
    } else {
      setIsPublishing(true);
      publisher.current = publisher;
    }
  }

Now, we need to listen to the videoElementCreated event that is dispatched on the publisher in this case. Now, instead of using the video element dispatched on the videoElementCreated event directly, we are going to access its mediaStream to feed it to our own video element. Check the implementation of the CustomPublisher component.

//Components/CustomPublisher
import React, { useEffect, useMemo, useState, useRef } from 'react';
function CustomPublisher({ mediaStream }) {
  const videoRef = useRef(null);
  useEffect(() => {
    if (mediaStream) {
      videoRef.current.srcObject = mediaStream;
    }
  }, [mediaStream]);
  return <video width="100%" ref="{videoRef}" autoplay="" playsinline="" muted=""></video>;
}

export default CustomPublisher;

We are adding the autoplay attribute, which will cause new streams assigned to the element to play automatically. The playsinline attribute allows video to play inline instead of only in full-screen. We’re also adding the muted attribute to avoid echo because the Vonage Video API will play audio through the video element created but not rendered on the DOM.

All we’re doing with our video element is populating the srcObject with the mediaStream provided by the videoElementCreated event. The following code shows how you get the mediaStream from the event listener.

//hooks/publisher.jsx

publisher.on('videoElementCreated', handleVideoElementCreated);
//hooks/publisher 

function handleVideoElementCreated({ element }) {
    const stream = element.srcObject;
    setPubStream(stream);
  }

Then in our Room page, we can just render our CustomPublisher component with the mediaStream as a prop. Note that mPublisher is just the import of the publisher() hook, and pubStream is the piece of state that contains our mediaStream

//Pages/Room.index.js
  {mPublisher.pubStream && <custompublisher mediastream="{mPublisher.pubStream}"></custompublisher>}

Subscriber

The approach we’re going to take to render the subscribers is similar but with one caveat. At the time of writing this blog post for customers using JS versions > 2.24.7, if you are working with mediaStreams on the subscriber side, you need to follow the steps outlined in this support article.

Firstly, we need to disable the default UI as we did for the Publisher. Then, in this case, we’re going to set 1 piece of state. We’re going to store the video element created by Vonage so that we can access the underlying mediaStream. You will see what we do with the video element in a bit. This logic is defined in the Session context

async function subscribe(stream, session, options = {}) {
    console.log('request to subscribe');
    if (session) {
      console.log(session);
      const finalOptions = Object.assign({}, options, {
        insertMode: 'append',
        width: '100%',
        height: '100%',
        insertDefaultUI: false,
      });
      const subscriber = session.current.subscribe(stream, null, finalOptions);
     subscriber.on('videoElementCreated', function (event) {
        const element = event.element;
        element.setAttribute('id', event.target.streamId);
        setSubscriberElements((prevStreams) => [...prevStreams, { element, subscriber     }]);
      });
      addSubscribers({ subscriber });
    }
  }

In the CustomSubscriber component, we’ll provide our video element and attach an event listener to the video element created by Vonage so that we can update our mediaStream when it changes, as explained in the article.

import React, { useEffect, useRef } from 'react';

function CustomSubscriber({ element }) {
  const videoRef = useRef(null);
  const mediaStream = element.srcObject;
  useEffect(() => {
    if (mediaStream && videoRef.current) {
      videoRef.current.srcObject = mediaStream;
      videoRef.current.setAttribute('id', element.id);
      const handleStreamChange = () => {
        if (mediaStream !== element.srcObject) {
          videoRef.current.srcObject = element.srcObject;
        }
      };

      element.addEventListener('play', handleStreamChange);

      return () => {
        element.removeEventListener('play', handleStreamChange);
      };
    }
  }, [element, mediaStream]);

  return <video width="100%" ref="{videoRef}" autoplay="" playsinline="" muted=""></video>;
}

export default CustomSubscriber;

Like in the Publisher case, we’re getting the mediaStream from the video element and attaching it to our video element. The difference is that we now need to add an event listener to understand when the mediaStream changes and update our video element with the new mediaStream if it changes.

At this point, we can render the subscribers in our Room Page.

{mSession.subscriberElements.length > 0 &&
            mSession.subscriberElements.map((element, index) => <customsubscriber key="{index}" element="{element}"></customsubscriber>)}

Note that you can also pass the subscriber object to the CustomSubscriber component to update your UI based on the state. For example, you can display a mic on/off overlay icon depending on the subscriber.stream.hasAudio property. This will allow you not to have to manipulate the DOM by inserting/removing HTML elements on top of the video element created by the Vonage Video API. Instead, you will render different states based on the different subscriber properties based on your application logic.

Important to note

Note that if you follow this approach and decide to use your video elements rather than the video elements created by the SDK, you won’t be able to use some features tied to the SDK's video element. You won’t be benefiting from publisher initials and backgroundImageUri as the logic for these features is built on top of the video element created by Vonage.

Conclusion

In conclusion, Vonage allows for a lot of flexibility when developing video conferencing applications. By default, Vonage creates a video element that you can attach to the DOM. However, we also support use cases where you need to provide your video element by accessing the mediaStreams of the video elements the Vonage Video API creates for you.

To get the latest news, connect with us on our Developer Community Slack, on X, previously known as Twitter, and at events.