Custom Video Rendering (Android)

Overview

This tutorial walks through the steps required to make minor modifications to the video renderer used by a Subscriber object. You can also use the same techniques to modify the video renderer used by a Publisher object (though this example only illustrates a custom renderer for a subscriber).

Setting up your project

The code for this section is available in the Basic-Video-Renderer-Java project of the opentok-android-sdk-samples repo. If you haven't already, you'll need to clone the repo into a local directory. On the command line, run:

git clone git@github.com:opentok/opentok-android-sdk-samples.git

Open the Basic-Video-Renderer-Java project in Android Studio to follow along.

Exploring the code

In this example, the app uses a custom video renderer to display a inverted color version of the video.

InvertedColorsVideoRenderer is a custom class that extends the BaseVideoRenderer class (defined in the OpenTok Android SDK). The BaseVideoRenderer class lets you define a custom video renderer to be used by an OpenTok publisher or subscriber:

subscriber = new Subscriber.Builder(this, stream)
    .renderer(new InvertedColorsVideoRenderer(this))
    .build();

publisher = new Publisher.Builder(MainActivity.this)
    .renderer(new InvertedColorsVideoRenderer(MainActivity.this))
    .build();

The InvertedColorsVideoRenderer() constructor sets a renderer property to a GLSurfaceView object. The app uses this object to display the video using OpenGL ES 2.0. The renderer for this GLSurfaceView object is set to a MyRenderer object. MyRenderer is a custom class that extends GLSurfaceView.Renderer, and it is used to render the video to the GLSurfaceView object:

public InvertedColorsVideoRenderer(Context context) {
    view = new GLSurfaceView(context);
    view.setEGLContextClientVersion(2);

    renderer = new MyRenderer();
    view.setRenderer(renderer);

    view.setRenderMode(GLSurfaceView.RENDERMODE_WHEN_DIRTY);
}

The onFrame() method of the video renderer is inherited from the BaseVideoRenderer class. The BaseVideoRenderer.onFrame() method is called when the publisher (or subscriber) renders a video frame to the video renderer. The InvertedColorsVideoRenderer implementation of this method, it takes the frame's image buffer (YUV representation of the frame), passes it to the displayFrame method of the MyRenderer object and calls the requestRender() method of the GLSurfaceView object:

@Override
public void onFrame(Frame frame) {
    renderer.displayFrame(frame);
    view.requestRender();
}

To render the video frames, the renderer class uses OpenGL shaders. In this sample shader produces the inverted color effect, more precisely this is achieved by this line which is inside the fragmentShaderCode string:

"y=1.0-1.1643*(y-0.0625);\n"

Congratulations! You've finished the Custom Video Rendering Tutorial for Android.
You can continue to play with and adjust the code you've developed here, or check out the Next Steps below.

Next steps

When you're finished here, continue building and enhancing your OpenTok application with these helpful resources: