Initializing capture

In this example, the app uses a custom video capturer to publish random pixels (white noise). This is done simply to illustrate the basic principles of setting up a custom video capturer.

In the main VonageVideoManager, after calling session.publish(publisher, error: &error) to initiate publishing of an audio-video stream, the videoCapture property of the OTPublisher object is set to an instance of BasicVideoCapturer:

publisher.videoCapture = BasicVideoCapturer()

BasicVideoCapturer is a custom class that implements the OTVideoCapture protocol (defined in the Vonage iOS SDK). This protocol lets you define a custom video capturer to be used by an OpenTok publisher.

The initCapture method initializes capture settings to be used by the custom video capturer. In this sample's custom implementation of OTVideoCapture (OTKBasicVideoCapturer), the initCapture method sets properties of the videoFormat property:

func initCapture() {
    let format = OTVideoFormat()
    format.pixelFormat = .ARGB
    format.bytesPerRow = [NSNumber(value: imageWidth * 4)]
    format.imageHeight = imageHeight
    format.imageWidth = imageWidth
    
    self.videoFormat = format
}

The OTVideoFormat class is defined by the Vonage iOS SDK. In this sample code, the format of the video capturer is set to use ARGB as the pixel format, with a specific number of bytes per row, height, and width.

The videoCaptureConsumer property sets an OTVideoCaptureConsumer object that the video consumer uses to transmit video frames to the publisher's stream. In BasicVideoCapturer, this property is synthesized from the protocol:

var videoCaptureConsumer: OTVideoCaptureConsumer?

Starting capture

The start method is called when a publisher starts capturing video to send as a stream to the Vonage session. This will occur after the Session.publish(_:error:) method is called. In the BasicVideoCapturer, this method triggers produceFrame on a background queue:

//
//  returns:
//  - a negative value for error
//  - 0 value when all is OK
//
func start() -> Int32 {
    self.captureStarted = true
    
    // Start the frame loop on a background queue
    DispatchQueue.global(qos: .background).asyncAfter(deadline: timerInterval) { [weak self] in
        self?.produceFrame()
    }
    
    return 0
}

Frame customization

The produceFrame method generates an OTVideoFrame object that represents a frame of video. In this case, the frame contains random pixels filling the defined height and width:

private func produceFrame() {
    guard let format = self.videoFormat else { return }
    
    let frame = OTVideoFrame(format: format)
    
    // Allocate buffer for image data
    let bufferSize = Int(imageWidth * imageHeight * 4)
    let buffer = UnsafeMutablePointer<UInt8>.allocate(capacity: bufferSize)
    
    // Generate random pixels
    for i in stride(from: 0, to: bufferSize, by: 4) {
        buffer[i]     = UInt8.random(in: 0...255) // A
        buffer[i + 1] = UInt8.random(in: 0...255) // R
        buffer[i + 2] = UInt8.random(in: 0...255) // G
        buffer[i + 3] = UInt8.random(in: 0...255) // B
    }
    
    // Pass pointer to Vonage
    var planePointer: UnsafeMutablePointer<UInt8> = buffer
    withUnsafeMutablePointer(to: &planePointer) { pointerToPlanePointer in
        frame.setPlanesWithPointers(UnsafeMutableRawPointer(pointerToPlanePointer).assumingMemoryBound(to: UnsafeMutablePointer<UInt8>.self), numPlanes: 1)
    }
    
    // Send frame to consumer
    self.videoCaptureConsumer?.consumeFrame(frame)
    
    buffer.deallocate()
    
    if self.captureStarted {
        DispatchQueue.global(qos: .background).asyncAfter(deadline: timerInterval) { [weak self] in
            self?.produceFrame()
        }
    }
}

The method passes the frame to the consumeFrame method of the OTVideoCaptureConsumer. This causes the publisher to send the frame of data to the video stream in the session.