Filtros y efectos
Visión general
Los SDK del cliente de video de Vonage te permiten usar filtros y efectos al publicar videos.
Este tutorial repasará:
- Publicación de un objeto MediaStreamTrack y aplicación de efectos
- Aplicar un filtro de desenfoque de fondo o de sustitución de fondo
- Aplicación del filtro avanzado de supresión de ruido
- Uso de la biblioteca Vonage Media Processor para aplicar transformaciones personalizadas
Publicación de un objeto MediaStreamTrack y aplicación de efectos
Puede utilizar un pista de audio MediaStream o un vídeo MediaStream pista como fuente de audio o vídeo para un flujo publicado. Con esta función, puedes aplicar filtros y efectos, como desenfoque de fondo o sustitución de fondo, al audio o vídeo publicado.
Puede utilizar la función OT.getUserMedia() para obtener una referencia a un MediaStream que utilice la cámara seleccionada por el usuario. A continuación, puede utilizar el MediaStreamTrack de vídeo obtenido de ese objeto MediaStream como origen de un elemento Video. A continuación, puede añadir ese elemento Video a un elemento HTML Canvas, aplicar filtros o efectos al lienzo y utilizar el objeto MediaStreamTrack de vídeo filtrado obtenido del lienzo como fuente de vídeo para un flujo publicado. Para ver un ejemplo, consulte la muestra Stream-Filter repositorio opentok-web-samples en GitHub.
Puede utilizar la función OT.getUserMedia() para obtener una referencia a un MediaStream que utilice el micrófono seleccionado por el usuario. A continuación, puede utilizar el MediaStreamTrack de audio obtenido a partir de ese objeto MediaStream como el método audioSource al llamar a OT.initPublisher(). A continuación, puede crear un AudioContext y llamar a su createMediaStreamSource() pasando el objeto MediaStream para crear un objeto MediaStreamAudioSourceNode . A continuación, puede aplicar filtros de audio al objeto MediaStreamAudioSourceNode, lo que hará que los filtros se apliquen al flujo publicado.
Aplicar un filtro de desenfoque de fondo o de sustitución de fondo
Use the OTPublisher.setVideoTransformers() method to apply a video filter for published video. You can apply a background blur filter or a background image filter.
Important: In version 2.28.0+ of the Vonage Video React Native SDK, you need to install the Vonage Media Library separately from the Vonage Video React Native SDK. For details, see Vonage Media Library integration.
The following code applies a background blur filter to a publisher's video:
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setVideoTransformers([
{
name: 'BackgroundBlur',
properties: JSON.stringify({
radius: 'low',
}),
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
Note that the OTPublisher.setVideoTransformers() method takes an array of transformer objects. Each object has two properties:
A name property, set to either 'BackgroundBlur' or 'BackgroundReplacement'.
A properties property, defining options for the filter.
For details, see the documentation for the OTPublisher.setVideoTransformers() method.
The following code applies a background replacement filter to a publisher's video (supported on iOS only):
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setVideoTransformers([
{
name: 'BackgroundReplacement',
properties: JSON.stringify({
image_file_path: 'path/to/image',
}),
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
To remove video filters from the published stream, call the OTPublisher.setVideoTransformers() method and pass in an empty array.
Use the Publisher.applyVideoFilter() method to apply a video filter for the publisher. You can apply a background blur filter or a background image filter. A publisher can have only one (or zero) filters applied at one time. When you set a new filter, a previously set filter is removed.
The following code applies a background blur filter to a publisher's video:
The following code applies a background replacement filter to a publisher's video:
You can also apply a video filter by setting the videoFilter option when calling the OT.initPublisher() method:
Important:
- Video filters require adequate processor support. Even in supported browsers, transformers may not be stable when background processes limit available processing resources. See Client requirements.
- Note about Content Security Policies (CSP): If the
script-srcdirective is set, make surewasm-unsafe-evalis specified. Otherwise WebAssembly, required for background blur or replacement filters, is blocked from loading and executing on the page. - The background image is resized to match the dimensions of the publisher's video. For best results, use an image that has the same dimensions (or at least the same aspect ratio) as the published video. Set the
mirroroption tofalsewhen calling theOT.initPublisher()method to have the background image appear in the correct orientation (not mirrored) in the publisher's page.
For details, see the reference documentation for Publisher.applyVideoFilter().
This method is incompatible with the Publisher.setVideoMediaProcessorConnector() method. Calling this method after Publisher.setVideoMediaProcessorConnector(connector) returns a promise that is rejected with an error, with the name property of the error set to 'OT_NOT_SUPPORTED'. You can remove the connector by calling Publisher.setVideoMediaProcessorConnector(null).
You can also set the initial video filter by setting a videoFilter option when calling OT.initPublisher() method.
To clear the video filter, call the Publisher.clearVideoFilter() method.
Notes:
- Video filters are only supported in recent versions of Chrome, Electron, Opera, Samsung Internet, Edge, and WebView Android. They are not supported in other (non-Chromium-based) browsers or on iOS. You can check if the client supports this feature by calling the
OT.hasMediaProcessorSupport()method. - The Vonage Media Library includes transformers that provide more options than are provided with the background blur and background image filters provided by the
Publisher.applyVideoFilter()method. Also, you can use the Vonage Media Library to write your own custom audio and video transformers. See the Using the Vonage Media Processor developer guide.
Aplicación del filtro avanzado de supresión de ruido
Use the OTPublisher.setAudioTransformers() method to apply an audio transformer to published video. One transformer, the noise suppression filter, is supported.
Important: The noise suppression filter uses the Vonage Media Library. You need to install the Vonage Media Library separately from the Vonage Video React Native SDK. For details, see Vonage Media Library integration.
The following code applies a noise suppression filter to a publisher's video:
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setAudioTransformers([
{
name: 'NoiseSuppression',
properties: '',
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
To remove audio transformers from the published stream, call the OTPublisher.setAudioTransformers() method and pass in an empty array.
Important: Audio transformers are resource-intensive and require devices with high processing power. It is recommended to only use these transformations on supported devices. See the following documentation:
Note: This is a beta feature.
Advanced noise suppression is an audio filter that greatly reduces the background noise around the publisher. This can be useful when a participant is in a crowded or noisy environment, like in a cafe, or if white noise is present, like from an HVAC system. It can work separately or in conjunction with the noiseSuppression property set when calling the OT.initPublisher method.
Use the Publisher.applyAudioFilter() method to apply an audio filter for the publisher. You can apply an advanced noise suppression filter. A publisher can have only one (or zero) filters applied at one time. When you set a new filter, any previously set filter is removed. This is a beta feature.
The following code applies an advanced noise suppression audio filter to a publisher's audio:
publisher.applyAudioFilter({
type: 'advancedNoiseSuppression',
});
You can also apply the advanced noise suppression audio filter by setting the audioFilter option when calling the OT.initPublisher() method:
const publisher = OT.initPublisher(null, {
audioFilter: {
type: 'advancedNoiseSuppression',
},
});
Important: Audio filters require adequate processor support. Even in supported browsers, transformers may not be stable when background processes limit available processing resources. See Client requirements.
For details, see the reference documentation for Publisher.applyAudioFilter().
This method is incompatible with the Publisher.setAudioMediaProcessorConnector() method. Calling this method after Publisher.setAudioMediaProcessorConnector(connector) returns a promise that is rejected with an error, with the name property of the error set to 'OT_NOT_SUPPORTED'. You can remove the connector by calling Publisher.setAudioMediaProcessorConnector(null).
To clear the advanced noise suppression filter, call the Publisher.clearAudioFilter() method.
Notes:
- Audio filters are only supported in recent versions of Chrome, Electron, Opera, Samsung Internet, Edge, and WebView Android. They are not supported in other (non-Chromium-based) browsers or on iOS. You can check if the client supports this feature by calling the
OT.hasMediaProcessorSupport()method. - When using
Publisher.applyAudioFilter(), assets are downloaded from Vonage servers. If you prefer to self-host these assets, please refer to this blog post for information on hosting, and for using thePublisher.setAudioMediaProcessorConnector()method to apply the advanced noise suppression filter with self-hosted assets.
Uso de la biblioteca de procesadores multimedia de Vonage para aplicar transformaciones personalizadas
Véase el Uso del procesador multimedia de Vonage guía del desarrollador.