Filter und Effekte
Übersicht
Mit den Vonage Video Client SDKs können Sie bei der Veröffentlichung von Videos Filter und Effekte verwenden.
In dieser Anleitung wird das Thema behandelt:
- Veröffentlichung eines MediaStreamTrack-Objekts und Anwendung von Effekten
- Anwenden eines Filters zum Verwischen oder Ersetzen des Hintergrunds
- Anwendung des erweiterten Rauschunterdrückungsfilters
- Verwendung der Vonage Media Processor-Bibliothek zur Anwendung benutzerdefinierter Transformationen
Veröffentlichen eines MediaStreamTrack-Objekts und Anwenden von Effekten
Sie können eine audio MediaStream-Spur oder eine Video MediaStream-Spur als Audio- oder Videoquelle für einen veröffentlichten Stream verwenden. Mit dieser Funktion können Sie Filter und Effekte, wie z. B. Hintergrundunschärfe oder Hintergrundersetzung, auf das veröffentlichte Audio oder Video anwenden.
Sie können die OT.getUserMedia() Methode, um einen Verweis auf einen MediaStream zu erhalten, der die vom Benutzer ausgewählte Kamera verwendet. Sie können dann den Video-MediaStreamTrack, den Sie von diesem MediaStream-Objekt erhalten, als Quelle für ein Video-Element verwenden. Sie können dieses Video-Element dann zu einem HTML-Leinwandelement hinzufügen, Filter oder Effekte auf die Leinwand anwenden und das gefilterte Video-MediaStreamTrack-Objekt, das Sie von der Leinwand erhalten, als Videoquelle für einen veröffentlichten Stream verwenden. Ein Beispiel finden Sie im Stream-Filter-Beispiel opentok-web-samples repo auf GitHub.
Sie können die OT.getUserMedia() Methode, um einen Verweis auf einen MediaStream zu erhalten, der das vom Benutzer ausgewählte Mikrofon verwendet. Sie können dann den Audio-MediaStreamTrack, den Sie von diesem MediaStream-Objekt erhalten, als die audioSource beim Aufruf von OT.initPublisher(). Sie können dann eine AudioKontext Objekt und rufen dessen createMediaStreamSource() Objekt, wobei das MediaStream-Objekt übergeben wird, um eine MediaStreamAudioSourceNode Objekt. Sie können dann Audiofilter auf das MediaStreamAudioSourceNode-Objekt anwenden, was dazu führt, dass die Filter auf den veröffentlichten Stream angewendet werden.
Anwenden eines Hintergrund-Weichzeichnungs- oder Hintergrund-Ersetzungsfilters
Use the OTPublisher.setVideoTransformers() method to apply a video filter for published video. You can apply a background blur filter or a background image filter.
Important: In version 2.28.0+ of the Vonage Video React Native SDK, you need to install the Vonage Media Library separately from the Vonage Video React Native SDK. For details, see Vonage Media Library integration.
The following code applies a background blur filter to a publisher's video:
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setVideoTransformers([
{
name: 'BackgroundBlur',
properties: JSON.stringify({
radius: 'low',
}),
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
Note that the OTPublisher.setVideoTransformers() method takes an array of transformer objects. Each object has two properties:
A name property, set to either 'BackgroundBlur' or 'BackgroundReplacement'.
A properties property, defining options for the filter.
For details, see the documentation for the OTPublisher.setVideoTransformers() method.
The following code applies a background replacement filter to a publisher's video (supported on iOS only):
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setVideoTransformers([
{
name: 'BackgroundReplacement',
properties: JSON.stringify({
image_file_path: 'path/to/image',
}),
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
To remove video filters from the published stream, call the OTPublisher.setVideoTransformers() method and pass in an empty array.
Use the Publisher.applyVideoFilter() method to apply a video filter for the publisher. You can apply a background blur filter or a background image filter. A publisher can have only one (or zero) filters applied at one time. When you set a new filter, a previously set filter is removed.
The following code applies a background blur filter to a publisher's video:
The following code applies a background replacement filter to a publisher's video:
You can also apply a video filter by setting the videoFilter option when calling the OT.initPublisher() method:
Important:
- Video filters require adequate processor support. Even in supported browsers, transformers may not be stable when background processes limit available processing resources. See Client requirements.
- Note about Content Security Policies (CSP): If the
script-srcdirective is set, make surewasm-unsafe-evalis specified. Otherwise WebAssembly, required for background blur or replacement filters, is blocked from loading and executing on the page. - The background image is resized to match the dimensions of the publisher's video. For best results, use an image that has the same dimensions (or at least the same aspect ratio) as the published video. Set the
mirroroption tofalsewhen calling theOT.initPublisher()method to have the background image appear in the correct orientation (not mirrored) in the publisher's page.
For details, see the reference documentation for Publisher.applyVideoFilter().
This method is incompatible with the Publisher.setVideoMediaProcessorConnector() method. Calling this method after Publisher.setVideoMediaProcessorConnector(connector) returns a promise that is rejected with an error, with the name property of the error set to 'OT_NOT_SUPPORTED'. You can remove the connector by calling Publisher.setVideoMediaProcessorConnector(null).
You can also set the initial video filter by setting a videoFilter option when calling OT.initPublisher() method.
To clear the video filter, call the Publisher.clearVideoFilter() method.
Notes:
- Video filters are only supported in recent versions of Chrome, Electron, Opera, Samsung Internet, Edge, and WebView Android. They are not supported in other (non-Chromium-based) browsers or on iOS. You can check if the client supports this feature by calling the
OT.hasMediaProcessorSupport()method. - The Vonage Media Library includes transformers that provide more options than are provided with the background blur and background image filters provided by the
Publisher.applyVideoFilter()method. Also, you can use the Vonage Media Library to write your own custom audio and video transformers. See the Using the Vonage Media Processor developer guide.
Anwendung des erweiterten Rauschunterdrückungsfilters
Use the OTPublisher.setAudioTransformers() method to apply an audio transformer to published video. One transformer, the noise suppression filter, is supported.
Important: The noise suppression filter uses the Vonage Media Library. You need to install the Vonage Media Library separately from the Vonage Video React Native SDK. For details, see Vonage Media Library integration.
The following code applies a noise suppression filter to a publisher's video:
<OTPublisher
eventHandlers={{
streamCreated: () => {
this.publisher.setAudioTransformers([
{
name: 'NoiseSuppression',
properties: '',
}
])
}
}}
ref={instance => {
this.publisher = instance;
}}
/>
To remove audio transformers from the published stream, call the OTPublisher.setAudioTransformers() method and pass in an empty array.
Important: Audio transformers are resource-intensive and require devices with high processing power. It is recommended to only use these transformations on supported devices. See the following documentation:
Note: This is a beta feature.
Advanced noise suppression is an audio filter that greatly reduces the background noise around the publisher. This can be useful when a participant is in a crowded or noisy environment, like in a cafe, or if white noise is present, like from an HVAC system. It can work separately or in conjunction with the noiseSuppression property set when calling the OT.initPublisher method.
Use the Publisher.applyAudioFilter() method to apply an audio filter for the publisher. You can apply an advanced noise suppression filter. A publisher can have only one (or zero) filters applied at one time. When you set a new filter, any previously set filter is removed. This is a beta feature.
The following code applies an advanced noise suppression audio filter to a publisher's audio:
publisher.applyAudioFilter({
type: 'advancedNoiseSuppression',
});
You can also apply the advanced noise suppression audio filter by setting the audioFilter option when calling the OT.initPublisher() method:
const publisher = OT.initPublisher(null, {
audioFilter: {
type: 'advancedNoiseSuppression',
},
});
Important: Audio filters require adequate processor support. Even in supported browsers, transformers may not be stable when background processes limit available processing resources. See Client requirements.
For details, see the reference documentation for Publisher.applyAudioFilter().
This method is incompatible with the Publisher.setAudioMediaProcessorConnector() method. Calling this method after Publisher.setAudioMediaProcessorConnector(connector) returns a promise that is rejected with an error, with the name property of the error set to 'OT_NOT_SUPPORTED'. You can remove the connector by calling Publisher.setAudioMediaProcessorConnector(null).
To clear the advanced noise suppression filter, call the Publisher.clearAudioFilter() method.
Notes:
- Audio filters are only supported in recent versions of Chrome, Electron, Opera, Samsung Internet, Edge, and WebView Android. They are not supported in other (non-Chromium-based) browsers or on iOS. You can check if the client supports this feature by calling the
OT.hasMediaProcessorSupport()method. - When using
Publisher.applyAudioFilter(), assets are downloaded from Vonage servers. If you prefer to self-host these assets, please refer to this blog post for information on hosting, and for using thePublisher.setAudioMediaProcessorConnector()method to apply the advanced noise suppression filter with self-hosted assets.
Verwenden der Vonage-Medienprozessorbibliothek zum Anwenden benutzerdefinierter Transformationen
Siehe die Verwendung des Vonage Medienprozessors Leitfaden für Entwickler.