){kind=small}
Now Receive Audio With the Vonage Video API Audio Connector
Time to read: 1 minute
There have already been a few blog posts already written about the Audio Connector feature in the Vonage Video API (“Enhance your Vonage Video Applications with Audio Connector”, “Video + AI: Live Translations With Audio Connector”, and “Video + AI: Configurable Audio Processing for Video Applications”). In each, you connect to a WebSocket server and then process the audio from the video call to do things like translation, transcription, and taking meeting notes.
By setting the bidirectional flag to true when you start the Audio Connector, you can send audio back into the video call. This opens up a world of new possibilities. Referring to the translation use case mentioned above that returned just translated text, it can now be translated into audio!
With Large Language Models and other AI tools, the audio returned can be generated as a response to the audio sent from the video call. For example, imagine a group of friends is planning a trip overseas. They can all get on a video call with an AI travel advisor at any time, where they can ask all their questions and discuss their ideas for however long they need. The AI travel advisor will be able to respond based on real-time data pulled from a multitude of sources. Having an audio conversation is a much better user experience. Once all the trip details have been decided, the AI travel advisor can then pass that information to a travel agent to handle completing the booking.
Want to try it out? Here is a Web sample application that is a basic video call that includes a WebSocket server that will echo back the audio from the video call. It is a good starter project to get a feel for how things work.
With the Audio Connector’s new bi-directional feature, you can now send and receive audio in real time, opening the door to translation, transcription, and AI-powered conversations. If you have any questions, feel free to reach out to us on the Community Slack.