Share:
){kind=small}
Benjamin Aronov is a developer advocate at Vonage. He is a proven community builder with a background in Ruby on Rails. Benjamin enjoys the beaches of Tel Aviv which he calls home. His Tel Aviv base allows him to meet and learn from some of the world's best startup founders. Outside of tech, Benjamin loves traveling the world in search of the perfect pain au chocolat.
Smarter Voice Agents: Use Knowledge AI to Avoid AI Hallucinations
Time to read: 8 minutes
Do you remember the last time you used an instruction manual? Maybe it was for building some furniture or fixing a microwave, but I’m sure the experience left you confused at times. Instruction manuals are frustrating because you have to know how to search them for exactly the problem you are having.
However, the great thing about instruction manuals and similar FAQs is that they are full of detailed and valuable instructions for end consumers. If only there were a way to turn all that data into a delightful user experience. Now there is, with AI Studio Voice agents and the new Knowledge AI.
Imagine your customers being able to ask questions in their own words—no need for rigid, pre-programmed commands—and receiving instant, accurate answers. With the rise of Generative AI, creating such flexible voicebots has become easier than ever. With Knowledge AI, you can now train a voicebot to understand a wide range of human questions and scope its answers precisely from a predefined knowledge base. This means your customers will receive the correct information every time, with none of the noise.
This tutorial will teach you how to build an AI FAQ voicebot using a company’s website and training materials to answer any user question flexibly and accurately.
Visual metaphor illustrating the mission to convert the Toddler 911 user manual into an interactive voice bot using Knowledge AI.
Our mission in this article is to turn a company’s website and Instruction Manual into a VoiceBot. Just as you might call into a company to learn about basic information, you may also want to get help, like when following an Instruction Manual for a broken refrigerator or toaster. We will use Knowledge AI to automate this interaction and save your company valuable employee time.
What user manual will we turn into an AI FAQ voicebot? Well, I decided it would be fun to create a fictitious company with a fictitious instruction manual, all generated by LLMs of course! We’re going to create an AI Voice Agent for “Toddler 911!”. This company helps parents “operate” their children.
Check out the Toddler 911 website and instruction manual. We’ll come back to use this later. I thought the AI made some pretty terrific company graphics on each page!
You will need a Vonage Developer Account
You will need to reference or download the resources in the Toddler 911! GitHub repo
To complete this tutorial, you will need a Vonage API account. If you don’t have one already, you can sign up today and start building with free credit. Once you have an account, you can find your API Key and API Secret at the top of the Vonage API Dashboard.
Before building out the agent, we need to set up our Knowledge Base. This will be the repository that will guide our Gen AI’s answers. Navigate to the Knowledge AI tab in the AI Studio Dashboard.
Knowledge AI introduces two new concepts: Sources and Indexes. Sources are individual pieces of information and data. For instance, a source can be a URL (page on a website) or a file. Bundled together, sources create indices. The full volume of information from which agents can generate answers is called an index.
Let’s get started creating our first index.
Within the Knowledge AI tab, click on Add Source. This will open a form to add a new source.
Add a new knowledge source in Vonage AI Studio by uploading a local file or providing a URL.
You’ll notice that sources can be either a local file (TXT, HTML, or PDF) or a publicly accessible URL.
Add the links to the company website:
Landing Page URL, Source Name: Toddler 911 Landing Page
About Us URL, Source Name: Toddler 911 About Us
Products URL, Source Name: Toddler 911 Products
Now add the links for the training material. But don’t add the 2024 Bathroom Training Update. We’ll add that later.
Make a Snack URL, Source Name: Toddler 911 Training - Snack
Clean Up Toys URL, Source Name: Toddler 911 Training - Snack
Expanded view of the 'Toddler 911 FAQs' index showing all five selected content sources in Knowledge AI.
Now that we have a few sources, we’ll be able to ask more flexible questions. Let’s move on to creating an Index.
Click on the Index Tab in the left-hand panel. Then click on Add Index. First, give your Index a nice name like “Toddler 911 FAQs”. Then check all of your newly created sources and hit Save & Exit.
Source dashboard for AI Studio showing five uploaded website sources for the Toddler 911 project with total text size of 40.15KB.
You should now see an Index available.
Screenshot of the AI Studio index dashboard showing a single index called 'Toddler 911 FAQs' connected to five content sources.
Knowledge AI allows you to test Indexes before hooking them up to agents. To do so hover over an index and a play icon will appear. Click on the play symbol and this will open up the Index Tester.
Tooltip and UI showing that the 'Toddler 911 FAQs' index can be tested within the AI Studio interface.
You can now ask any questions you want and the test agent will retrieve answers from your Index. Try to ask questions from the company, as LLMs might need to be given extra context in certain situations. One case I found was that the LLM had a hard time distinguishing complex information. For example, on the About Us page, it says, "Founder &Creative Director of Imaginative Play:Dr. Seuss—site like, “What is Toddler 911?”, or questions from the training material like, “How do I get my child to clean his room?”. You can also try asking questions that are not in the Index to ensure the LLM isn’t hallucinating. For example, try to ask, “How do I get my child to start using the bathroom?”.
Thorough testing is important as LLMs might need to be given extra context in certain situations. One case I found was the LLM had a hard time distinguishing complex information. For example, on the About Us page, it says, "Founder & Creative Director of Imaginative Play: Dr. Seuss – A parent and entrepreneur passionate about creating solutions that empower families.”
However, when I ask, “Who is the founder?”, the LLM responds, “Unable to find relevant answer”. When I update the question to, “Who is the founder and creative director of imaginative play?” the LLM answers, “Dr. Seuss.”
We showed that the LLM is not yet capable of answering questions from the2024 Bathroom Training Update. So let’s add that as a new Source. You can either add it as a URL - Cloud PDF or download the PDF and upload it as a Local File. Update your Index to include the new Source.
Now try the tester again and ask our previous question, “How do I get my child to start using the bathroom?” This time you should get a nice answer back from the agent.
You can ask follow-up questions like, “What is an example of celebrating success?” or “What if my child had an accident?”. Testing aims to find holes and problem areas in your Index.
Test and refine your Knowledge AI responses by creating and querying an index based on uploaded sources.
Notice the little books symbol in the Knowledge AI response. You can click on it to see which Sources were used to generate the answer. You can use this to help tweak sources for an index to give better results.
Now that we have an Index that we can assign to our Generative AI agent, let’s build our agent!
To begin we’ll need to create a WhatsApp agent. You can open AI Studio from inside the Vonage Developer Dashboard and then follow the instructions found in the AI Studio documentation. There are three important options for our agent, select:
Type: Telephony
Template: Start From Scratch
Event: Inbound
Let's greet our users with a welcome message. Let’s use the Speak node. Since our company name is said as “Toddler Nine One One” instead of Ninehundred and eleven, we’ll need to make use of the SSML say-as tag:
<speak> Hello and welcome to Toddler <say-as interpret-as='digits'>911</say-as>!</speak>Next, we'll accept our user's response with a Collect Input node. We’ll store the response in a parameter called inquiry. Add a simple prompt, “How can I help you?”. Also, let’s add 2 for “number of retries” and a retry prompt of “Sorry I didn’t get that. Can you ask again?”.
Unless otherwise specified, all parameters in this blog post will use the @sys.any entity. For more information about entities (different types of parameters) in AI Studio, please read the documentation.
As this article is focused on Knowledge AI, we won't bother with classifying our User Intent or creating multiple flows. We can continue directly with the Q&A node from the conversation section.
Animated walkthrough showing how to add a FAQ node in AI Studio that uses Knowledge AI to answer user questions.
The configuration for our node will be:
Index: Toddler 911 FAQs (or your Index name)
User Input Parameter: inquiry
Response Parameter: ai_response
Waiting time: 2 seconds
The maximum time that AI Studio will wait for a response. If the response time exceeds 2 seconds then the Failed exit path will be triggered.
Now that we’ve hooked up our node to get FAQ answers, let’s send them to our users! The whole flow will look like this:
Overview of the full FAQ conversation flow in AI Studio, including AI-generated responses, fallback handling, and repeat prompts.
This may look kind of complicated but most of it is error handling and improving the user experience. The FAQ node creates 3 exit points: $ai_response (success), Don’t Know, and Failed. These exit points provide us with some error handling depending on the result we get back.
To build it out:
For $ai_response exit point:Use a Speak node and have it simply send the $ai_response parameter. Connect our $ai_response exit point to this node.
For Don’t Know exit point:Use a Speak node and have it send an error message. Something like Sorry, I don't know how to answer that. Let's try a different question.” Connect our Don’t Know exit point to this node.
For Failed exit point:Use a Speak node and have it say something like, “Sorry, something went wrong. Let's try again.”. Connect our Failed exit point to this node.
We want to allow the user to ask multiple questions of our voicebot.
To build it out:
Create a Collect Input node called Collect Repeat that will save a new Parameter called repeat. Just like before let’s have a prompt: “Would you like to ask another question?”. We’ll also give 2 retries and a retry prompt of, “Sorry I didn't get that. Please try again.”
Use a Condition Node which will check against repeat. Create one condition to check if repeat is equal to yes, and another to check if repeat is equal to no.
Connect the true condition again to the Collect Inquiry node, the first collect input node we used which stores the value for inquiry
Connect the false condition to a Speak node form before, which will thank the user for using the agent:
<speak> Thank you for using Toddler <say-as interpret-as='digits'>911</say-as>!</speak>5. Connect the Thank You message node to an End Call node.
👏Give yourself a round of applause! You did it! You built your first Generative AI Voice agent. Now let’s test using the built-in tester.
Did you receive the phone call? Were you able to ask questions about all the different features of Toddler 911!? Did you learn about Dr. Seuss and Big Bird? Now that you’ve made the agent work, how would you improve it? Maybe we could create separate flows for the company FAQs and the instruction manual using Intents and Classification.
However, your agent behaves, let us know! Join the Vonage Developer Community Slack and tell me what you're building in the #ai-studio channel. Or find us on X, formerly known as Twitter.
Share:
Benjamin Aronov is a developer advocate at Vonage. He is a proven community builder with a background in Ruby on Rails. Benjamin enjoys the beaches of Tel Aviv which he calls home. His Tel Aviv base allows him to meet and learn from some of the world's best startup founders. Outside of tech, Benjamin loves traveling the world in search of the perfect pain au chocolat.