Handle user input with ASR
A code snippet that shows how to handle a user input with Automatic Speech Recognition (ASR). The user says their input at the prompt and their input is acknowledged via a speech-to-text message.
Example
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
npm install express body-parser
Create a file named asr.js
and add the following code:
const app = require('express')()
const bodyParser = require('body-parser')
app.use(bodyParser.json())
Write the code
Add the following to asr.js
:
const onInboundCall = (request, response) => {
const ncco = [{
action: 'talk',
text: 'Please say something',
},
{
action: 'input',
type: ['speech'],
eventUrl: [`${request.protocol}://${request.get('host')}/webhooks/asr`],
speech: {
endOnSilence: 1,
language: "en-US",
uuid: [request.query.uuid]
}
}
]
response.json(ncco)
}
const onInput = (request, response) => {
const speech = request.body.speech.results[0].text
const ncco = [{
action: 'talk',
text: `You said ${speech}`
}]
response.json(ncco)
}
app
.get('/webhooks/answer', onInboundCall)
.post('/webhooks/asr', onInput)
app.listen(3000)
Run your code
Save this file to your machine and run it:
node asr.js
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Add the following to `build.gradle`:
compile 'com.vonage:client:6.2.0'
compile 'com.sparkjava:spark-core:2.7.2'
Write the code
Add the following to the main
method of the AsrInput
class:
/*
* Route to answer incoming calls.
*/
Route answerCallRoute = (req, res) -> {
TalkAction intro = TalkAction
.builder("Please say something")
.build();
SpeechSettings speechSettings = new SpeechSettings();
speechSettings.setLanguage(SpeechSettings.Language.ENGLISH_UNITED_STATES);
InputAction input = InputAction.builder()
.type(Collections.singletonList("speech"))
.eventUrl(String.format("%s://%s/webhooks/asr", req.scheme(), req.host()))
.speech(speechSettings)
.build();
res.type("application/json");
return new Ncco(intro, input).toJson();
};
/*
* Route which returns NCCO saying which word was recognized.
*/
Route speechInputRoute = (req, res) -> {
InputEvent event = InputEvent.fromJson(req.body());
TalkAction response = TalkAction.builder(String.format("You said %s, Goodbye.",
event.getSpeech().getResults().iterator().next().getText()
)).build();
res.type("application/json");
return new Ncco(response).toJson();
};
Spark.port(3000);
Spark.get("/webhooks/answer", answerCallRoute);
Spark.post("/webhooks/asr", speechInputRoute);
Run your code
We can use the application
plugin for Gradle to simplify the running of our application.
Update your build.gradle
with the following:
apply plugin: 'application'
mainClassName = project.hasProperty('main') ? project.getProperty('main') : ''
Run the following gradle
command to execute your application, replacing com.vonage.quickstart.voice
with the package containing AsrInput
:
gradle run -Pmain=com.vonage.quickstart.voice.AsrInput
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
Install-Package Vonage
Create a file named AsrController.cs
and add the following code:
using Vonage.Voice.AnswerWebhooks;
using Vonage.Voice.EventWebhooks;
using Vonage.Voice.Nccos;
using Vonage.Utility;
Write the code
Add the following to AsrController.cs
:
[HttpGet("[controller]/webhooks/answer")]
public IActionResult Answer()
{
var host = Request.Host.ToString();
//Uncomment the next line if using ngrok with --host-header option
//host = Request.Headers["X-Original-Host"];
var request = WebhookParser.ParseQuery<Answer>(Request.Query);
var eventUrl = $"{Request.Scheme}://{host}/webhooks/asr";
var speechSettings = new SpeechSettings { Language = "en-US", EndOnSilence = 1, Uuid = new[] { request.Uuid } };
var inputAction = new MultiInputAction { Speech = speechSettings, EventUrl = new[] { eventUrl } };
var talkAction = new TalkAction { Text = "Please speak now" };
var ncco = new Ncco(talkAction, inputAction);
return Ok(ncco.ToString());
}
[HttpPost("/webhooks/asr")]
public async Task<IActionResult> OnInput()
{
var input = await WebhookParser.ParseWebhookAsync<MultiInput>(Request.Body, Request.ContentType);
var talkAction = new TalkAction();
talkAction.Text = input.Speech.SpeechResults[0].Text;
var ncco = new Ncco(talkAction);
return Ok(ncco.ToString());
}
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
composer require slim/slim:^3.8
Write the code
Add the following to index.php
:
use Vonage\Voice\NCCO\Action\Talk;
use Vonage\Voice\NCCO\Action\Input;
use \Psr\Http\Message\ResponseInterface as Response;
use \Psr\Http\Message\ServerRequestInterface as Request;
require 'vendor/autoload.php';
$app = new \Slim\App;
$app->get('/webhooks/answer', function (Request $request, Response $response) {
$uri = $request->getUri();
$url = $uri->getScheme().'://'.$uri->getHost().':'.$uri->getPort().'/webhooks/asr';
$inputAction = new Input();
$inputAction
->setSpeechEndOnSilence(true)
->setSpeechLanguage('en-US')
->setEventWebhook(new Webhook($url))
;
$ncco = new NCCO();
$ncco
->addAction(new Talk('Please say something'))
->addAction($inputAction)
;
return $response->withJson($ncco->toArray());
});
$app->map(['GET', 'POST'], '/webhooks/asr', function (Request $request, Response $response) {
/** @var InputWebhook $input */
$input = Factory::createFromRequest($request);
$ncco = new NCCO();
$ncco->addAction(new Talk('You said ' . $input->getSpeech()['results'][0]['text']));
return $response->withJson($ncco->toArray());
});
$app->run();
Run your code
Save this file to your machine and run it:
php -t . -S localhost:3000
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
pip install Flask request jsonify
Create a file named handle-user-input-with-asr.py
and add the following code:
#!/usr/bin/env python3
from flask import Flask, request, jsonify
app = Flask(__name__)
Write the code
Add the following to handle-user-input-with-asr.py
:
@app.route("/webhooks/answer", methods=["POST", "GET"])
def answer_call():
ncco = [
{"action": "talk", "text": "Please, tell me something",},
{
"action": "input",
"type": ["speech"],
"eventUrl": [
"{host}{endpoint}".format(
host=request.host_url, endpoint="webhooks/asr"
)
],
"speech": {
"endOnSilence": 1,
"language": "en-US",
"uuid": [request.args.get("uuid")],
},
},
]
return jsonify(ncco)
@app.route("/webhooks/asr", methods=["POST", "GET"])
def answer_asr():
body = request.get_json()
if body is not None and "speech" in body:
speech = body["speech"]["results"][0]["text"]
ncco = [
{"action": "talk", "text": "Hello ,you said {speech}".format(speech=speech)}
]
else:
ncco = [{"action": "talk", "text": "Sorry, i don't undertand. Bye"}]
return jsonify(ncco)
if __name__ == "__main__":
app.run(port=3000)
Run your code
Save this file to your machine and run it:
python handle-user-input-with-asr.py
Prerequisites
A Vonage application contains the required configuration for your project. You can create an application using the Vonage CLI (see below) or via the dashboard. To learn more about applications see our Vonage concepts guide.
Install the CLI
npm install -g nexmo-cli
Create an application
Once you have the CLI installed you can use it to create a Vonage application. Run the following command and make a note of the application ID that it returns. This is the value to use in NEXMO_APPLICATION_ID
in the example below. It will also create private.key
in the current directory which you will need in the Initialize your dependencies step
Vonage needs to connect to your local machine to access your answer_url
. We recommend using ngrok to do this. Make sure to change demo.ngrok.io
in the examples below to your own ngrok URL.
nexmo app:create "User Input ASR Example" http://demo.ngrok.io/webhooks/answer http://demo.ngrok.io/webhooks/events --keyfile private.key
gem install sinatra sinatra-contrib rack-contrib
Create a file named answer-inbound-call-with-asr.rb
and add the following code:
require 'sinatra'
require 'sinatra/multi_route'
require 'rack/contrib'
use Rack::JSONBodyParser
Write the code
Add the following to answer-inbound-call-with-asr.rb
:
before do
content_type :json
end
route :get, :post, '/webhooks/answer' do
[
{
action: 'talk',
text: 'Please say something'
},
{
action: 'input',
type: [ 'speech' ],
eventUrl: ["#{request.base_url}/webhooks/asr"],
speech: {
endOnSilence: 1,
uuid: [params[:uuid]],
language: 'en-US'
}
}
].to_json
end
route :post, '/webhooks/asr' do
[{
action: 'talk',
text: "You said #{params["speech"]["results"][0]["text"]}"
}].to_json
end
route :post, '/webhooks/event' do
puts params
end
set :port, 3000
Run your code
Save this file to your machine and run it:
ruby answer-inbound-call-with-asr.rb
Try it out
Call your Vonage Number. When the call is answered you will be asked to say a message. When you are finished, you will then hear your message repeated back to you via speech-to-text.