Integration with IBM Watson Assistant

Blueworx Voice Response can connect to an IBM Watson Assistant using CCXML using the Watson Assistant v2 API.

Using IBM Watson Assistant with BVR

Integrating a BVR CCXML application with Watson Assistant is achieved using a dialogstart with the src field specifying the IBM Watson Assistant service URL and the parameters field specifying other information required to connect to the Watson Assistant. Alternatively, the Watson Assistant can be determined by VIRTUAL_ASSISTANT_WATSON Call Features by setting the src parameter to "use_call_features", in which case none of the other parameters (such as watson_api_key) are necessary

In order for an application to use IBM Watson Assistant, the application must have a call feature for TTS and a call feature for STT attached to it. See here for details: BAM Command Line Utility Call Features Panel

Along with the IBM Watson Assistant service URL to be used for the src parameter, the following CCXML variables must be set up for use with the dialogstart:

watson_assistant_id The alias for the bot to connect to. At the time of writing, an alias can be set up for a Watson Assistant bot in the Aliases section of the Settings tab in the IBM Cloud Web UI.
watson_api_key The API key for the Watson Assistant service.

The following optional parameters may be supplied

bargein Whether or not to allow bargein during audio prompts. If not specified, this will default to true.
input_detection_mode Sets the method BVR uses to determine whether speech has started (which affects when prompts are barged in to and when the max speech timer starts). Valid options are NOISE and FIRST_INTERIM_RESULT. NOISE is where BVR considers input to have started if initial_speech_duration milliseconds of noise has been detected. FIRST_INTERIM_RESULT is where BVR considers input to have started when a result is returned from the STT engine. NOISE is faster, but can be triggered by unexpected non-speech sounds. The default is NOISE.
initial_speech_duration Determines the amount of speech (in milliseconds) that is considered enough noise to be indicate the start of input. Used when input detection mode is set to NOISE.
no_input_timeout Sets the no input timeout (in milliseconds). If not specified, the no input timeout defaults to 30 seconds. The no input timer is started after any text has finished rendering (regardless of the bargein parameter). If the no input is triggered, BVR will send a message to Watson Assistant with empty input text and sets the custom context variable bw_no_inputto true
max_speech_duration Sets the maximum amount of speech to detect. The timer starts when initial speech has been detected. If the timer fires before a final result comes back from the STT engine, the STT is stopped and any result is sent to Watson Assistant.
stt_confidence_threshold Sets the confidence threshold to use when getting an STT response. If the STT result is below the confidence threshold, BVR will send a message to Watson Assistant with empty input text and sets the custom context variable bw_no_match to true
initial_input_text If specified, BVR will send the text value of this variable to Watson Assistant when it first starts the dialog, which will behave as if the words were spoken to start with. If this is not specified, Watson Assistant will await voice input.
request_timeout The amount of time to wait before an HTTP request to the Watson Assistant API is considered a failure. This is a human readable time (e.g. "20 seconds"). If no time unit is specified, seconds will be assumed.

Additional more complex parameters can be supplied in the hints

watson_custom_context_vars An object containing key/value pairs to send as context variables to the Watson Assistant
stt_parms An object containing key/value pairs to send to the STT engine

Once these variables have been setup, the IBM Watson Assistant dialog can be started with a dialogstart like the following:

<dialogstart src="watson_service_url" type="'application/watson-assistant'" connectionid="evt.connectionid" parameters="watson_assistant_id watson_api_key"/>
            

Alternatively, to use VIRTUAL_ASSISTANT_WATSON Call Feature definitions:

<dialogstart src="'use_call_features'" type="'application/watson-assistant'" connectionid="evt.connectionid"/>
            

The provided sample application has an example of defining and using those variables.

The IBM Watson Assistant dialog finishes when the Watson Assistant bw_action context variable is set to an action that ends the dialog, such as "end_dialog" or "transfer"

Sending custom context variables from BVR to Watson Assistant

BVR sets some context variables by default. These are:

Name Sent on Description
bw_DNIS First message The "DNIS" or called number (the "number", or user-part of the SIP URI that was used to call BVR)
bw_ANI First message The ANI or calling number (the "number", or user-part of the SIP URI that called into BVR)
bw_ccx_session_id First message The CCXML session ID that is being used to connect to Watson Assistant
bw_confidence Every message The confidence value returned from the STT engine
bw_no_input Every message A boolean value. If set to true this indicates that no input was received
bw_no_match Every message A boolean value. If set to true this indicates that the input received by the STT engine was below the confidence threshold (set by bw_stt_confidence_threshold)

It is also possible to send custom context variables on the first message sent to the Watson Assistant session. These can be supplied as an object called "watson_custom_context_vars" in supplied hints when starting the dialog.

      <!-- Create the hints object -->
      <var name="watson_hints" expr="new Object()"/>
      <!-- Create the custom context variabled object that will contain all custom context variables we want to send to Watson Assistant -->
      <var name="watson_custom_context_obj" expr="new Object()"/>
      <!-- Setup some custom context variables -->
      <assign name="watson_custom_context_obj.my_variable" expr="'some value'"/>
      <assign name="watson_custom_context_obj.my_other_variable" expr="'some other value'"/>
      <!-- Set these custom context variables to the hints object under the name watson_custom_context_vars -->
      <assign name="watson_hints.watson_custom_context_vars" expr="watson_custom_context_obj"/>

      <!-- Now supply these hints when starting the dialog -->
      <dialogstart src="watson_service_url" type="'application/watson-assistant'" connectionid="evt.connectionid" parameters="watson_assistant_id watson_api_key" hints="watson_hints"/>
         

Using context variables to control BVR

It is possible to trigger certain actions on BVR by setting context variables in your Watson Assistant application. These context variables can be set at any time on a dialog. For further information on setting context variables in Watson Assistant dialogs, please refer to the IBM Watson Assistant documentation.

The bw_action context variable

end_dialog Ends the current Watson Assistant dialog, triggering a CCXML dialog.exit transition
transfer Triggers the dialog.transfer CCXML transition and ends the dialog. It is possible to set a transfer target by setting the session variable "bw_transfer_target". Note: If setting "bw_transfer_target" to a SIP URI in a Watson Assistant dialog, it is important to note that the "@" character is reserved and must be escaped with "\\". For example, "sip:test\\@test.com". Further information on transferring in CCXML can be found on the CCXML Call Transfer and Bridging page.
bw_bargein Sets bargein for this one prompt. After the prompt, bargein will be set back to the default set by the bargein parameter when the dialog started.
bw_no_input_timeout Sets no input timeout in milliseconds. After the prompt, the no input timeout will be set back to the default set by the no_input_timeout parameter when the dialog started.
bw_max_speech_duration Sets the max speech duration in milliseconds. After the prompt, the max speech duration will be set back to the default set by the max_speech_duration parameter when the dialog started.
bw_stt_confidence_threshold Sets the confidence threshold in milliseconds. After the prompt, the confidence threshold will be set back to the default set by the stt_confidence_threshold parameter when the dialog started.

Watson Assistant Exit Values

When a Watson Assistant dialog has finished, some values will be set to the "watson" property on the dialog.exit event values evt.values.watson. Within this object are the following values:

user_defined A map containing user defined values that were set in the Watson Assistant session
full_output A map containing a representation of the entirety of the output from the last message received from Watson Assistant.

Any variable names within these structures with a space character will have that space automatically replaced with an underscore '_' character

If the exit values contain any arrays, they will be presented as a map which contains a "count" property to determine the number of items in the array. Each item in the array will be a property of the map where the key is the index (starting from 0)

As an example, if the user_defined data from Watson Assistant contains an array called "custom data" with 2 entries, there will be the following values:

evt.values.watson.user_defined.custom_data.count The number of items in the array - in this case, the value for this will be 2
evt.values.watson.user_defined.custom_data.0 The first item in the array
evt.values.watson.user_defined.custom_data.1 The second item in the array

It is possible to loop through an array in CCXML. Here is an example of some Javascript code that can be added to a CCXML document in the <script> tag and invoked by calling printArray(arrayObject). This script relies on the existence of the printObject function

function printArray(toLog) {
       var output = "";

       if (toLog == null) {
          return "Cannot turn null object into an array";
       }
       var count = toLog.count;
       if (count == null) {
          return "This object is not an array";
       }
       //Total now has the last index value for the list of attributes in toPrint
       output += "Array has " + count + " items\n";
       var i;
       for (i = 0; i &lt; count; i++) {
          output += "\nItem " + i + ": " + toLog[i];
       }
       return output;
    }
         

Example hints usage

The hints structure can be used to pass more complex objects to the Watson Assistant or to the STT engine. Below is an example of a hints structure that sets the context variable "my_variable" and sets the "end_of_phrase_silence_time" parameter on the STT engine. Note that the STT parms are platform specific. In this case, we are using the IBM STT cloud service and setting the "end_of_phrase_silence_time" to 0.4 seconds. For all available parameters please refer to the IBM STT Websockets API documentation. Note that URL parameters and message body parameters can both be used in the stt_parms structure for IBM STT. If the model parameter is set in stt_parms it will override the model in the call feature definition.

 
<!-- Create the hints object -->
<var name="watson_hints" expr="new Object()"/>

<!-- Create the custom context object -->
<var name="customcontext" expr="new Object()"/>

<!-- Set the "my_variable" parameter in the custom context object -->
<assign name="customcontext.my_variable" expr="'test value'"/>
                  
<!-- Assign the custom context object to the hints -->
<assign name="watson_hints.watson_custom_context_vars" expr="customcontext"/>

<!-- Create the STT parameters object -->
<var name="stt_parms" expr="new Object()"/>
                  
<!-- Set the "end_of_phrase_silence_time" parameter to 0.4 seconds in the STT parameters -->
<assign name="stt_parms.end_of_phrase_silence_time" expr="'0.4'"/>
                  
<!-- Assign the stt parameters to the hints -->
<assign name="watson_hints.stt_parms" expr="stt_parms"/>
                  
                  
<!-- Supply the hints on the dialogstart -->
<dialogstart src="watson_service_url" type="'application/watson-assistant'" connectionid="evt.connectionid" parameters="watson_assistant_id watson_api_key" hints="watson_hints"/>
         

Sample Applications

We have provided sample applications that describe the integration and can be used to connect to IBM Watson Assistant. The sample applications are installed on the BVR server in the following location:

/opt/blueworx/vr/sample/IBMWatsonAssistant
         

Debugging IBM Watson Assistant applications

Watson Assistant applications have a single trace component: WATSON_AST