Integration with Google Dialogflow

Blueworx Voice Response can connect to an Google Dialogflow using CCXML.

Using Google Dialogflow with BVR

Integrating a BVR CCXML application with Google Dialogflow is achieved using a dialogstart with the src field specifying a path to the Google Dialogflow project's JSON credentials.

The application MAY use external TTS and STT engines if desired by attaching external TTS and STT call features to the application. If there are no TTS or STT call features specified or external_tts/external_stt parameters are set to false, Dialogflow will use its own internal TTS and STT capabilities. See here for details: BAM Command Line Utility Call Features Panel

Along with the Google Dialogflow JSON credentials path to be used for the src parameter, the following CCXML variables may optionally be set up for use with the dialogstart:

The following optional parameters may be supplied

external_tts Determines whether to attempt to use external TTS call features. If set to false, Dialogflow's built in TTS capabilities will always be used. If set to true and no matching external TTS call features are attached to the application, Dialogflow's built in TTS capabilities will be used. If set to true and matching external TTS call features are assigned to the application, they will be used.
external_stt Determines whether to attempt to use external STT call features. If set to false, Dialogflow's built in STT capabilities will always be used. If set to true and no matching external STT call features are attached to the application, Dialogflow's built in STT capabilities will be used. If set to true and matching external STT call features are assigned to the application, they will be used.
input_detection_mode Sets the method BVR uses to determine whether speech has started (which affects when prompts are barged in to and when the max speech timer starts). Valid options are NOISE and FIRST_INTERIM_RESULT. NOISE is where BVR considers input to have started if initial_speech_duration milliseconds of noise has been detected. FIRST_INTERIM_RESULT is where BVR considers input to have started when a result is returned from the STT engine. NOISE is faster, but can be triggered by unexpected non-speech sounds. The default is NOISE.
initial_speech_duration Determines the amount of speech (in milliseconds) that is considered enough noise to be indicate the start of input. Used when input detection mode is set to NOISE.
no_input_timeout Sets the no input timeout (in milliseconds). If not specified, the no input timeout defaults to 30 seconds. The no input timer is started after any text has finished rendering (regardless of the bargein parameter). If the no input is triggered, BVR will send a message to Dialogflow with empty input text and sets the custom context variable bw_no_inputto true
max_speech_duration Sets the maximum amount of speech to detect. The timer starts when initial speech has been detected. If the timer fires before a final result comes back from the STT engine, the STT is stopped and any result is sent to Dialogflow.
initial_input_text If specified, BVR will send the text value of this variable to Dialogflow when it first starts the dialog, which will behave as if the words were spoken to start with. If this is not specified, Dialogflow will await voice input.

Additional more complex parameters can be supplied in the hints

context An object containing Context objects to set the initial input context for Dialogflow. As these are not arbitrary variables, they must be in a specific format. See "Setting initial input contexts"
payload An object containing a custom initial payload to send to Dialogflow. See "Setting the initial payload"
stt_parms An object containing key/value pairs to send to the STT engine

Once these variables have been setup, the Google Dialogflow dialog can be started with a dialogstart like the following:

<dialogstart src="dialogflow_credentials_json" type="'application/google-dialogflow'" connectionid="evt.connectionid" parameters="list of parameter variable names" hints="hints_obj"/>
            

The provided sample application has an example of defining and using those variables.

The Google Dialogflow dialog finishes when Dialogflow returns a JSON payload containing the bw_action variable, and it is set to an action that ends the dialog, such as "end_dialog" or "transfer"

Setting initial input contexts

It is possible to set the initial Context(s) when starting a Dialogflow session. This is an Object containing the Context name as the key, and an Object containing parameters relating to the context as the body

The only mandatory parameter for the context object to contain is the lifespanCount, however it is also possible to attach parameters to the input context if desired

            
                <!-- Create the hints object -->
				<var name="dialogflow_hints" expr="new Object()"/>
				<!-- Create the context map that will contain all input contexts we want to send to Dialogflow -->
				<var name="dialogflow_context_map" expr="new Object()"/>

				<!-- Create a context -->
				<var name="my_input_context" expr="new Object()"/>
				<assign name="my_input_context.lifespanCount" expr="'2'"/>
				<!-- Optionally, you can add parameters to the context -->
				<var name="my_input_context_parms" expr="new Object()"/>
				<assign name="my_input_context_parms.custom_parameter" expr="'custom_parm_value'"/>
				<assign name="my_input_context.parameters" expr="my_input_context_parms"/>

				<!-- Add the context to the contexts map -->
				<assign name="dialogflow_context_map.input_context_name" expr="my_input_context"/>
				<!-- Set these custom context variables to the hints object under the name dialogflow_custom_context_vars -->

				<assign name="dialogflow_hints.context" expr="dialogflow_context_map"/>

				<!-- Now supply these hints when starting the dialog -->
				<dialogstart src="dialogflow_service_url" type="'application/google-dialogflow'" connectionid="evt.connectionid" hints="dialogflow_hints"/>
								  
         

Setting the initial payload

An initial JSON payload is sent on the first message sent to Dialogflow

BVR sets some variables in a payload by default. These can b e overridden in custom payload variables. The defaults are:

bw_DNIS The "DNIS" or called number (the "number", or user-part of the SIP URI that was used to call BVR)
bw_ANI The ANI or calling number (the "number", or user-part of the SIP URI that called into BVR)
bw_ccx_session_id The CCXML session ID that is being used to connect to Dialogflow

It is also possible to include custom variables in the initial payload. These can be supplied as an object called "payload" in supplied hints when starting the dialog. Note that these will be merged with the variables supplied by default.

            
                <!-- Create the hints object -->
				<var name="dialogflow_hints" expr="new Object()"/>
				<!-- Create the payload object that will contain all custom payload variables we want to send to Dialogflow -->
				<var name="dialogflow_payload_obj" expr="new Object()"/>
				<!-- Setup some custom payload variables -->
				<assign name="dialogflow_payload_obj.my_variable" expr="'some value'"/>
				<assign name="dialogflow_payload_obj.my_other_variable" expr="'some other value'"/>
				<assign name="dialogflow_payload_obj.my_number_variable" expr="9"/>
				<assign name="dialogflow_payload_obj.my_boolean_variable" expr="true"/>
				<!-- Set these custom payload variables to the hints object under the name dialogflow_custom_payload_vars -->
				<assign name="dialogflow_hints.payload" expr="dialogflow_payload_obj"/>

				<!-- Now supply these hints when starting the dialog -->
				<dialogstart src="dialogflow_service_url" type="'application/google-dialogflow'" connectionid="evt.connectionid" hints="dialogflow_hints"/>
							   
         

Using custom payloads to control BVR

It is possible to trigger certain actions on BVR by setting a custom payload response in your Dialogflow project. This response must be in JSON format and can be set in the "Responses" section of the intent in the Dialogflow console. Note that if multiple payloads are supplied, BVR will merge these internally and treat them as a single payload, so ensure that multiple payloads don't contain conflicting bw_* variables.

The bw_action payload variable

end_dialog Ends the current Dialogflow dialog, triggering a CCXML dialog.exit transition
transfer Triggers the dialog.transfer CCXML transition and ends the dialog. It is possible to set a transfer target by setting the session variable "bw_transfer_target". Further information on transferring in CCXML can be found on the CCXML Call Transfer and Bridging page.
bw_bargein Sets bargein for this one prompt. After the prompt, bargein will be set back to the default set by the bargein parameter when the dialog started.
bw_no_input_timeout Sets no input timeout in milliseconds. After the prompt, the no input timeout will be set back to the default set by the no_input_timeout parameter when the dialog started.
bw_max_speech_duration Sets the max speech duration in milliseconds. After the prompt, the max speech duration will be set back to the default set by the max_speech_duration parameter when the dialog started.

Example of the custom payload format

            
			 {
				"bw_action": "end_dialog"
			 }
         
         

Dialogflow Exit Values

When a Dialogflow dialog has finished, some values will be set to the "dialogflow" property on the dialog.exit event values evt.values.dialogflow. Within this object are the following values:

user_defined A map containing user defined values that were set in the last Dialogflow pyaload

Any variable names within these structures with a space character will have that space automatically replaced with an underscore '_' character

If the exit values contain any arrays, they will be presented as a map which contains a "count" property to determine the number of items in the array. Each item in the array will be a property of the map where the key is the index (starting from 0)

As an example, if the user_defined data from Dialogflow contains an array called "custom data" with 2 entries, there will be the following values:

evt.values.dialogflow.user_defined.custom_data.count The number of items in the array - in this case, the value for this will be 2
evt.values.dialogflow.user_defined.custom_data.0 The first item in the array
evt.values.dialogflow.user_defined.custom_data.1 The second item in the array

It is possible to loop through an array in CCXML. Here is an example of some Javascript code that can be added to a CCXML document in the <script> tag and invoked by calling printArray(arrayObject). This script relies on the existence of the printObject function

function printArray(toLog) {
       var output = "";

       if (toLog == null) {
          return "Cannot turn null object into an array";
       }
       var count = toLog.count;
       if (count == null) {
          return "This object is not an array";
       }
       //Total now has the last index value for the list of attributes in toPrint
       output += "Array has " + count + " items\n";
       var i;
       for (i = 0; i &lt; count; i++) {
          output += "\nItem " + i + ": " + toLog[i];
       }
       return output;
    }
         

Sample Applications

We have provided sample applications that describe the integration and can be used to connect to Google Dialogflow. The sample applications are installed on the BVR server in the following location:

/opt/blueworx/vr/sample/GoogleDialogflow/GoogleDialogflowSample.ccxml
         

Debugging Google Dialogflow applications

Dialogflow applications have a single trace component: G_DIALOGFLOW