Integration with Google Dialogflow

Using Google Dialogflow with BVR

Integrating a BVR CCXML application with Google Dialogflow is achieved using a dialogstart with the src field specifying a path to the Google Dialogflow project's JSON credentials. Alternatively, the application can get the Dialogflow definition from VIRTUAL_ASSISTANT_DIALOGFLOW_ES, VIRTUAL_ASSISTANT_DIALOGFLOW_CX or VIRTUAL_ASSISTANT_NU_WORKFORCE call features by setting the src to "use_call_features"

The application MAY use external TTS and STT engines if desired by attaching external TTS and STT call features to the application. If there are no TTS or STT call features specified or external_tts/external_stt parameters are set to false, Dialogflow will use its own internal TTS and STT capabilities. See here for details: BAM Command Line Utility Call Features Panel

Along with the Google Dialogflow JSON credentials path to be used for the src parameter, the following CCXML variables may optionally be set up for use with the dialogstart:

The following optional parameters may be supplied

Additional more complex parameters can be supplied in the hints

For more on how to configure STT parameters, please see Dialogflow Virtual Assistant Specifics

Once these variables have been setup, the Google Dialogflow dialog can be started with a dialogstart like the following:

<dialogstart src="dialogflow_credentials_json" type="'application/google-dialogflow'" connectionid="evt.connectionid" parameters="list of parameter variable names" hints="hints_obj"/>
            

Or alternatively, to use VIRTUAL_ASSISTANT_DIALOGFLOW_ES, VIRTUAL_ASSISTANT_DIALOGFLOW_CX, or VIRTUAL_ASSISTANT_NU_WORKFORCE Call Feature definitions:

<dialogstart src="'use_call_features'" type="'application/google-dialogflow'" connectionid="evt.connectionid" parameters="list of parameter variable names" hints="hints_obj"/>
            

The provided sample application has an example of defining and using those variables.

The Google Dialogflow dialog finishes when Dialogflow returns a JSON payload containing the bw_action variable, and it is set to an action that ends the dialog, such as "end_dialog" or "transfer"

​

Setting initial input contexts

It is possible to set the initial Context(s) when starting a Dialogflow session. This is an Object containing the Context name as the key, and an Object containing parameters relating to the context as the body

The only mandatory parameter for the context object to contain is the lifespanCount, however it is also possible to attach parameters to the input context if desired

            
<!-- Create the hints object -->
<var name="dialogflow_hints" expr="new Object()"/>
<!-- Create the context map that will contain all input contexts we want to send to Dialogflow -->
<var name="dialogflow_context_map" expr="new Object()"/>

<!-- Create a context -->
<var name="my_input_context" expr="new Object()"/>
<assign name="my_input_context.lifespanCount" expr="'2'"/>
<!-- Optionally, you can add parameters to the context -->
<var name="my_input_context_parms" expr="new Object()"/>
<assign name="my_input_context_parms.custom_parameter" expr="'custom_parm_value'"/>
<assign name="my_input_context.parameters" expr="my_input_context_parms"/>

<!-- Add the context to the contexts map -->
<assign name="dialogflow_context_map.input_context_name" expr="my_input_context"/>
         
<!-- Set these custom context variables to the hints object under the name dialogflow_custom_context_vars -->
<assign name="dialogflow_hints.context" expr="dialogflow_context_map"/>

<!-- Now supply these hints when starting the dialog -->
<dialogstart src="dialogflow_service_url" type="'application/google-dialogflow'" connectionid="evt.connectionid" hints="dialogflow_hints"/> 
         

​

Setting TTS parameters

Just like with any VoiceXML started by CCXML, you can override your Text To Speech parameters for a specific VoiceXML session - see here for details.

Here's a dialogflow specific example:

            
<!-- Create the hints object -->
<var name="dialogflow_hints" expr="new Object()"/>
<!-- Create the tts_params object that will contain all TTS params -->
<var name="dialogflow_tts_params_obj" expr="new Object()"/>
<!-- Create the google object that will contain all Google TTS specific params -->
<var name="dialogflow_google_obj" expr="new Object()"/>
<!-- Setup some custom google variables -->
<assign name="dialogflow_google_obj.my_variable" expr="'some value'"/>
<assign name="dialogflow_google_obj.my_other_variable" expr="'some other value'"/>
<assign name="dialogflow_google_obj.my_number_variable" expr="9"/>
<assign name="dialogflow_google_obj.my_boolean_variable" expr="true"/>
<!-- Set the Google TTS parameters to the TTS object -->
<assign name="dialogflow_tts_params_obj.google" expr="dialogflow_google_obj"/>
<!-- Set the TTS params object to the hints object -->
<assign name="dialogflow_hints.google" expr="dialogflow_tts_params_obj"/>

<!-- Now supply these hints when starting the dialog -->
<dialogstart src="dialogflow_service_url" type="'application/google-dialogflow'" connectionid="evt.connectionid" hints="dialogflow_hints"/> 
         

Setting the initial payload

An initial JSON payload is sent on the first message sent to Dialogflow

BVR sets some variables in a payload by default. These can be overridden in custom payload variables. The defaults are:

It is also possible to include custom variables in the initial payload. These can be supplied as an object called "payload" in supplied hints when starting the dialog. Note that these will be merged with the variables supplied by default.

            
<!-- Create the hints object -->
<var name="dialogflow_hints" expr="new Object()"/>
<!-- Create the payload object that will contain all custom payload variables we want to send to Dialogflow -->
<var name="dialogflow_payload_obj" expr="new Object()"/>
<!-- Setup some custom payload variables -->
<assign name="dialogflow_payload_obj.my_variable" expr="'some value'"/>
<assign name="dialogflow_payload_obj.my_other_variable" expr="'some other value'"/>
<assign name="dialogflow_payload_obj.my_number_variable" expr="9"/>
<assign name="dialogflow_payload_obj.my_boolean_variable" expr="true"/>
<!-- Set these custom payload variables to the hints object under the name dialogflow_custom_payload_vars -->
<assign name="dialogflow_hints.payload" expr="dialogflow_payload_obj"/>

<!-- Now supply these hints when starting the dialog -->
<dialogstart src="dialogflow_service_url" type="'application/google-dialogflow'" connectionid="evt.connectionid" hints="dialogflow_hints"/> 
         

​

Using custom payloads to control BVR

It is possible to trigger certain actions on BVR by setting a custom payload response in your Dialogflow project. This response must be in JSON format and can be set in the "Responses" section of the intent in the Dialogflow console. Note that if multiple payloads are supplied, BVR will merge these internally and treat them as a single payload, so ensure that multiple payloads don't contain conflicting bw_* variables.

The custom payload variables

The bw_action payload variable

Example of the custom payload format

            
{
   "bw_action": "end_dialog"
} 
         

DTMF detection and Dialogflow

To enable DTMF detection the bw_dtmf_config custom payload must be set to a JSON object with the following parameters. Once the set of prompts has finished the DTMF detection will be turned off. If any DTMFs are detected that match the conditions specified in the bw_dtmf_config this is reported to DialogFlow as a DTMF input.

Example of enabling DTMF detection in a custom payload format

{
  "bw_dtmf_config": {
    "term_char": "#",
    "term_timeout": "1s",
    "max_digits": 4,
    "min_digits": 2,
    "inter_digit_timeout": "5s"
  },
  "bw_no_input_timeout": 60000
} 
         

Dialogflow Exit Values

When a Dialogflow dialog has finished, some values will be set to the "dialogflow" property on the dialog.exit event values evt.values.dialogflow. Within this object are the following values:

Any variable names within these structures with a space character will have that space automatically replaced with an underscore '_' character

If the exit values contain any arrays, they will be presented as a map which contains a "count" property to determine the number of items in the array. Each item in the array will be a property of the map where the key is the index (starting from 0)

As an example, if the user_defined data from Dialogflow contains an array called "custom data" with 2 entries, there will be the following values:

It is possible to loop through an array in CCXML. Here is an example of some Javascript code that can be added to a CCXML document in the <script> tag and invoked by calling printArray(arrayObject). This script relies on the existence of the printObject function

function printArray(toLog) {
   var output = "";

   if (toLog == null) {
      return "Cannot turn null object into an array";
   }
   var count = toLog.count;
   if (count == null) {
      return "This object is not an array";
   }
   //Total now has the last index value for the list of attributes in toPrint
   output += "Array has " + count + " items\n";
   var i;
   for (i = 0; i < count; i++) {
      output += "\nItem " + i + ": " + toLog[i];
   }
   return output;
} 

​

Sample Applications

We have provided sample applications that describe the integration and can be used to connect to Google Dialogflow. The sample applications are installed on the BVR server in the following location:

/opt/blueworx/vr/sample/GoogleDialogflow/GoogleDialogflowSample.ccxml 

Debugging Google Dialogflow applications

Dialogflow applications have a single trace component: G_DIALOGFLOW