actions on google - Asking while using SSML in Dialogflow webhook


Question: 

I am trying to build an Actions on Google Agent via DialogFlow and keep getting errors when trying to ask the user a question while including ssml.

I have built the agent on DialogFlow, have logic implemented using the fulfillment webhook (implemented via the node module dialogflow-fulfillment) and have been able to test on DialogFlow successfully using the test console on the right side DialogFlow.

I therefore hooked up the DialogFlow Integrations to Google Assistant.

I first tried unsuccessfully:

const client = new WebhookClient({ req, res });
let qToSnd = 'Hi <break time=\"500ms\"/> Can I help you?';
let conv = client.conv();
conv.ask(qToSnd);
client.add(conv);

The above would work (not give errors) but would result in the question being asked while speaking out the <break> tag.

I have also tried:

conv.ask(
  new Text({
    text: _stripTags(qToSnd),
    ssml: qToSnd
}));

However, when I test this using the Actions on Google simulator I get the error message:

[Agent] isn't responding right now. Try again soon.

Digging into the logs viewer shows the following error message:

MalformedResponse: ErrorId: ... Failed to parse Dialogflow response into AppResponse because of invalid platform response. : Could not find a RichResponse or SystemIntent in the platform response for agentId: ... and intentId: ...

My fulfillment API is returning:

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
            "text": "Hi - Can I help you?",
            "ssml": "Hi <break time=\"500ms\"/> Can I help you?"
          }
        ]
      }
    }
  }
}

I will appreciate any pointers in the right direction.




2 Answers: 

Looking at the JSON snippet for a simple response in the documentation, you should wrap your item in a simpleResponse element. Additionally, the keys you are using for text and audio responses are incorrect, and should be textToSpeech and displayText.

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
            "simpleResponse": {
              "textToSpeech": "Howdy, this is GeekNum. I can tell you fun facts about almost any number, my favorite is 42. What number do you have in mind?",
              "displayText": "Howdy! I can tell you fun facts about almost any number. What do you have in mind?"
            }
          }
        ]
      }
    }
  }
}
 

Inspired by @NickFelker's answer below and researching more into this topic, I was able to get the SSML working by making sure to add the <speak> tags. So this works:

const client = new WebhookClient({ req, res });
let qToSnd = 'Hi <break time=\"500ms\"/> Can I help you?';
let conv = client.conv();
conv.ask('<speak>' + qToSnd + '</speak>');
client.add(conv);

The fulfillment API returns:

{
  "payload": {
    "google": {
      "expectUserResponse": true,
      "richResponse": {
        "items": [
          {
              "simpleResponse": {
                  "textToSpeech": "<speak>Hi <break time=\"500ms\"/> Can I help you</speak>"
              }
          }
        ]
      }
    }
  }
}
 

More Articles


java - JFrame as a dialog

I want to customize the jframe and that frame should work like a joptionpane. which is relative with parent. can i do like this. if it is possible please any one help me.

Magento: Where does Magento 1.8 store payment method options in the database?

Where does Magento store payment method options in the database?I looked around and did not find it.

dialogflow - how to define 'App' statement in firestore database?

I have to define 'app' inorder to store user details from dialogflow to firestore databaseconst app = new App({request:request, response:response});inorder to use this line of code i need 'App' to be defined in firestore code,for which i have used this line,const App = require('firebase-functions')


java - How can I use the response from a JOptionPane?

This is the first time I have used a confirm box and I'd like some advice on how to use it please, I want to use the users input of "Yes or No" but not sure how to do it? If I wanted to reference the input from the JOptionPane in an if statement how would go about it?JOptionPane.showConfirmDialog(nu

actions on google - Request Sync always returns 404 : "Error: Requested entity was not found."

I'm having a hard time implementing requestSync. It always returns "error": { "code": 404, "message": "Requested entity was not found.", "status": "NOT_FOUND" }I use Node.js/Express for the backend. The linking/unlinking with the Google Home app work and my actions work as well. It's rea

azure - Microsoft Bot Framework testing connection now only results in internal service errors

Whenever I test my bot using the "test connection to your bot" feature, it only response with "internal service error" and a 500 status code. It didn't do this yesterday, and I havent changed anything since then.What more, the bot works fine when I run it on the bot framework emulator on my local ma


python - Callback URL error when setting up webhook for messenger

I'm trying to follow this tutorial to set a chatbot for messenger. I'm stuck on the webhook setup. I added the page token and verify token to heroku, but when I try to add the heroku URL as the callback URL I get The URL couldn't be validated. Callback verification failed with the following error

localization - Vaadin ConfirmDialog changing the title

I want to change the title "Confirm" of a ConfirmDialog to something in my language. I looked at this, but the code does not make sense to me since the show method in my vaadin addon is a static method and it appears to be impossible to show a dialog returned by the factory.I used this code to wrap

magento - How to get id of last purchase with multiple addresses?

I have a payment method that needs to be redirected to the page it like paypal, I can add a code block on the page of success but need to know if petido that was just completed was paid with my payment method, so I can redirect.Note: several people say it is not possible to redirect payments with mu

How to use API in actions-on-google

I have tried the following javascript code, using a railway API in aog for getting names of trains between 2 stations.train : function(conv,src,dst) {const options={ method:'GET', url : `https://api.railwayapi.com/v2/between/source/${src}/dest/${dst}/date/23-09-2018/apikey/<api_key>/`,